Fruit and vegetable picking robot based on machine vision

Abstract

The utility model discloses a fruit and vegetable picking robot based on machine vision, which relates to the technical field of fruit and vegetable picking, solves the problems of high manufacturing cost, insufficient convenience and environmental adaptation limitation of the robot, and comprises a crawler frame, a vehicle body, a visual identification device, a multi-degree-of-freedom mechanical arm, a picking manipulator and a control device, the control device is used for controlling the walking action of the crawler frame, the folding and unfolding action of the multi-degree-of-freedom mechanical arm and the grabbing action of the picking mechanical arm and collecting visual information in real time through the visual recognition device. The fruit and vegetable picking robot has remarkable advantages in the picking aspect. The fruit and vegetable picking robot system can be freely moved to any position according to actual requirements, comprehensive picking of fruits and vegetables is ensured, configuration of the fruit and vegetable picking robot system is very flexible, the structure can be dynamically adjusted, the degree of freedom can be freely increased and decreased, flexible picking of the fruits and vegetables is completed, and the fruit and vegetable picking robot system can freely run in various complex terrains; comprising muddy and soft soil, ramps and areas with overgrown weeds, and work is convenient and fast.

Description

TECHNICAL FIELD

[0001] The utility model relates to the technical field of fruit and vegetable picking, and especially relates to a fruit and vegetable picking robot based on machine vision. BACKGROUND

[0002] In the middle of 1980s, based on the increasingly mature industrial robot technology, vision and graphics processing technology and artificial intelligence technology, many countries such as Europe and the United States, Japan and the like have successively launched a variety of fruit and vegetable picking robot research. In 1984, Kawamura of Kyoto University in Japan and others developed a 5-degree-of-freedom joint robot based on the research of tomato picking, which marked the birth of the first picking robot in the strict sense in Japan. Since the 1990s, scholars such as Naoki Kondo in Japan have done a lot of research in the field of agricultural robots, and the tomato picking robot developed by them in 1993 had a great influence at that time. In 1996, a cucumber picking robot applied to greenhouse operation was developed by the Institute of Agricultural and Environmental Engineering (IMAG) in the Netherlands, and the target crop was a high-stretching line winding type hanging growth. In October 2010, a team of the European Union led by Wageningen University began to develop a sweet pepper picking robot, which was a European Union Seventh Framework Program (FP7), and the robot included a picking mechanical arm, a guide rail compressor, a control circuit, an industrial computer, an end effector and a mobile carrying platform, etc. The team completed the final robot prototype and research in September 2014. The cucumber picking robot developed by Henten and others in the Netherlands adopts a clamping method to clamp the fruit, and then uses a high-voltage electrode to burn off the fruit stem, which is beneficial to prevent bacterial infection. The success rate of picking is about 80%, and the average time for picking a single cucumber is 45s.

[0003] China's research on agricultural picking robots began in the mid-1990s and has achieved some encouraging results. Li Wei's team at China Agricultural University developed a 4-DOF joint robot and a clamping-shearing integrated two-finger pneumatic end effector, equipped with a binocular vision system. The facility agriculture robot system developed by JIN Wang and Botian Robot includes fruit and vegetable picking robots, intelligent inspection robots, intelligent transportation robots, intelligent spraying robots, and cloud brain control platforms. The core team comes from China Agricultural University. Botian intelligent picking robot integrates artificial intelligence and multi-sensor technology, based on deep learning vision algorithm, and can autonomously complete fruit and vegetable maturity judgment. The six-axis collaborative robot and the flexible picking hand claw have flexible motion and high-precision control capabilities, and can move and pick with precise speed and force. Botian intelligent inspection robot has intelligent environmental perception and autonomous obstacle avoidance, multi-terrain operation and positioning navigation functions. Laser navigation, automatic turning, and automatic up-and-down track meet the needs of various application scenarios such as facility road and track. Based on RGB-Deep depth stereo vision technology, it can real-time statistics of mature and semi-mature fruit and vegetable yield, and data sharing through Botian's self-developed cloud brain intelligent control platform. China's picking robot technology started late, and with the disappearance of the demographic dividend, the labor shortage problem has quickly become a bottleneck restricting the development of agriculture, especially the labor-intensive fruit and vegetable industry. Picking robot technology has gone from being a forward-looking research to a practical demand.

[0004] Existing automated robot technology has made significant achievements in terms of function type, innovation, and safety, meeting user requirements for robot functionality, safety, and other aspects. However, there are obvious shortcomings in terms of practicality and convenience in fruit and vegetable picking, with advantages and disadvantages mainly in the following aspects:

[0005] Advantages

[0006] Improved production efficiency: Robots can perform picking operations for a long time, significantly improving production efficiency and work speed. In addition, automated robots can be accurately positioned and accurately controlled.

[0007] Improved safety: Robots can replace humans in dangerous and difficult-to-work environments, monitor environmental abnormalities, and reduce workplace safety risks and hazards.

[0008] High flexibility: Robots usually have high flexibility, can adapt to different production needs and working environments. Through programming or artificial intelligence, robots can learn new skills and complete tasks to meet changing market demands.

[0009] Disadvantages

[0010] Too high cost: the existing automated robot function is comprehensive but most of them are too high in cost, which is a great economic pressure for small and medium-sized farmers, limiting the popularization of technology.

[0011] Low sensitivity: most automated robot devices are large in size, and it is not convenient for traditional automated robots to move in the complex environment of orchards.

[0012] Limited adaptability: the adaptability of robotic arms and gripping devices to different crops is poor, which can easily cause damage to fruits, affecting picking efficiency and fruit quality. Practical new content

[0013] With the continuous development of automation technology, robots are widely used in various fields around the world. However, there are some problems in the practicality and convenience of automated robots in daily life, including:

[0014] Too high cost: traditional robots are mostly suitable for engineering applications and are usually equipped with high-precision sensors and high-quality materials, which makes their cost high, but for small and medium-sized farmers, the economic pressure is great, limiting the popularization of technology.

[0015] Insufficient convenience: traditional automated robot devices are large in size and are not convenient to move in small orchard environments, making it difficult to accurately pick fruits and vegetables.

[0016] Environmental adaptability limitation: most robots use a single algorithm for a specific function, which makes the robot susceptible to algorithm limitations. When the environment changes, the algorithm is affected, and the function and work of the robot are also affected, such as the visual SLAM algorithm which is easily affected by surrounding environmental features, light intensity, and running mode.

[0017] In view of the above problems of high cost, insufficient convenience and environmental adaptability limitation of robots, the purpose of the present utility model is to provide a fruit and vegetable picking robot based on machine vision.

[0018] In order to achieve the above purpose, the technical scheme adopted by the present utility model is:

[0019] A fruit and vegetable picking robot based on machine vision, comprising: a tracked frame 1, a vehicle body 2, a visual recognition device 3, a multi-degree-of-freedom manipulator 4, a picking manipulator 5 and a control device 6, the vehicle body 2 is installed on the tracked frame 1, the front side of the vehicle body 2 is provided with an assembly slot, the visual recognition device 3 is installed in the assembly slot, the vehicle body 2 is provided with a top-open accommodating slot, the slot opening of the accommodating slot is provided with at least one rotatable multi-degree-of-freedom manipulator 4, and the end of each multi-degree-of-freedom manipulator 4 is provided with one picking manipulator 5;

[0020] The vehicle body 2 is provided with an openable and closable door plate communicating with a containing groove, the containing groove is used for placing picked fruits and vegetables or baskets for containing fruits and vegetables, the picking mechanical hand 5 is used for picking fruits and vegetables and moving to the containing groove through the multi-degree-of-freedom mechanical arm 4, the visual recognition device 3 is used for collecting information and transmitting to the control device 6, and the control device 6 is used for controlling the operation of the crawler frame 1, the multi-degree-of-freedom mechanical arm 4 and the picking mechanical hand 5.

[0021] The fruit and vegetable picking robot based on machine vision, wherein the vehicle body 2 comprises a bottom plate, side plates, a front side support plate and a front partition plate, the four side plates and the bottom plate form a containing groove with an open top side, the upper end of the side plate at the front side is connected to the front side edge of the front side support plate, the two ends of the front side support plate are respectively connected to the two side plates at the left and right sides, the lower end of the front partition plate is connected to the rear side edge of the front side support plate, and the two ends of the front partition plate are respectively connected to the two side plates at the left and right sides.

[0022] The fruit and vegetable picking robot based on machine vision, wherein a fruit taking opening for taking out fruits and vegetables is arranged between the side plate at the rear side and the bottom plate, the upper end of the door plate is hingedly connected to the lower side edge of the side plate and is lockably connected to the bottom plate.

[0023] The fruit and vegetable picking robot based on machine vision, wherein the multi-degree-of-freedom mechanical arm 4 comprises a mounting base 401, a rotating shoulder, a rear swing arm 402, a front swing arm 403 and a rotating wrist 404, the mounting base 401 is mounted at any corner of the slot opening of the containing groove, the rotating shoulder is rotatably mounted on the mounting base 401, the lower end of the rear swing arm 402 is rotatably connected to one side of the rotating shoulder, the rear end of the front swing arm 403 is rotatably connected to the upper end of the rear swing arm 402, and the rotating wrist 404 is rotatably mounted on the front end of the front swing arm 403.

[0024] The fruit and vegetable picking robot based on machine vision, wherein the multi-degree-of-freedom mechanical arm 4 further comprises a first driving motor 408 and a second driving motor 405, a first mounting groove is formed in the top of the rotating shoulder, a second mounting groove is formed in the other side of the rotating shoulder, the housing of the first driving motor 408 is mounted in the first mounting groove, the output end of the first driving motor 408 is connected to the mounting base 401, and the first driving motor 408 is used to drive the rotating shoulder to rotate around its own axis to adjust the grabbing direction of the picking mechanical hand 5; the housing of the second driving motor 405 is mounted in the second mounting groove, the output end of the second driving motor 405 is rotatably connected to the lower end of the rear swing arm 402, and the second driving motor 405 is used to drive the rear swing arm 402 to swing forward.

[0025] The multi-degree-of-freedom mechanical arm 4 further comprises a third driving motor 406, a third mounting groove is formed in the rear end of the front swing arm 403, the shell of the third driving motor 406 is mounted in the third mounting groove, the output end of the third driving motor 406 is rotationally connected with the upper end of the rear swing arm 402, and the third driving motor 406 is used for driving the front swing arm 403 to swing up and down.

[0026] The multi-degree-of-freedom mechanical arm 4 further comprises a fourth driving motor 407, the fourth driving motor 407 is mounted at the front end of the front swing arm 403, and the fourth driving motor 407 is used for driving the rotating wrist 404 to swing up and down.

[0027] The fruit and vegetable picking robot based on machine vision has the following advantages: the crawler frame 1 comprises a chassis 10 and two crawler frames 11, the two crawler frames 11 are symmetrically arranged and are rotationally mounted on the left and right sides of the chassis 10.

[0028] The fruit and vegetable picking robot based on machine vision has the following advantages: the picking mechanical hand 5 comprises a fifth driving motor 501, a limiting support 502, a lead screw 503, a nut 504, a first connecting rod 505, a second connecting rod 506 and a finger 507, the shell rear end of the fifth driving motor 501 is mounted on the rotating wrist 404, the limiting support 502 is mounted on the shell front end of the fifth driving motor 501, the rear end of the lead screw 503 is connected with the output end of the fifth driving motor 501, the lead screw 503 and the limiting support 502 are coaxially arranged, the front end of the lead screw 503 is rotationally connected with the front end of the limiting support 502, the nut 504 is mounted on the lead screw 503, a plurality of limiting through holes are arranged on the outer periphery of the nut 504, a plurality of limiting shafts are arranged on the limiting support 502, each limiting shaft is located in one limiting through hole, the plurality of limiting shafts are parallel to the lead screw 503, a plurality of fingers 507 are equidistantly arranged around the circumferential direction of the axis of the lead screw 503, the plurality of fingers 507 are rotationally mounted on the front end of the limiting support 502, the rear end of each first connecting rod 505 is rotationally connected with the outer periphery of the nut 504, the front end of each first connecting rod 505 is rotationally connected with the rear end of one second connecting rod 506, and the front end of each second connecting rod 506 is rotationally connected with one finger 507, and the fifth driving motor 501 is used for driving the lead screw 503 to rotate around the axis thereof, so as to realize the displacement of the nut 504 along the lead screw 503.

[0029] The fruit and vegetable picking robot based on machine vision has the following advantages: the vision recognition device 3 is provided with a depth camera and a memory, and the depth camera and the memory are connected with the control device 6 in a circuit.

[0030] The machine vision-based fruit and vegetable picking robot has the upper computer and the single-chip microcomputer on the control device 6, the upper computer is used for processing information from the visual identification device 3, identifying fruits and vegetables and planning a path, and the single-chip microcomputer is used for controlling the operation of the tracked chassis 1, the multi-degree-of-freedom mechanical arm 4 and the picking manipulator 5.

[0031] The machine vision-based fruit and vegetable picking robot has the tracked chassis 11, which comprises a sixth driving motor 101, an inner mounting plate 102, a driving wheel 103, a tensioning device 104, a tensioning wheel 105, a driven wheel 106, a limiting pin shaft 107, a connecting shaft 108, an outer mounting plate and a track, the inner mounting plate 102 is rotatably mounted on the left side / right side of the chassis 10 through the connecting shaft 108, an arc-shaped limiting through hole is formed in the inner mounting plate 102 and is arranged below the connecting shaft 108, the limiting pin shaft 107 is slidably mounted in the arc-shaped limiting through hole, the end of the limiting pin shaft 107 is connected with the chassis 10, the driving wheel 103 and the driven wheels 106 are rotatably mounted on the inner mounting plate 102 through connecting shafts, the sixth driving motor 101 is used for driving the driving wheel 103 to rotate, the tensioning device 104 is mounted on the inner mounting plate 102, the tensioning wheel 105 is rotatably mounted on the adjusting end of the tensioning device 104, the track is arranged around the driving wheel 103, the driven wheels 106 and the tensioning wheel 105, the track is in a tensioning state by adjusting the tensioning device 104, a plurality of limiting protrusions are arranged on the inner mounting plate 102, a plurality of strip-shaped grooves are arranged on the outer mounting plate, each limiting protrusion is located in a strip-shaped groove, and the outer mounting plate and the inner mounting plate 102 are assembled through a plurality of screws.

[0032] The machine vision-based fruit and vegetable picking robot has the following advantages compared with the prior art:

[0033] (1) In the utility model, the tracked chassis is adopted for the picking robot mobile platform chassis, the contact area with the ground is increased, the weight can be better dispersed during the driving process of the robot, the chassis frame is made of aluminum alloy, has the characteristics of "four wheels and one track" of the general tracked walking mechanism, a tensioning device and a buffer spring, the chassis body is made of stainless steel plate, the chassis body has sufficient strength and rigidity and is not easy to deform, better damping effect can be provided, and the impact of uneven road on the robot is reduced; a plurality of motion modes are provided, including forward movement, transverse movement, inclined movement and rotation, so that the robot can freely drive in various complex terrains, including muddy and soft soil, slopes and areas full of weeds;

[0034] (2) The fruit and vegetable picking robot has obvious advantages in picking. It can freely move to any position according to actual needs, ensure comprehensive picking of fruits and vegetables, and the configuration of the fruit and vegetable picking robot system is also very flexible, can dynamically adjust the structure, freely increase or decrease the degrees of freedom, and complete flexible picking of fruits and vegetables.

[0035] (3) The fruit and vegetable picking robot has a relatively moderate size, and can freely travel in various complex terrains, including muddy, soft soil, slopes and areas with dense weeds, and is convenient to work. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a structure diagram of a fruit and vegetable picking robot based on machine vision.

[0037] Figure 2 is a structure diagram of a track frame of a fruit and vegetable picking robot based on machine vision.

[0038] Figure 3 is a structure diagram of a track walking mechanism of a fruit and vegetable picking robot based on machine vision.

[0039] Figure 4 is a structure diagram of a multi-degree-of-freedom mechanical arm of a fruit and vegetable picking robot based on machine vision.

[0040] Figure 5 is a picking manipulator of a multi-degree-of-freedom mechanical arm of a fruit and vegetable picking robot based on machine vision.

[0041] Figure 6 is a work flow chart of a fruit and vegetable picking robot based on machine vision.

[0042] In the drawings: 1, track frame; 2, vehicle body; 3, visual identification device; 4, multi-degree-of-freedom mechanical arm; 5, picking manipulator; 6, control device; 10, chassis; 11, track walking mechanism; 101, sixth driving motor; 102, inner side mounting plate; 103, driving wheel; 104, tensioning device; 105, tensioning wheel; 106, driven wheel; 107, limit pin shaft; 108, connecting shaft; 401, mounting base; 402, rear swing arm; 403, front swing arm; 404, rotating wrist; 405, second driving motor; 406, third driving motor; 407, fourth driving motor; 408, first driving motor; 501, fifth driving motor; 502, limit support; 503, lead screw; 504, nut; 505, first connecting rod; 506, second connecting rod; 507, finger. DETAILED DESCRIPTION

[0043] The utility model will be further described below in connection with the drawings and specific embodiments, but not as the limitation of the utility model.

[0044] Please refer to Figures 1 to 6 As shown in the figure, a kind of fruit and vegetable picking robot based on machine vision is shown, including: tracked vehicle frame 1, vehicle body 2, visual identification device 3, multi-degree-of-freedom mechanical arm 4, picking manipulator 5 and control device 6, control device 6 is used to control the walking action of tracked vehicle frame 1, the folding and unfolding action of multi-degree-of-freedom mechanical arm 4 and the grabbing action of picking manipulator 5 and collects visual information in real time by visual identification device 3.

[0045] The above is only the preferred embodiment of the utility model, and does not limit the implementation and protection scope of the utility model.

[0046] The utility model further has the following implementation mode on the basis of the above:

[0047] In further embodiments of the utility model, picking manipulator 5 further includes: visual camera and pressure sensor, visual camera is installed at the front end of limiting support 502 and is located on the axis of lead screw 503, visual camera is used to assist visual identification device 3, and the position information of fruit and vegetable relative to picking manipulator 5 is accurately identified, picking manipulator 5 is assisted to grab fruit and vegetable, and one pressure sensor is installed on the inner side of each finger 507, to detect the surface pressure of fruit and vegetable by finger 507 in the process of picking manipulator 5 grabbing, to avoid that fruit and vegetable surface is damaged due to the picking force of picking manipulator 5 being too large, and visual camera and pressure sensor are connected with control device 6 using line or signal, and control device 6 is used to collect the information obtained by visual camera and pressure sensor.

[0048] In further embodiments of the utility model, in tracked walking mechanism 11, tensioning device 104 is composed of a fixed rod member and an active rod member sliding along the fixed rod member, and tensioning device 104 can adopt a tensioning mechanism for pushing active rod member inside fixed rod member through buffer spring.

[0049] In further embodiments of the utility model, in tracked walking mechanism 11, inner side mounting plate 102 and chassis 10 are rotatably connected through connecting shaft 108, and the maximum rotation angle of inner side mounting plate 102 is limited through limiting pin shaft 107, and the structure can meet that fruit and vegetable picking robot walks on road with large slope or rugged, and the maximum rotation angle of inner side mounting plate 102 is limited through limiting pin shaft 107, to effectively avoid the situation that fruit and vegetable picking robot rolls over, and ensure the stability of walking.

[0050] In further embodiments of the utility model, in the crawler walking mechanism 11, each sixth drive motor 101 drives a driving wheel 103 to rotate separately, realizing the separate driving of the two side crawler walking mechanisms 11, when the two sixth drive motors 101 work simultaneously, the fruit and vegetable picking robot can move forward and backward, when one sixth drive motor 101 works and the other sixth drive motor 101 does not work, the fruit and vegetable picking robot can turn left and right.

[0051] In further embodiments of the utility model, the control device 6 is provided with a power supply device for supplying power to the visual identification device 3, the multi-degree-of-freedom mechanical arm 4, the picking manipulator 5, the crawler walking mechanism 11, the upper computer and the single-chip microcomputer, realizing the electric drive structure as a whole, and the fruit and vegetable picking robot can be remotely controlled and does not need to be manually driven, and can work in a harsh environment.

[0052] In further embodiments of the utility model, the vehicle body 2 is provided with a containing groove with an open top and a cargo outlet for taking out fruits and vegetables, the containing groove can be used for directly placing fruits and vegetables, after the door plate is opened, the fruits and vegetables can be directly taken out from the cargo outlet, or a basket containing fruits and vegetables can be placed in the containing groove, and the basket can be pushed into or taken out from the cargo outlet.

[0053] In further embodiments of the utility model, the fruit and vegetable picking robot of the utility model is mainly applied to picking fruits and vegetables with hard peels and shells, after the picking manipulator 5 picks the fruits and vegetables, the picking manipulator 5 is moved above the containing groove through the movement of the multi-degree-of-freedom mechanical arm 4, the picking manipulator 5 is opened, and the fruits and vegetables fall into the containing groove.

[0054] In further embodiments of the utility model, the fruit and vegetable picking robot of the utility model can also be applied to picking fruits and vegetables with peels and shells that are easy to be damaged, the control device 6 controls the multi-degree-of-freedom mechanical arm 4 to move the picking manipulator 5 into the containing groove, the picking manipulator 5 is opened, a light placing action is realized, then the control device 6 controls the multi-degree-of-freedom mechanical arm 4 to move the picking manipulator 5 out of the containing groove and pick the next fruit and vegetable.

[0055] In further embodiments of the utility model, in the structural design of the picking manipulator 5, the linear motion is converted into the rotary motion of the motor for driving through the combination of the slider-crank mechanism motion and the screw rod transmission, the motion is more stable, and the intact rate of the fruits in the picking process is increased. In the grabbing process, the fifth drive motor 501 drives the screw rod 503 to rotate, under the action of the plurality of limiting shafts of the limiting support 502, the nut 504 does not rotate and is displaced towards the fifth drive motor 501, the first connecting rod 505 and the second connecting rod 506 are pulled, and the closing grabbing action of the plurality of fingers 507 is realized. The fifth drive motor 501 drives the screw rod 503 to rotate reversely, and the opening action of the plurality of fingers 507 is realized.

[0056] The utility model provides a fruit and vegetable picking robot based on machine vision, and the specific technical scheme is as follows:

[0057] The robot's appearance design: the picking robot mobile platform chassis adopts a track design, increasing the ground contact area, so that the robot can better disperse the weight during driving; the vehicle chassis support uses aluminum alloy as the frame, having the characteristics of a general track-type walking mechanism "four wheels and one belt", a tightening device and a buffer spring; the chassis body is made of stainless steel plate, so that the vehicle body has sufficient strength and rigidity and is not easy to deform, can provide better shock absorption effect, and reduce the impact of uneven road on the robot; there are various motion modes, including forward movement, lateral movement, diagonal movement, rotation and various motion modes, so that the robot can freely drive in various complex terrains, including muddy, soft soil, slopes and areas with dense grass.

[0058] Sensor selection and use: according to the characteristics of fruits and vegetables, we select a specific force sensor to convert the pressure applied to the sensor membrane area into a change in resistance value, thereby obtaining pressure information. Thus, different fruits and vegetables can be operated, grabbed, picked and placed.

[0059] The fruit and vegetable picking robot based on machine vision of the prior art has the following good effects:

[0060] All-around picking: the fruit and vegetable picking robot has a significant advantage in picking. It can freely move to any position according to actual needs, ensuring the overall picking of fruits and vegetables.

[0061] Flexible configuration: the configuration of the fruit and vegetable picking robot system is also very flexible, and the structure can be dynamically adjusted, the degrees of freedom can be freely increased or decreased, and the flexible picking of fruits and vegetables can be completed.

[0062] Five, convenience: the fruit and vegetable picking robot has a relatively moderate size, can freely drive in various complex terrains, including muddy, soft soil, slopes and areas with dense grass, and is convenient to work.

[0063] 1. System overall design

[0064] The fruit and vegetable picking robot based on machine vision is composed of multiple modules, including a mechanical arm module, a vision recognition module and a data processing module. The system realizes the monitoring of the picking environment and the indoor environment through these modules.

[0065] 2. Main modules and their functions

[0066] 2.1 Mechanical arm module

[0067] The mechanical arm structure based on a rotating mechanism is adopted, and a force sensor is carried at the metal grabbing end of the mechanical arm, so that the joint configuration can be dynamically adjusted, the degrees of freedom can be freely increased or decreased, the flexibility and operability of the mechanical arm are improved, the force sensor can send a voltage signal to the control system, and then the force of the mechanical arm grabbing is controlled, and the quality of the fruits is not damaged.

[0068] 2.2 visual recognition module

[0069] The hardware includes an AstraPro Plus depth camera, a memory card and auxiliary circuits, etc. The AstraPro Plus depth camera can synchronously capture images, effectively obtain three-dimensional information through a binocular stereo vision algorithm and time flight (ToF), and provide accurate environment perception and fruit and vegetable position information for the robot; the memory card records the processed image data, and the auxiliary circuit ensures the hardware connection.

[0070] 2.4 data processing module

[0071] The control system is composed of an upper computer Raspberry Pi and a single-chip microcomputer STM32, the Raspberry Pi processes the three-dimensional information provided by the depth camera, completes fruit and vegetable recognition and path planning, and the single-chip microcomputer STM32 controls the mobile platform and the mechanical arm to complete picking and placing work.

[0072] 3. Data processing and transmission process

[0073] The intelligent fruit and vegetable picking robot judges the travel route according to the environment in which it is located, avoids obstacles, models the environment and detects targets, etc., and the monitoring function is to monitor the yield and quality of fruits in real time. The user starts the robot system and gives the direction of the robot path planning, and the robot plans the specific fruit and vegetable picking path according to the given direction. The robot identifies the fruits and vegetables through the depth camera during driving, and gives the three-dimensional information provided by the depth camera to the Raspberry Pi for processing, the Raspberry Pi accurately locates the fruit and vegetable position, determines the picking scheme, and transmits the data processing result to the STAM32. The robot and the mechanical arm are controlled by the data of the Raspberry Pi, and after picking is completed, repeated identification is carried out to identify the successful picking and start the same process again.

[0074] The above is only the preferred embodiment of the utility model, and does not limit the implementation and protection scope of the utility model, and those skilled in the art should realize that any equivalent replacement and obvious change obtained by applying the contents of the utility model specification and drawings should be included in the protection scope of the utility model.

Claims

1. A fruit and vegetable harvesting robot based on machine vision, characterized in that, include: The vehicle includes a tracked frame (1), a vehicle body (2), a vision recognition device (3), a multi-degree-of-freedom robotic arm (4), a harvesting robot (5), and a control device (6). The vehicle body (2) is mounted on the tracked frame (1). The front side of the vehicle body (2) is provided with an assembly slot. The vision recognition device (3) is installed in the assembly slot. The vehicle body (2) is provided with a receiving slot with a top opening. At least one rotatable multi-degree-of-freedom robotic arm (4) is installed at the opening of the receiving slot. Each multi-degree-of-freedom robotic arm (4) has a harvesting robot (5) installed at its end. The vehicle body (2) is provided with an openable and closable door panel that communicates with the accommodating slot. The accommodating slot is used to place harvested fruits and vegetables or baskets for holding fruits and vegetables. The harvesting robot (5) is used to harvest fruits and vegetables and transfer them to the accommodating slot through the multi-degree-of-freedom robotic arm (4). The visual recognition device (3) is used to collect information and transmit it to the control device (6). The control device (6) is used to control the operation of the tracked vehicle frame (1), the multi-degree-of-freedom robotic arm (4) and the harvesting robot (5).

2. The fruit and vegetable harvesting robot based on machine vision according to claim 1, characterized in that, The vehicle body (2) includes: a floor plate, side plates, a front support plate and a front partition plate. The four side plates and the floor plate together form a receiving groove with a top side opening. The upper end of the side plate located on the front side is connected to the front side edge of the front support plate. The two ends of the front support plate are respectively connected to the two side plates located on the left and right sides. The lower end of the front partition plate is connected to the rear side edge of the front support plate. The two ends of the front partition plate are respectively connected to the two side plates located on the left and right sides.

3. The fruit and vegetable harvesting robot based on machine vision according to claim 1, characterized in that, There is a retrieval opening for taking out fruits and vegetables between the rear side panel and the bottom panel. The upper end of the door panel and the lower side of the side panel are hinged and can be locked to the bottom panel.

4. The fruit and vegetable harvesting robot based on machine vision according to claim 1, characterized in that, The multi-degree-of-freedom robotic arm (4) includes: a mounting base (401), a rotating shoulder, a rear swing arm (402), a front swing arm (403), and a rotating wrist (404). The mounting base (401) is mounted on any corner of the opening of the receiving slot. The rotating shoulder is rotatably mounted on the mounting base (401). The lower end of the rear swing arm (402) is rotatably connected to one side of the rotating shoulder. The rear end of the front swing arm (403) is rotatably connected to the upper end of the rear swing arm (402). The rotating wrist (404) is rotatably mounted on the front end of the front swing arm (403).

5. The machine vision-based fruit and vegetable harvesting robot according to claim 4, characterized in that, The multi-degree-of-freedom robotic arm (4) also includes: a first drive motor (408) and a second drive motor (405). A first mounting groove is provided on the top of the rotating shoulder, and a second mounting groove is provided on the other side of the rotating shoulder. The housing of the first drive motor (408) is installed in the first mounting groove. The output end of the first drive motor (408) is connected to the mounting base (401). The first drive motor (408) is used to drive the rotating shoulder to rotate around its own axis to adjust the grasping direction of the picking robot (5). The housing of the second drive motor (405) is installed in the second mounting groove. The output end of the second drive motor (405) is rotatably connected to the lower end of the rear swing arm (402). The second drive motor (405) is used to drive the rear swing arm (402) to swing forward. The multi-degree-of-freedom robotic arm (4) also includes: a third drive motor (406), a third mounting groove is provided at the rear end of the front swing arm (403), the housing of the third drive motor (406) is installed in the third mounting groove, the output end of the third drive motor (406) is rotatably connected to the upper end of the rear swing arm (402), and the third drive motor (406) is used to drive the front swing arm (403) to swing up and down; The multi-degree-of-freedom robotic arm (4) also includes a fourth drive motor (407), which is mounted on the front end of the front swing arm (403) and is used to drive the rotating wrist (404) to swing up and down.

6. The fruit and vegetable harvesting robot based on machine vision according to claim 1, characterized in that, The tracked vehicle frame (1) includes a chassis (10) and a tracked walking mechanism (11). The two tracked walking mechanisms (11) are symmetrically arranged and rotatably installed on the left and right sides of the chassis (10).

7. The machine vision-based fruit and vegetable harvesting robot according to claim 4, characterized in that, The harvesting robot (5) includes: a fifth drive motor (501), a limiting bracket (502), a lead screw (503), a nut (504), a first connecting rod (505), a second connecting rod (506), and fingers (507). The rear end of the housing of the fifth drive motor (501) is mounted on the rotating wrist (404). The limiting bracket (502) is mounted on the front end of the housing of the fifth drive motor (501). The rear end of the lead screw (503) is connected to the output end of the fifth drive motor (501). The lead screw (503) and the limiting bracket (502) are coaxially arranged. The front end of the lead screw (503) is rotatably connected to the front end of the limiting bracket (502). The nut (504) is mounted on the lead screw (503). The outer periphery of the nut (504) is provided with multiple limiting through holes. The positioning bracket (502) is provided with multiple limiting shafts, each of which is located in a limiting through hole. The multiple limiting shafts are parallel to the lead screw (503). Multiple fingers (507) are provided at equal intervals around the axis of the lead screw (503). The multiple fingers (507) are rotatably installed at the front end of the positioning bracket (502). The rear end of each first link (505) is rotatably connected to the outer circumference of the nut (504). The front end of each first link (505) is rotatably connected to the rear end of a second link (506). The front end of each second link (506) is rotatably connected to a finger (507). The fifth drive motor (501) is used to drive the lead screw (503) to rotate around its own axis, thereby realizing the displacement of the nut (504) along the lead screw (503).

8. The fruit and vegetable harvesting robot based on machine vision according to claim 1, characterized in that, The visual recognition device (3) is equipped with a depth camera and a memory, both of which are connected to the control device (6) via circuitry.

9. The machine vision-based fruit and vegetable harvesting robot according to claim 6, characterized in that, The control device (6) is equipped with a host computer and a microcontroller. The host computer is used to process information from the visual recognition device (3), identify fruits and vegetables and plan paths. The microcontroller is used to control the operation of the tracked vehicle frame (1), the multi-degree-of-freedom robotic arm (4) and the picking robot (5).

10. The machine vision-based fruit and vegetable harvesting robot according to claim 6, characterized in that, The tracked walking mechanism (11) includes: a sixth drive motor (101), an inner mounting plate (102), a drive wheel (103), a tensioning device (104), a tensioning wheel (105), a driven wheel (106), a limiting pin (107), a connecting shaft (108), an outer mounting plate, and tracks. The inner mounting plate (102) is rotatably mounted on the left / right side of the chassis (10) via the connecting shaft (108). An arc-shaped limiting through hole is provided on the inner mounting plate (102), which is located below the connecting shaft (108). The limiting pin (107) is slidably mounted in the arc-shaped limiting through hole, and the end of the limiting pin (107) is connected to the chassis (10). One drive wheel (103) and multiple driven wheels... The driving wheels (106) are all rotatably mounted on the inner mounting plate (102) via connecting shafts. The sixth drive motor (101) is used to drive the driving wheel (103) to rotate. The tensioning device (104) is mounted on the inner mounting plate (102). The tensioning wheel (105) is rotatably mounted on the adjusting end of the tensioning device (104). The track is arranged around a driving wheel (103), multiple driven wheels (106) and a tensioning wheel (105). The track is tensioned by adjusting the tensioning device (104). The inner mounting plate (102) is provided with multiple limiting protrusions. The outer mounting plate is provided with multiple strip grooves. Each limiting protrusion is located in a strip groove. The outer mounting plate and the inner mounting plate (102) are assembled by multiple screws.

Images