A tender shoot tea picking machine and tea picking method based on infrared spectrum multimodal recognition

Abstract

A tender shoot tea picking machine and a tea picking method based on infrared spectrum multimodal recognition, which relate to the technical field of tea garden machinery and are used for recognizing and picking tender shoots. It includes a frame, a fan assembly, a slide rail assembly, an identification and picking assembly and a manipulator. The bottom of the frame is provided with a traveling mechanism. The fan assembly includes a fan, a heating wire, a temperature sensor and a wind guide shell. The heating wire is used to heat the air in the wind guide shell, and the temperature sensor is used to detect the air temperature at the outlet of the wind guide shell; The identification and picking assembly includes a suction pipe, a fixing frame and a manipulator. The fixing frame is located on the slide rail assembly and drives the vertical and / or horizontal movement of the fixing frame through the slide rail assembly. The fixing frame is provided with a lead screw guide rail, a depth camera, a thermal imager and a hyperspectral spectrometer. One end of the suction pipe is connected to a collection box located on the frame, and the other end of the suction pipe is fixed on the positioning slider of the lead screw guide rail; The manipulator is fixed on the frame, and the manipulator is provided with a laser beam. The invention can pick tender shoots of tea leaves.

Description

Technical Field

[0001] The present invention relates to the technical field of tea garden machinery, and specifically to a tender shoot tea picking machine and a tea picking method based on infrared spectrum multimodal recognition. Background Art

[0002] In recent years, the tea industry in China has developed rapidly, and the planting area has been continuously increasing. However, the contradiction between tea picking and labor employment is a major problem existing in the current development of the tea industry. The tea picking operation has relatively high requirements. At present, it mainly relies on manual picking, but manual picking is time-consuming and laborious, and cannot meet the needs of large-area tea picking. When the existing tea picking machines perform the picking operation, they cannot distinguish between old leaves and tender shoots for all the tea leaves within the visual area. Picking old leaves and tender shoots together makes them mixed, affecting the overall quality of the tea. Subsequently, manual screening and classification have to be carried out, increasing the cost investment. Summary of the Invention

[0003] The purpose of the present invention is to provide a tender shoot tea picking machine and a tea picking method based on infrared spectrum multimodal recognition, which are used to distinguish between old leaves and tender shoots and pick the tender shoots.

[0004] The technical solution adopted by the present invention to solve its technical problems is as follows: A tender tea shoot picking machine based on infrared spectrum multimodal recognition, including a frame, a fan assembly, a slide rail assembly, an identification and picking assembly, and a manipulator. The bottom of the frame is provided with a traveling mechanism. The fan assembly includes a fan drive motor, a fan, a heating wire, a temperature sensor, and a wind guide shell. The fan and the fan drive motor are both fixed on the top of the frame, and the fan drive motor is used to drive the fan to work. The wind guide shell is located at the air outlet of the fan. The heating wire and the temperature sensor are located inside the wind guide shell. The heating wire is used to heat the air inside the wind guide shell to 40 degrees Celsius. The temperature sensor is used to detect the air temperature at the outlet of the wind guide shell. The slide rail assembly is located on the top of the frame. The identification and picking assembly includes a suction pipe, a fixing frame, and a manipulator. The fixing frame is located on the slide rail assembly and is driven by the slide rail assembly to move vertically and / or horizontally. The fixing frame is provided with a lead screw guide rail, a depth camera, a thermal imager, and a hyperspectral spectrometer. Among them, the thermal imager identifies the tender tea shoots through thermal imaging images. The hyperspectral spectrometer is used to detect the cellulose content of the object identified by the thermal imager to confirm whether it is a tender tea shoot. The depth camera identifies and locates the tender tea shoots confirmed by the hyperspectral spectrometer. The intersection of the sight lines of the thermal imager, the depth camera, and the hyperspectral spectrometer is the identification area. One end of the suction pipe is connected to a collection box located on the frame, and the other end of the suction pipe is fixed on the positioning slider of the lead screw guide rail. The other end of the suction pipe moves vertically with the positioning slider. The manipulator is fixed on the frame, and the manipulator is provided with a laser beam. The frame is also provided with a control cabinet, and the control cabinet is signal-connected to the traveling mechanism, the fan assembly, the slide rail assembly, the identification and picking assembly, and the manipulator to control the working state of the entire tea picking machine.

[0005] Further, the traveling mechanism includes a traveling support. The traveling support is provided with traveling wheels and a traveling drive motor for driving the traveling wheels to work. The front end of the traveling support is provided with two lidar sensors distributed left and right and inclined downward. The lidar sensors are used for map building and path planning.

[0006] Further, the frame is provided with two motor mounting brackets. Each motor mounting bracket is respectively provided with a fan drive motor. Each fan drive motor output shaft is provided with a fan. The air outlets of the two fans are both connected to the upper end of the wind guide shell, and the lower end of the wind guide shell faces the identification area.

[0007] Further, the slide rail assembly includes a vertical slide rail, a horizontal slide rail, a vertical slider, and a horizontal slider. The horizontal slide rail is fixed to the top of the frame. The horizontal slider is slidably disposed on the horizontal slide rail. The vertical slide rail is fixed to the horizontal slider and is perpendicular to the horizontal slide rail. The vertical slider is slidably disposed on the vertical slide rail. A first drive motor for driving the horizontal slider to move is provided on the horizontal slide rail. A second drive motor for driving the vertical slider to move is provided on the vertical slide rail. The fixing frame is fixedly connected to the vertical slider.

[0008] Further, one end of the straw is fixed to the positioning slider through a straw buckle.

[0009] Further, the cross-sectional dimension of the upper end of the air guide housing is larger than that of the lower end of the air guide housing.

[0010] The present invention also provides a tender shoot tea picking method based on infrared spectrum multimodal recognition, including the following steps:

[0011] (1) The lidar performs mapping positioning and path planning, and the entire tea picking machine advances along the planned path through the traveling mechanism;

[0012] (2) The fan assembly works to blow a constant temperature hot air of 40 °C to the tea tree bud leaves through the air guide housing, blowing off the dirt on the bud leaves, and identifying the tender shoots using the thermal imager according to the temperature difference between the bud and the leaves when heated;

[0013] (3) Detect the cellulose content of the tender shoots identified by the thermal imager using the hyperspectrometer, and confirm the accuracy of the identification by the thermal imager according to the cellulose content; after being detected as tender shoots by the hyperspectrometer, execute step (4);

[0014] (4) Use the depth camera to identify and position the tender shoots;

[0015] (5) The manipulator is in place and the laser beam is aligned with the picking point on the tender shoot, and then the straw is in place;

[0016] (6) The laser beam works to cut the tender shoot, and at the same time the straw works to adsorb the cut tender shoot into the collection box, completing the picking of the tender shoot, and then the straw and the manipulator are reset;

[0017] (7) Under the action of the traveling mechanism, the tea picking machine continues to advance, and continues to execute steps (2)-(6) until all the tea picking work is completed.

[0018] The beneficial effects of the present invention are:

[0019] (1) The air in the fan is heated to a constant temperature of 40 °C by the heating wire, and then blown to the tea tree leaves. On the one hand, it causes a temperature difference between the bud and the tea tree leaves when heated, and on the other hand, the hot air blows to the leaves to make the leaves shake and blow off the dirt on the leaves;

[0020] (2) First, the present invention identifies the heated tea buds through a thermal imager, and then uses hyperspectral to detect the cellulose content of the tea buds, and determines the identification of the buds through dual detection;

[0021] (3) The present invention combines deep learning to identify and locate the picking points of fresh tea buds, and uses a laser beam for cutting, with precise cutting;

[0022] (4) The present invention moves the picking component according to the height of the tea tree and tea leaves, and can identify and pick the fresh tea buds of tea trees with different canopy heights. Description of the Drawings

[0023] Figure 1 is a three-dimensional view of the present invention;

[0024] Figure 2 is a front view of the traveling mechanism;

[0025] Figure 3 is a front view of the fan assembly;

[0026] Figure 4 is a front view of the slide rail assembly;

[0027] Figure 5 is a three-dimensional view of the identification and picking component;

[0028] Figure 6 is a three-dimensional position schematic diagram of the thermal imager, hyperspectral instrument and depth camera;

[0029] Figure 7 is a schematic diagram when picking the buds Figure 1 ;

[0030] Figure 8 is a schematic diagram when picking the buds Figure 2 ;

[0031] Figure 9 is a schematic diagram when picking the buds Figure 3 ;

[0032] In the figure: 1 traveling mechanism, 11 traveling bracket, 12 traveling wheels, 13 lidar, 14 traveling drive motor, 2 frame, 3 fan assembly, 31 fan, 32 fan drive motor, 33 motor mounting bracket, 34 heating wire, 35 temperature sensor, 36 air guide housing, 4 slide rail assembly, 41 first drive motor, 42 vertical slide rail, 43 vertical slider, 44 horizontal slider, 45 horizontal slide rail, 46 second drive motor, 5 identification and picking component, 51 straw, 52 servo, 53 fixing bracket, 54 lead screw guide rail, 55 straw buckle, 56 depth camera, 57 thermal imager, 58 hyperspectral instrument, 59 laser beam, 6 manipulator, 61 control cabinet, 62 collection box, 7 buds. Detailed implementation mode

[0033] As shown Figures 1 to 9 in the figure, the present invention includes a traveling mechanism 1, a frame 2, a fan assembly 3, a slide rail assembly 4, an identification and picking assembly 5, a manipulator 6 and a control cabinet 61. The present invention will be described in detail below with reference to the accompanying drawings.

[0034] As shown Figures 1 to 6 in the figure, a tender shoot tea picking machine based on infrared spectrum multimodal recognition includes a frame 2, a fan assembly 3, a slide rail assembly 4, an identification and picking assembly 5 and a manipulator 6. The bottom of the frame 2 is provided with a traveling mechanism 1, and the whole tea picking machine is moved in the tea garden through the arrangement of the traveling mechanism 1. As shown Figure 2 in the figure, the traveling mechanism 1 includes a traveling support 11, on which there are traveling wheels 12 and a traveling drive motor 14 for driving the traveling wheels 12 to work. At the front end of the traveling support 11, there are two lidar sensors 13 distributed left and right and arranged obliquely downward. The lidar sensors 13 are used for map building and path planning, so that the traveling wheels 12 move forward along the path planned by the lidar sensors 13.

[0035] As shown Figure 1 and Figure 3 in the figure, the fan assembly 3 includes a fan drive motor 32, a fan 31, a heating wire 34, a temperature sensor 35 and a wind guide shell 36. The fan 31 and the fan drive motor 32 are both fixed on the top of the frame 2, and the fan drive motor 32 is used to drive the fan 31 to work. The wind guide shell 36 is located at the air outlet of the fan 31. The heating wire 34 and the temperature sensor 35 are located inside the wind guide shell 36. The heating wire 34 is used to heat the air inside the wind guide shell 36 and heat the air inside the wind guide shell 36 to 40 degrees Celsius. The temperature sensor 35 is used to detect the air temperature at the outlet of the wind guide shell 36. When the air temperature at the outlet of the wind guide shell 36 is lower than 40 degrees Celsius, the power of the heating wire 34 increases to raise the air temperature at the outlet of the wind guide shell 36, so as to ensure that the hot air blown to the tea leaves by the wind guide shell 36 is at 40 degrees Celsius. When the air temperature at the outlet of the wind guide shell 36 is higher than 40 degrees Celsius, the power of the heating wire 34 decreases to reduce the air temperature at the outlet of the wind guide shell 36 until the air temperature at the outlet of the wind guide shell 36 reaches 40 degrees Celsius. Through the cooperation of the temperature sensor 35 and the heating wire 34, the air temperature at the outlet of the wind guide shell 36 is stabilized at 40 degrees Celsius. For the convenience of assembly, there are two motor mounting brackets 33 on the frame 2. Each motor mounting bracket 33 is respectively provided with a fan drive motor 32, and each fan drive motor 32 output shaft is provided with a fan 31. The air outlets of the two fans 31 are both connected to the upper end of the wind guide shell 36. In this way, the wind guide shell 36 is located between the two fans 31. When the fans 31 work, they blow air into the wind guide shell 36. The function of the fan assembly 3 is to blow out constant temperature air at 40 degrees Celsius.

[0036] As shown Figure 1As shown, the slide rail assembly 4 is located at the top of the frame 2. Such as Figure 5 , Figure 6 As shown, the identification and picking assembly 5 includes a straw 51, a fixing frame 53, and a manipulator 6. The fixing frame 53 is located on the slide rail assembly 4 and drives the vertical movement and / or horizontal movement of the fixing frame 53 through the slide rail assembly 4. Such as Figure 4 As shown, the slide rail assembly 4 includes a vertical slide rail 42, a horizontal slide rail 45, a vertical slider 43, and a horizontal slider 44. The horizontal slide rail 45 is fixed to the top of the frame 2, and the horizontal slider 44 is slidably arranged on the horizontal slide rail 45; the vertical slide rail 42 is fixed to the horizontal slider 44 and is perpendicular to the horizontal slide rail 45, and the vertical slider 43 is slidably arranged on the vertical slide rail 42. The horizontal slide rail 45 is provided with a first driving motor 41 for driving the horizontal slider 44 to move, and the vertical slide rail 42 is provided with a second driving motor 46 for driving the vertical slider 43 to move. When the first driving motor 41 works, the vertical slide rail 42 moves horizontally in the horizontal plane along with the horizontal slider 44. When the second driving motor 46 works, the vertical slider 43 moves vertically on the vertical slide rail 42. The fixing frame 53 is fixedly connected to the vertical slider 43, and thus the fixing frame 53 is moved vertically or / and horizontally under the action of the slide rail assembly 4.

[0037] Such as Figure 5 As shown, the fixing frame 53 is provided with a lead screw guide rail 54, a depth camera 56, a thermal imager 57, and a hyperspectral spectrometer 58. Among them, the thermal imager 57 identifies the tender shoots through thermal imaging images, and the hyperspectral spectrometer 58 is used to detect the cellulose content of the object identified by the thermal imager to confirm whether the object identified by the thermal imager 57 is a tender shoot. The depth camera 56 identifies and locates the tender shoots confirmed by the hyperspectral spectrometer 58. Such as Figure 5As shown in the figure, the depth camera 56, the thermal imager 57, and the hyperspectral spectrometer 58 are all fixed to the bottom of the fixing frame 53, and the depth camera 56, the thermal imager 57, and the hyperspectral spectrometer 58 are arranged in sequence. The intersection of the sight lines of the depth camera 56, the thermal imager 57, and the hyperspectral spectrometer 58 is the recognition area, and the lower end of the air guide shell 36 faces the recognition area. The cross-sectional dimension of the upper end of the air guide shell 36 is larger than that of the lower end of the air guide shell 36, which can increase the air pressure discharged from the lower end of the air guide shell 36. The lead screw guide rail 54 is located between the depth camera 56, the thermal imager 57, and the hyperspectral spectrometer 58 and is arranged vertically. The top of the lead screw guide rail 54 is fixedly connected to the fixing frame 53. One end of the suction pipe 51 is connected to the collection box 62 located on the machine frame 2, and the other end of the suction pipe 51 is fixed to the positioning slider of the lead screw guide rail 54. For the convenience of assembly, the other end of the suction pipe 51 is fixed to the positioning slider through the suction pipe buckle 55, and thus the other end of the suction pipe 51 can move vertically with the positioning slider. There is a servo motor 52 on the fixing frame 53. The servo motor 52 is used to drive the movement of the positioning slider on the lead screw guide rail 54, and further drive the vertical movement of the other end of the suction pipe 51, so that the other end of the suction pipe 51 enters the recognition area. The manipulator 6 is fixed to the machine frame 2, and there is a laser beam 59 on the manipulator 6. Thus, the laser beam 59 can move under the drive of the manipulator 6 until it reaches the appropriate position to perform the cutting and picking operation. There is also a control cabinet 61 on the machine frame 2. The control cabinet 61 is signal-connected to the traveling mechanism 1, the fan assembly 3, the slide rail assembly 4, the recognition and picking assembly 5, and the manipulator 6 and controls the working state of the entire tea picking machine.

[0038] The present invention also provides a method for picking tender tea shoots based on infrared spectrum multimodal recognition, including the following steps:

[0039] (1) The lidar 13 performs mapping positioning and path planning, and the entire tea picking machine advances along the planned path through the traveling mechanism;

[0040] (2) The fan assembly works to blow 40-degree constant-temperature hot air to the tea shoot leaves through the air guide shell, blow off the dirt on the shoot leaves, and identify the tender shoots using the thermal imager according to the temperature difference between the shoot and the leaves when heated;

[0041] (3) As Figure 7 shown in the figure, use the hyperspectral spectrometer to detect the cellulose content of the tender shoots identified by the thermal imager 57, and confirm the accuracy of the thermal imager recognition according to the cellulose content; after being detected as tender shoots by the hyperspectral spectrometer 58, perform step (4);

[0042] (4) Use the depth camera 56 to identify and position the tender shoots;

[0043] (5) The control cabinet 61 sends signals to the manipulator 6 and the servo motor 52, so that the manipulator 6 is in place and the laser beam 59 is aligned with the picking point on the tender shoot, and then the servo motor 52 works to make the suction pipe 51 in place;

[0044] As shown in Figure 8 Figure [not provided], the laser beam 59 operates to cut the tender buds. As shown in Figure 9 Figure [not provided], at the same time, the straw 51 operates to suck the cut tender buds into the collection box 62, completing the picking of the tender buds 7. Subsequently, the straw 51 and the manipulator 6 return to their original positions;

[0045] (7) Under the action of the traveling mechanism, the tea picking machine continues to move forward and continues to execute steps (2)-(6) until all the tea picking work is completed.

[0046] The beneficial effects of the present invention are as follows: The air in the blower is heated to a constant temperature of 40 °C by the heating wire and then blown onto the tea leaves of the tea tree. On the one hand, it causes a temperature difference between the bud heads and the tea tree leaves due to heat, and on the other hand, the hot air blowing onto the tea leaves makes the tea leaves shake and blows off the dirt on the tea leaves; The present invention first identifies the heated tea tender buds through a thermal imager, and then uses a hyperspectral spectrometer to detect the cellulose content of the tea tender buds, and determines the tender buds through dual detection; The present invention combines deep learning for identification and positioning of the picking points of fresh tea tender buds, and uses a laser beam for cutting, with precise cutting; The present invention moves the identification and picking components according to the height of the tea tree leaves, and can identify and pick the fresh tea tender buds of tea trees with different canopy heights.

Claims

1. A tender shoot tea picking machine based on infrared spectrum multimodal recognition, characterized in that, it includes a frame, a fan assembly, a slide rail assembly, an identification and picking assembly, and a manipulator; The bottom of the frame is provided with a traveling mechanism; The fan assembly includes a fan drive motor, a fan, a heating wire, a temperature sensor, and a wind guide shell. The fan and the fan drive motor are both fixed on the top of the frame, and the fan drive motor is used to drive the fan to work. The wind guide shell is located at the air outlet of the fan. The heating wire and the temperature sensor are located inside the wind guide shell. The heating wire is used to heat the air inside the wind guide shell, and the temperature sensor is used to detect the air temperature at the outlet of the wind guide shell; The slide rail assembly is located on the top of the frame. The identification and picking assembly includes a suction pipe, a fixing frame, and a manipulator. The fixing frame is located on the slide rail assembly and is driven by the slide rail assembly to move vertically and / or horizontally. The fixing frame is provided with a lead screw guide rail, a depth camera, a thermal imager, and a hyperspectral spectrometer. Among them, the thermal imager identifies the tender shoots through the thermal imaging image, and the hyperspectral spectrometer is used to detect the cellulose content of the object identified by the thermal imager to confirm whether it is a tender shoot. The depth camera identifies and locates the tender shoots confirmed by the hyperspectral spectrometer. The intersection of the lines of sight of the thermal imager, the depth camera, and the hyperspectral spectrometer is the identification area; One end of the suction pipe is connected to a collection box located on the frame, and the other end of the suction pipe is fixed on the positioning slider of the lead screw guide rail, and the other end of the suction pipe moves vertically with the positioning slider; The manipulator is fixed on the frame. The manipulator is provided with a laser beam. The frame is also provided with a control cabinet. The control cabinet is signal-connected to the traveling mechanism, the fan assembly, the slide rail assembly, the identification and picking assembly, and the manipulator and controls the working state of the entire tea picking machine.

2. The tender shoot tea picking machine based on infrared spectrum multimodal recognition according to claim 1, characterized in that, The traveling mechanism includes a traveling bracket. The traveling bracket is provided with traveling wheels and a traveling drive motor for driving the traveling wheels to work. The front end of the traveling bracket is provided with two lidars distributed left and right and inclined downward. The lidars are used for mapping positioning and path planning.

3. The tender shoot tea picking machine based on infrared spectrum multimodal recognition according to claim 1, characterized in that, The frame is provided with two motor mounting brackets. Each motor mounting bracket is respectively provided with a fan drive motor. Each fan drive motor output shaft is provided with a fan. The air outlets of the two fans are both connected to the upper end of the wind guide shell, and the lower end of the wind guide shell faces the identification area.

4. The tender shoot tea picking machine based on infrared spectrum multimodal recognition according to claim 1, characterized in that, The slide rail assembly includes a vertical slide rail, a transverse slide rail, a vertical slider and a transverse slider. The transverse slide rail is fixed on the top of the frame, the transverse slider is slidably set on the transverse slide rail, the vertical slide rail is fixed on the transverse slider and is perpendicular to the transverse slide rail, the vertical slider is slidably set on the vertical slide rail, the transverse slide rail has a first drive motor for driving the transverse slider to move, the vertical slide rail has a second drive motor for driving the vertical slider to move, and the fixing frame is fixedly connected to the vertical slider.

5. The tender bud tea picking machine based on infrared spectrum multimodal recognition according to claim 1, It is characterized in that One end of the straw is fixed on the positioning slide block through a straw buckle.

6. The tender bud tea picking machine based on infrared spectrum multimodal recognition according to claim 1, It is characterized in that The cross-sectional dimension of the upper end of the air guide shell is greater than the cross-sectional dimension of the lower end of the air guide shell.

7. A tea picking method using a young bud tea picking machine based on infrared spectrum multimodal recognition according to any one of claims 1 to 6, It is characterized in that The following steps are involved: (1) The laser radar performs mapping and path planning, and the walking mechanism enables the entire tea picking machine to move along the planned path; (2) The fan assembly blows 40 degrees Celsius constant temperature hot air to the tea tree buds and leaves through the air guide shell, blowing off the dirt on the buds and leaves, and uses a thermal imager to identify the tender buds based on the temperature difference between the buds and leaves; (3) Using a hyperspectral instrument to detect the cellulose content of the tender shoots identified by the thermal imager, and confirming the accuracy of the thermal imager's identification based on the cellulose content; after the tender shoots are detected by the hyperspectral instrument, executing step (4); (4) Using a depth camera to identify and locate the sprouts; (5) The robot arm is in place and the laser beam is aimed at the picking point on the bud, and then the straw is in place; (6) The laser beam works to cut the tender buds, and at the same time, the straw works to absorb the cut tender buds into the collection box, completing the picking of the tender buds, and then the straw and the manipulator are reset; (7) The tea picking machine continues to move forward under the action of the walking mechanism, and continues to execute steps (2) to (6) until the tea picking work is completed.

Images