New shoot tea picking machine and tea picking method based on multimodal recognition using infrared spectrum

Abstract

To provide a sprout tea harvesting machine and a tea harvesting method based on multi modal recognition using infrared ray spectrum.SOLUTION: A sprout tea harvesting machine includes a frame, a fan assembly, a slide rail module, a recognition harvesting assembly, and a manipulator. On a bottom of the frame, a travel mechanism is provided. The recognition harvesting assembly includes a suction pipe, a fixing frame, and a manipulator. The fixing frame is positioned at the slide rail assembly and drives movement of the fixing frame in a vertical direction and / or a lateral direction through the slide rail assembly, one end of the suction pipe is connected to a collection box positioned at the frame, the other end of the suction pipe is fixed to a positioning slider of a lead screw guide rail, the manipulator is fixed to the frame, and the manipulator has a laser generator.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to the technical field of tea garden machinery, and specifically to a new shoot tea picking machine and a tea picking method based on multimodal recognition by infrared spectrum.

Background Art

[0002] In recent years, the tea industry in China has developed rapidly, and the cultivation area has been continuously increasing. The contradiction between tea leaf picking and workers is a major problem existing in the current development of the tea industry. The tea leaf picking operation has high requirements and currently mainly relies on manual picking, but manual picking is time-consuming and laborious and cannot meet the needs of large-area tea leaf picking. When existing tea picking machines perform the picking operation, they cannot distinguish old leaves from new shoots for all tea leaves within the visual area, and old leaves and new shoots are picked together, resulting in a mixture of old leaves and new shoots, which affects the quality of the whole tea leaves. Subsequently, it is necessary to manually select and classify them, increasing the cost input.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The object of the present invention is to provide a new shoot tea picking machine and a tea picking method based on multimodal recognition by infrared spectrum, which can distinguish old leaves from new shoots and are used for picking new shoots.

Means for Solving the Problems

[0004] The technical solution used by the present invention to solve its technical problems is as follows.The new shoot tea picking machine based on multimodal recognition by infrared spectrum includes a frame, a fan assembly, a slide rail assembly, a recognition and picking assembly, and a manipulator. The bottom of the frame has a traveling mechanism. The fan assembly includes a fan drive motor, a fan, a heating wire, a temperature sensor, and a air guide case. The fan and the fan drive motor are fixed at the topmost part of the frame. The fan drive motor is used to drive the fan to operate. The air guide case is located at the air outlet of the fan. The heating wire and the temperature sensor are located inside the air guide case. The heating wire is used to heat the air inside the air guide case to 40 degrees Celsius. The temperature sensor is used to detect the air temperature at the outlet of the air guide case. The slide rail assembly is located at the topmost part of the frame. The recognition and picking assembly includes a suction pipe, a fixed frame, and a manipulator. The fixed frame is located on the slide rail assembly and drives the vertical movement and / or lateral movement of the fixed frame through the slide rail assembly. The fixed frame has a lead screw guide rail, a depth camera, a thermography, and a hyperspectral imaging system. The thermography recognizes new shoots through a thermography image. The hyperspectral imaging system is used to detect the cellulose content of the object recognized by the thermography to confirm whether it is a new shoot. The depth camera recognizes and positions the new shoots confirmed by the hyperspectral imaging system. The intersection of the lines of sight of the thermography, the depth camera, and the hyperspectral imaging system is the recognition area. One end of the suction pipe is connected to a collection box located on the frame. The other end of the suction pipe is fixed to the positioning slider of the lead screw guide rail. The other end of the suction pipe moves vertically with the positioning slider. The manipulator is fixed to the frame. The manipulator has a laser generator. The frame further has a control cabinet. The control cabinet is signal-connected to the traveling mechanism, the fan assembly, the slide rail assembly, the recognition and picking assembly, and the manipulator to control the operating state of the entire tea picking machine.

[0005] Furthermore, the traveling mechanism includes a traveling bracket, the traveling bracket has traveling wheels and a traveling drive motor for driving the traveling wheels to operate, and at the front end of the traveling bracket, there are two laser radars distributed left and right and installed obliquely downward. The laser radars are used for map creation, positioning, and route planning.

[0006] Furthermore, the frame has two motor mounting frames, each motor mounting frame has one fan drive motor, each output shaft of the fan drive motor has one fan, the air outlets of the two fans are connected to the upper end of the air guide case, and the lower end of the air guide case faces the recognition area.

[0007] Furthermore, the slide rail assembly includes a vertical slide rail, a horizontal slide rail, a vertical slider, and a horizontal slider. The horizontal slide rail is fixed to the top of the frame, the horizontal slider is slidably installed on the horizontal slide rail, the vertical slide rail is fixed to the horizontal slider and installed perpendicular to the horizontal slide rail, the vertical slider is slidably installed on the vertical slide rail, the horizontal slide rail has a first drive motor for driving the horizontal slider to move, the vertical slide rail has a second drive motor for driving the vertical slider to move, and the fixed frame is fixedly connected to the vertical slider.

[0008] Furthermore, one end of the suction pipe is fixed to the positioning slider through a suction pipe fastener.

[0009] Furthermore, the cross-sectional dimension of the upper end of the air guide case is larger than the cross-sectional dimension of the lower end of the air guide case.

[0010] The present invention is a new shoot tea picking method based on multimodal recognition by infrared spectrum, comprising (1) The laser radar performs map creation, positioning, and path planning, and the traveling mechanism advances the entire tea picking machine along the planned path; (2) The fan assembly operates to blow hot air at a constant temperature of 40°C onto the leaf blade at the tip of the bud of the tea tree through the air duct, blow off the dirt on the leaf blade at the tip of the bud, and utilize thermography to recognize new buds based on the temperature difference between the tip of the bud and the leaf blade; (3) Using the hyperspectral imaging system, detect the cellulose content of the new buds recognized by thermography, confirm the recognition accuracy of thermography based on the cellulose content, and after being detected as new buds by the hyperspectral imaging system, execute step (4); (4) Using the depth camera, recognize and position the new buds; (5) Place the manipulator at a predetermined position, align the laser generator with the picking point on the new bud, and then place the suction pipe at a predetermined position; (6) When the laser generator operates to cut the new buds, at the same time, the suction pipe operates to adsorb the cut new buds into the collection box, complete the picking of the new buds, and then reset the suction pipe and the manipulator; (7) The tea picking machine continues to advance by the traveling mechanism and continuously executes steps (2) to (6) until all the tea picking operations are completed, providing a new bud tea picking method based on multimodal recognition by the infrared spectrum.

Advantages of the Invention

[0011] The beneficial effects of the present invention are as follows. (1) After heating the air in the fan to a constant temperature of 40°C with the heating wire and blowing it onto the tea leaves of the tea tree, on the one hand, the tip of the bud and the leaf blade of the tea tree are heated to form a temperature difference, and on the other hand, the hot air is blown onto the tea leaves to shake the tea leaves and blow off the dirt on the tea leaves. (2) The present invention first recognizes the new shoots of tea leaves heated by thermography, and then uses a hyperspectral imaging system to detect the cellulose content of the new shoots of tea leaves, and recognizes and determines the new shoots by double detection. (3) The present invention combines deep learning to recognize and position the picking points of the new shoots of fresh leaves, and cuts them with a laser generator, and the cutting is accurate. (4) The present invention can move the recognition and picking assembly according to the height of the tea leaves on the tea tree, and recognize and pick the new shoots of the fresh leaves of the tea trees at different canopy heights.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying out the Invention

[0013] As shown in FIGS. 1 to 9, the present invention includes a traveling mechanism 1, a frame 2, a fan assembly 3, a slide rail assembly 4, a recognition and picking assembly 5, a manipulator 6 and a control cabinet 61. Hereinafter, the present invention will be described in detail with reference to the drawings.

[0014] As shown in FIGS. 1 to 6, the new sprout tea picking machine based on multimodal recognition by infrared spectrum includes a frame 2, a fan assembly 3, a slide rail assembly 4, a recognition and picking assembly 5, and a manipulator 6. The bottom of the frame 2 has a traveling mechanism 1, and the installation of the traveling mechanism 1 realizes the movement of the entire tea picking machine in the tea garden. As shown in FIG. 2, the traveling mechanism 1 includes a traveling bracket 11, and the traveling bracket 11 has a traveling wheel 12 and a traveling drive motor 14 that drives the traveling wheel 12 to operate. At the front end of the traveling bracket 11, there are two laser radars 13 that are distributed left and right and installed obliquely downward. The laser radar 13 is used for map creation, positioning, and path planning. The traveling wheel 12 moves forward according to the path planned by the laser radar 13.

[0015] As shown in FIGS. 1 and 3, the fan assembly 3 includes a fan drive motor 32, a fan 31, a heating wire 34, a temperature sensor 35, and a wind guide case 36. The fan 31 and the fan drive motor 32 are fixed to the uppermost part of the frame 2. The fan drive motor 32 is used to drive the fan 31 to operate. The wind guide case 36 is located at the air outlet of the fan 31. The heating wire 34 and the temperature sensor 35 are located inside the wind guide case 36. The heating wire 34 heats the air inside the wind guide case 36 and is used to heat the air inside the wind guide case 36 to 40 degrees Celsius. The temperature sensor 35 is used to detect the air temperature at the outlet of the wind guide case 36. When the air temperature at the outlet of the wind guide case 36 is lower than 40 degrees Celsius, the power of the heating wire 34 is increased to raise the air temperature at the outlet of the wind guide case 36, ensuring that the temperature of the hot air blown from the wind guide case 36 onto the tea leaves reaches 40 degrees Celsius. When the air temperature at the outlet of the wind guide case 36 is higher than 40 degrees Celsius, the power of the heating wire 34 is decreased to lower the air temperature at the outlet of the wind guide case 36 until the air temperature at the outlet of the wind guide case 36 reaches 40 degrees Celsius. Through the cooperation of the temperature sensor 35 and the heating wire 34, the air temperature at the outlet of the wind guide case 36 stabilizes at 40 degrees Celsius. To facilitate assembly, the frame 2 has two motor mounting frames 33. Each motor mounting frame 33 has one fan drive motor 32 respectively. Each output shaft of the fan drive motor 32 has one fan 31. The air outlets of the two fans 31 are connected to the upper end of the wind guide case 36. In this way, the wind guide case 36 is located between the two fans 31. When the fans 31 operate, air is blown into the wind guide case 36. The fan assembly 3 serves to blow out air at a constant temperature of 40 degrees Celsius.

[0016] As shown in FIG. 1, the slide rail assembly 4 is located at the top of the frame 2. As shown in FIGS. 5 and 6, the recognition and picking assembly 5 includes a suction pipe 51, a fixed frame 53, and a manipulator 6. The fixed frame 53 is located on the slide rail assembly 4 and drives the vertical movement and / or lateral movement of the fixed frame 53 via the slide rail assembly 4. As shown in FIG. 4, the slide rail assembly 4 includes a vertical slide rail 42, a lateral slide rail 45, a vertical slider 43, and a lateral slider 44. The lateral slide rail 45 is fixed to the top of the frame 2. The lateral slider 44 is slidably installed on the lateral slide rail 45. The vertical slide rail 42 is fixed to the lateral slider 44 and is installed perpendicular to the lateral slide rail 45. The vertical slider 43 is slidably installed on the vertical slide rail 42. The lateral slide rail 45 has a first drive motor 46 for driving the lateral slider 44 to move. The vertical slide rail 42 has a second drive motor 41 for driving the vertical slider 43 to move. When the first drive motor 46 operates, the vertical slide rail 42 moves laterally in the horizontal plane along with the lateral slide block 44. When the second drive motor 41 operates, the vertical slider 43 moves vertically on the vertical slide rail 42. The fixed frame 53 is fixedly connected to the vertical slider 43, thereby realizing the vertical and / or lateral movement of the fixed frame 53 under the action of the slide rail assembly 4.

[0017] As shown in Fig. 5, the fixed frame 53 has a lead screw guide rail 54, a depth camera 56, a thermography 57, and a hyperspectral imaging system 58. The thermography 57 recognizes new shoots through a thermographic image. The hyperspectral imaging system 58 is used to detect the cellulose content of the object recognized by the thermography to confirm whether the object recognized by the thermography 57 is a new shoot. The depth camera 56 recognizes and positions the new shoots confirmed by the hyperspectral imaging system 58. As shown in Fig. 5, the depth camera 56, the thermography 57, and the hyperspectral imaging system 58 are fixed to the bottom of the fixed frame 53, and the depth camera 56, the thermography 57, and the hyperspectral imaging system 58 are installed sequentially. The intersection of the lines of sight of the depth camera 56, the thermography 57, and the hyperspectral imaging system 58 is the recognition area, and the lower end of the air guide case 36 faces the recognition area. The cross-sectional dimension of the upper end of the air guide case 36 is larger than that of the lower end of the air guide case 36. In this way, the pressure of the air discharged from the lower end of the air guide case 36 can be increased. The lead screw guide rail 54 is located between the depth camera 56, the thermography 57, and the hyperspectral imaging system 58 and is installed vertically. The uppermost part of the lead screw guide rail 54 is fixedly connected to the fixed frame 53. One end of the suction pipe 51 is connected to the collection box 62 located in the frame 2, and the other end of the suction pipe 51 is fixed to the positioning slider of the lead screw guide rail 54. To facilitate assembly, the other end of the suction pipe 51 is fixed to the positioning slider via a suction pipe fastener 55. Thus, the other end of the suction pipe 51 can move vertically along with the positioning slider. The fixed frame 53 has a servo 52, and the servo 52 is used to drive and move the positioning slider on the lead screw guide rail 54. Thereby, the other end of the suction pipe 51 is driven to move vertically so that the other end of the suction pipe 51 enters the recognition area.The manipulator 6 is fixed to the frame 2. The manipulator 6 has a laser generator 59. Thus, the laser generator 59 can move under the drive of the manipulator 6 until it reaches an appropriate position to perform cutting and picking operations. The frame 2 further has a control cabinet 61. The control cabinet 61 is signal-connected to the traveling mechanism 1, the fan assembly 3, the slide rail assembly 4, the recognition and picking assembly 5, and the manipulator 6, and controls the operating state of the entire tea picking machine.

[0018] The present invention is a new shoot tea picking method based on multimodal recognition by infrared spectrum, comprising: (1) A step in which the laser radar 13 creates a map, positions, and plans a route, and the traveling mechanism advances the entire tea picking machine along the planned route; (2) A step in which the fan assembly operates to blow hot air at a constant temperature of 40 degrees Celsius onto the leaf blade at the tip of the shoot of the tea tree through the air duct case, blow off the dirt on the leaf blade at the tip of the shoot, and recognize the new shoot using thermography based on the difference in heating temperature between the tip of the shoot and the leaf blade; (3) As shown in FIG. 7, using a hyperspectral imaging system, detecting the cellulose content of the new shoot recognized by the thermography 57, confirming the recognition accuracy of the thermography based on the cellulose content, and after being detected as a new shoot by the hyperspectral imaging system 58, executing step (4); (4) A step of recognizing and positioning the new shoot using the depth camera 56; (5) A step in which the control cabinet 61 sends signals to the manipulator 6 and the servo 52, arranges the manipulator 6 at a predetermined position, aligns the laser generator 59 with the picking point on the new shoot, and then the servo 52 operates to arrange the suction pipe 51 at a predetermined position; (6) As shown in FIG. 8, the laser generator 59 operates to cut the new shoots, and as shown in FIG. 9, simultaneously, the suction pipe 51 operates to adsorb the cut new shoots into the collection box 62, completing the picking of the new shoots 7, and then resetting the suction pipe 51 and the manipulator 6. (7) A method for picking new shoots of tea based on multimodal recognition using an infrared spectrum is provided, including the step of continuously executing steps (2) to (6) until all the tea picking operations are completed while the tea picker continues to move forward by the traveling mechanism.

[0019] The beneficial effects of the present invention are as follows. After heating the air in the fan to a constant temperature of 40°C with a heating wire and blowing it onto the tea leaves of the tea tree, on the one hand, the tip of the bud and the leaf blade of the tea tree are heated to form a temperature difference, and on the other hand, the hot air is blown onto the tea leaves to shake the tea leaves and blow off the dirt on the tea leaves. The present invention first recognizes the new shoots of the tea leaves heated by thermography, and then uses a hyperspectral imaging system to detect the cellulose content of the new shoots of the tea leaves, and determines the new shoots by double detection. The present invention combines deep learning to recognize, position, and cut with a laser generator the picking points of the new shoots of the fresh leaves, and the cutting is accurate. The present invention can move the recognition and picking assembly according to the height of the leaves of the tea tree, and recognize and pick the new shoots of the fresh leaves of the tea trees with different canopy layer heights.

Explanation of Reference Numerals

[0020] In the figure: 1 traveling mechanism, 11 traveling bracket, 12 traveling wheels, 13 laser radar, 14 traveling drive motor, 2 frame, 3 fan assembly, 31 fan, 32 fan drive motor, 33 motor mounting frame, 34 heating wire, 35 temperature sensor, 36 air duct case, 4 slide rail assembly, 46 first drive motor, 42 vertical slide rail, 43 vertical slider, 44 horizontal slider, 45 horizontal slide rail, 41 second drive motor, 5 recognition and picking assembly, 51 suction pipe, 52 servo, 53 fixed frame, 54 lead screw guide rail, 55 suction pipe stopper, 56 depth camera, 57 thermography, 58 hyperspectral imaging system, 59 laser generator, 6 manipulator, 61 control cabinet, 62 collection box, 7 new shoots.

Claims

1. A new tea bud picking machine based on multimodal recognition by infrared spectrum, comprising a frame, a fan assembly, a slide rail assembly, a recognition picking assembly and a manipulator; The bottom of the frame has a running mechanism, the fan assembly includes a fan drive motor, a fan, a heating wire, a temperature sensor, and an air guide case, the fan and the fan drive motor are fixed to a top of a frame, the fan drive motor is used to drive and operate the fan, the air guide case is located at an air outlet of the fan, the heating wire and the temperature sensor are located in the air guide case, the heating wire is used to heat the air in the air guide case, and the temperature sensor is used to detect the air temperature at the outlet of the air guide case, The slide rail assembly is located at the top of the frame, and the recognition and picking assembly includes a suction tube, a fixed frame, and a manipulator. The fixed frame is located on the slide rail assembly and drives the vertical and / or lateral movement of the fixed frame via the slide rail assembly. The fixed frame has a lead screw guide rail, a depth camera, a thermography, and a hyperspectral imaging system. The thermography recognizes sprouts through a thermography image, and the hyperspectral imaging system is used to detect the cellulose content of the object recognized by the thermography to confirm whether it is a sprout. The depth camera recognizes and positions the sprouts recognized by the hyperspectral imaging system. The intersection of the lines of sight of the thermography, the depth camera, and the hyperspectral imaging system is a recognition area. One end of the suction tube is connected to a collection box located on the frame, and the other end of the suction tube is fixed to a positioning slider of the lead screw guide rail, and the other end of the suction tube moves vertically along with the positioning slider. The manipulator is fixed to a frame, the manipulator has a laser generator, and the frame further has a control cabinet, the control cabinet is signal-connected to the traveling mechanism, the fan assembly, the slide rail assembly, the recognition picking assembly and the manipulator, and controls the operating state of the entire tea picking machine.

2. The tea bud picking machine based on multimodal recognition by infrared spectrum as described in claim 1, characterized in that the traveling mechanism includes a traveling bracket, the traveling bracket has a traveling wheel and a traveling drive motor for driving the traveling wheel to operate, and the front end of the traveling bracket has two laser radars distributed on the left and right and installed diagonally downward, and the laser radars are used for map creation, positioning and route planning.

3. The tea bud picking machine based on multi-modal recognition by infrared spectrum, as described in claim 1, characterized in that the frame has two motor mounting frames, each of the motor mounting frames has one fan driving motor, each of the fan driving motors has one fan on its output shaft, the air outlets of the two fans are connected to the upper end of the air guide case, and the lower end of the air guide case faces the recognition area.

4. 2. The tea bud picking machine based on multimodal recognition by infrared spectrum according to claim 1, wherein the slide rail assembly includes a vertical slide rail, a lateral slide rail, a vertical slider, and a lateral slider, the lateral slide rail is fixed to a top of a frame, the lateral slider is slidably installed on the lateral slide rail, the vertical slide rail is fixed to the lateral slider and installed perpendicular to the lateral slide rail, the vertical slide rail is slidably installed on the vertical slide rail, the lateral slide rail has a first drive motor for driving and moving the lateral slider, the vertical slide rail has a second drive motor for driving and moving the vertical slider, and the fixed frame is fixedly connected to the vertical slider.

5. The tea bud picking machine based on multimodal recognition by infrared spectrum as claimed in claim 1, characterized in that one end of the suction tube is fixed to the positioning slider via a suction tube fastener.

6. The tea bud picking machine based on multi-modal recognition by infrared spectrum as claimed in claim 1, characterized in that the cross-sectional dimension of the upper end of the air guide case is larger than the cross-sectional dimension of the lower end of the air guide case.

7. (1) A laser radar performs mapping, positioning and route planning, and the whole tea picking machine is advanced according to the planned route by a traveling mechanism; (2) the fan assembly is operated to blow hot air at a constant temperature of 40 degrees Celsius onto the leaf blade at the tip of the tea plant bud through the air guide case, to blow off dirt on the leaf blade at the tip of the bud, and to recognize the new buds by using thermography based on the heating temperature difference between the tip of the bud and the leaf blade; (3) using a hyperspectral imaging system to detect the cellulose content of the sprouts recognized by thermography, and confirming the recognition accuracy of the thermography based on the cellulose content; and performing step (4) after the sprouts are detected as sprouts by the hyperspectral imaging system; (4) recognizing and locating the sprouts using a depth camera; (5) placing the manipulator in position, aligning the laser generator with the picking point on the sprout, and then placing the suction tube in position; (6) the laser generator operates to cut the sprouts, and at the same time, the suction tube operates to suck the cut sprouts into the collection box, completing the picking of the sprouts, and then resetting the suction tube and the manipulator; (7) a step of continuing to perform steps (2) to (6) of the tea picking machine continuing to move forward by the traveling mechanism until all tea picking work is completed.

Images