Tea leaf picking robot and tea leaf positioning picking method

Abstract

The application discloses a kind of tea leaf recognition and picking technical field's tea leaf picking robot and tea leaf positioning picking method, to solve the problem of high tea leaf damage rate, low picking accuracy in prior art.It includes: by illuminating light source to tea plant to reduce environmental interference, image recognition module can improve tea leaf picking accuracy by introducing the recognition of the depth distance of tea plant by depth camera, three degrees of freedom suction head moving device controls vacuum suction head to pick tea leaves, which can reduce the damage to tea quality during picking process;While the tea leaf positioning picking method provided by the application measures the depth distance of the coordinates of the pixels containing tea leaf features, and calculates the three-dimensional coordinates of the tea leaf in the world coordinate system, according to the three-dimensional coordinates, the part to be picked can be accurately positioned in the tea plant, avoiding the interference of the remaining part of the tea plant, and the picking accuracy is high.

Description

Technical Field

[0001] This invention relates to a tea-picking robot and a tea-positioning and picking method, belonging to the field of tea identification and picking technology. Background Technology

[0002] The emergence and application of tea-picking robots have changed the traditional manual picking method, improving picking efficiency, improving the picking environment, and reducing the intensity of manual labor. However, unlike picking brightly colored and shaped fruits such as apples, oranges, and strawberries, the growing environment of tea is complex and variable. The tea leaves to be picked are roughly the same color as the rest of the tea plant. Interference from the environmental background, branches, leaves, and variable natural light all make it difficult to identify and pick the tea leaves. Current tea-picking robots cannot accurately locate and pick the tea leaves. Secondly, tea leaves are fragile, and the current picking method using robotic arms can easily reduce the quality of the tea leaves. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a tea-picking robot with low tea damage rate and high picking accuracy, as well as a tea-positioning and picking method.

[0004] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0005] In a first aspect, the present invention provides a tea-picking robot.

[0006] It includes a walking device for walking along the distribution of tea plantations and a frame that passes over the surface of the tea plants. The frame is equipped with a light source and a depth camera facing the tea plants, and also includes an image recognition module that is signal-connected to the depth camera.

[0007] It also includes a three-degree-of-freedom suction head moving device connected to the frame and a vacuum suction head movably connected to the three-degree-of-freedom suction head moving device, as well as a suction head moving device control module that is signal-connected to the three-degree-of-freedom suction head moving device, the vacuum suction head and the image recognition module;

[0008] The depth camera is used to acquire image information of the tea plant and perform depth distance measurement. The image recognition module is used to identify the coordinates of pixels containing tea plant features and calculate the three-dimensional coordinates of the tea plant. The suction head moving device control module is used to control the position of the vacuum suction head and control the opening and closing of the vacuum suction head according to the three-dimensional coordinates of the tea plant.

[0009] In some embodiments of the first aspect, the three-degree-of-freedom suction head moving device includes a front and rear slide rail, a lateral slide rail slidably connected to the front and rear slide rail, and a vertical slide rail slidably connected to the lateral slide rail. The front and rear slide rails are distributed along the walking direction of the walking device, and the vertical slide rails are distributed vertically along the ground. The distribution of the lateral slide rails, the front and rear slide rails, and the vertical slide rails is perpendicular to each other.

[0010] It also includes a front and rear drive device for driving the lateral slide rail to slide on the front and rear slide rails, a lateral drive device for driving the vertical slide rail to slide on the lateral slide rails, and a vertical drive device for driving the vacuum suction head to slide on the vertical slide rails.

[0011] In some embodiments of the first aspect, the vacuum suction head includes an air pump mounting plate connected to a three-degree-of-freedom suction head moving device, an air pump connected to the air pump mounting plate, and a second L-shaped connecting plate. The normal of the mounting surface of the second L-shaped connecting plate is parallel to the vertical slide rail. The mounting surface is connected to a connecting chamber. The connecting chamber is provided with a suction head air inlet and a suction nozzle facing the tea plant. The connecting chamber is also provided with an air exchange valve for controlling the opening and closing of the suction nozzle. The air exchange valve is signal-connected to the suction head moving device control module.

[0012] The air pump is connected to the connecting chamber, and the air pump has an air pump outlet connected to the tea storage device. The air pump generates airflow from the connecting chamber to the air pump outlet.

[0013] In some embodiments of the first aspect, the air pump is connected to the connecting chamber via a first delivery pipe, which is fitted into the cable chain.

[0014] In some embodiments of the first aspect, the vacuum suction head further includes a tea storage tank connected to the air pump mounting plate, the tea storage tank being in communication with both the air pump and the connecting chamber.

[0015] In some embodiments of the first aspect, the light sources are symmetrically distributed around the depth camera along the walking direction of the walking device, and the positions of the light sources and the depth camera are located on the central axis of the frame along the walking direction of the walking device.

[0016] Secondly, the present invention also provides a method for tea leaf positioning and harvesting using a tea-picking robot, comprising:

[0017] Step A: Obtain image information of the tea plant;

[0018] Step B: Locate the coordinates of pixels containing tea leaf features from the image information;

[0019] Step C: Measure the depth distance of the pixels containing tea leaf features;

[0020] Step D: Normalize the coordinates and depth distance of pixels containing tea leaf features into the three-dimensional coordinates of tea leaves in the camera coordinate system;

[0021] Step E: Convert the three-dimensional coordinates of the tea leaves in the camera coordinate system to the three-dimensional coordinates of the tea leaves in the world coordinate system;

[0022] Step F: Harvest the tea leaves according to their three-dimensional coordinates in the world coordinate system.

[0023] In some embodiments of the first aspect, step B: finding the coordinates of pixels containing tea leaf features from the image information includes:

[0024] The pixels of the acquired tea plant image information are binarized;

[0025] The YOLOX-Nano model network, which uses a convolutional neural network as its backbone, finds pixels containing tea-like features from image information and adds depthwise separable convolutions for assistance.

[0026] Convolutional neural networks include channel attention mechanisms and spatial attention mechanisms;

[0027] The feature map formula output by the channel attention mechanism is:

[0028] The formula for the feature map output by the spatial attention mechanism is:

[0029] In the formula, F is the input feature matrix, F' is the feature map after processing by the channel attention mechanism, and M... c For channel compression weight matrix, Here, F" is the operator for element-wise multiplication of matrices, F" is the feature matrix output by the spatial attention mechanism, and M is the matrix multiplied by M. s It is a spatially compressed weight matrix.

[0030] In some embodiments of the first aspect, step D: normalizing the coordinates and depth distances of pixels containing tea leaf features to three-dimensional coordinates of the tea leaves in the camera coordinate system based on the intrinsic parameter matrix of the depth camera (2), including:

[0031] The following formula normalizes the coordinates and depth distance of pixels containing tea leaf features to the 3D coordinates of tea leaves in the camera coordinate system:

[0032]

[0033] in:

[0034]

[0035] m = [u, v]T ;

[0036]

[0037] M = [X, Y, Z]T;

[0038]

[0039] Z = d;

[0040] In the formula, m represents the coordinates of the pixel containing the tea leaf feature, and M represents the three-dimensional coordinates of the tea leaf in the camera coordinate system. Let m be the homogeneous coordinates. Let M be the homogeneous coordinates, K[Rt] be the intrinsic parameter matrix of the depth camera, K be a third-order matrix, R be the rotation matrix, γ be the radial distortion parameter, t be the translation matrix, s be the normalized scale factor, α and β be the fusion of focal length and pixel aspect ratio, u and v be the coordinates of pixels containing tea leaf features, T be the transpose matrix, X, Y and Z be the three-dimensional coordinates of the tea leaf in the camera coordinate system, and d be the depth distance.

[0041] In some embodiments of the first aspect, step E involves converting the three-dimensional coordinates of the tea leaves in the camera coordinate system to three-dimensional spatial coordinates in the world coordinate system based on the pose information of the depth camera (2), including:

[0042] Attitude information includes an extrinsic parameter matrix, which is calculated using the following formula:

[0043]

[0044] In the formula, H is the Homographic matrix, λ is the extrinsic parameter matrix of the depth camera, K[Rt] is the intrinsic parameter matrix of the depth camera, K is a third-order matrix, R represents the rotation matrix, and t represents the translation matrix;

[0045] The formula for converting the three-dimensional coordinates of tea leaves in the camera coordinate system to three-dimensional spatial coordinates in the world coordinate system is as follows:

[0046] N=[U, V, W]=λ[X, Y, Z];

[0047] Z = d;

[0048] In the formula, N represents the three-dimensional spatial coordinates of the tea leaves in the world coordinate system, U, V, and W represent the three-dimensional spatial coordinate values ​​of the tea leaves in the world coordinate system, X, Y, and Z represent the three-dimensional coordinate values ​​of the tea leaves in the camera coordinate system, and d represents the depth distance.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0050] The tea-picking robot provided by this invention reduces environmental interference by illuminating the tea plants with a light source. The image recognition module, through a depth camera, identifies the depth and distance of the tea plants, improving the tea-picking accuracy of the three-degree-of-freedom suction head movement device. The three-degree-of-freedom suction head movement device controls the vacuum suction head to pick the tea leaves, reducing damage to the tea quality during the picking process. Furthermore, the tea-picking positioning method provided by this invention measures the depth and distance of pixels containing tea-like features, and then calculates the three-dimensional coordinates of the tea leaves in the world coordinate system. Picking tea leaves based on these three-dimensional coordinates allows for precise positioning of the part to be picked within the tea plant, avoiding interference from similar colors and shapes of other parts of the plant, resulting in high picking accuracy. Attached Figure Description

[0051] Figure 1 This is a side view structural diagram of the tea-picking robot provided in an embodiment of the present invention;

[0052] Figure 2 yes Figure 1 The front view;

[0053] Figure 3 yes Figure 1 A three-dimensional image;

[0054] Figure 4 This is a schematic diagram of the vacuum suction head of the tea-picking robot provided in Embodiment 2 of the present invention;

[0055] Figure 5 yes Figure 4 A schematic diagram of a medium vacuum suction head fitted with a tea storage tank;

[0056] Figure 6 This is a flowchart of the tea leaf location and picking method provided in the embodiments of the present invention;

[0057] In the picture:

[0058] 1-Light source;

[0059] 2-Depth camera;

[0060] 3-Three-degree-of-freedom suction head moving device; 3.1-Vacuum suction head;

[0061] 4-Power supply;

[0062] 5-Traversing device; 5.1-Power box; 5.2-Drive chain; 5.3-Track wheels;

[0063] 6- Control module for the suction head moving device; 6.1- PLC control cabinet;

[0064] 7-Image recognition module; 7.1-Industrial control computer;

[0065] 8-Air pump outlet; 9-Air pump; 10-Air pump mounting plate; 11-Drag chain; 11.1-Nylon drag chain; 12-Cylinder head connecting plate; 13-First connecting plate; 14-Second L-shaped connecting plate; 14.1-Mounting surface; 15-Suction head air inlet; 16-Suction nozzle; 17-Air exchange valve; 17.1-Hand-operated air exchange valve;

[0066] 18-Front and rear slide rails; 18.1-Front and rear drive units; 19-Side slide rails; 19.1-Side drive units; 20-Vertical slide rails; 20.1-Vertical drive units;

[0067] 21-First conveying pipe; 22-Tea storage tank; 24-Second conveying pipe; 25.1-First tea outlet; 25.2-Second tea outlet; 26-Connecting chamber;

[0068] 23-Frame. Detailed Implementation

[0069] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0070] Example 1

[0071] Existing technologies already include robots for harvesting fruit. The shape of the fruit differs greatly from the surrounding leaves and branches, and it often has a bright color. The harvesting robots are designed based on two-dimensional recognition hardware and algorithms to address the shape and color differences between the fruit and the environment. However, tea harvesting is different from fruit harvesting. The part of the tea plant to be harvested is generally the tender buds of the tea leaves. The shape and color of the tea leaves to be harvested are very similar to the rest of the tea plant. Existing tea harvesting robots, which are improvements on fruit harvesting robots, have difficulty accurately locating the tea leaves after completing image recognition, and are prone to picking the wrong tea leaves. In addition, existing tea harvesting robots often use robotic arms or robotic fingers for harvesting, which can easily damage the tea leaves and lead to a decline in quality.

[0072] This embodiment provides a tea-picking robot and a tea-positioning and picking method to solve the problems of high tea damage rate and low picking accuracy in the prior art.

[0073] refer to Figure 1 , Figure 2 , Figure 3The tea-picking robot provided in this embodiment includes a walking device 5 for walking along the distribution of tea plantations and a frame 23 for passing over the surface of the tea plants. The frame 23 is equipped with a light source 1 facing the tea plants and a depth camera 2. In this embodiment, the light source 1 shines obliquely downwards, and the depth camera 2 has a downward viewing angle. The light source 1 can provide stable illumination for the tea plants to improve the recognition rate of the color and shape of the tea leaves in the tea plants by the depth camera 2. It also includes an image recognition module 7 that is signal-connected to the depth camera 2. The image recognition module 7 is used to process the image information and depth distance measured by the depth camera 2, and can also control the depth camera 2 to measure the depth distance of specific pixel areas.

[0074] It also includes a three-degree-of-freedom suction head moving device 3 connected to the frame 23 and a vacuum suction head 3.1 movably connected to the three-degree-of-freedom suction head moving device 3. The three-degree-of-freedom suction head moving device 3 drives the vacuum suction head 3.1 to move in three-dimensional space. In different embodiments, it can be a multi-joint robotic arm or a multi-slide rail robotic arm. It also includes a suction head moving device control module 7 that is signal-connected to the three-degree-of-freedom suction head moving device 3, the vacuum suction head 3.1 and the image recognition module 7.

[0075] The depth camera 2 is used to acquire image information of the tea plant and perform depth distance measurement. The image recognition module 7 is used to identify the coordinates of pixels containing tea plant features and calculate the three-dimensional coordinates of the tea plant. The suction head movement device control module 7 is used to control the position of the three-degree-of-freedom suction head movement device 3 to move the vacuum suction head 3.1 and control the opening and closing of the vacuum suction head 3.1 according to the three-dimensional coordinates of the tea plant to be picked from the image recognition module 7. The suction head movement device control module 7 controls the three-dimensional movement of the vacuum suction head 3.1 and controls the opening and closing of the vacuum suction head 3.1 according to the three-dimensional coordinates of the tea plant to be picked. The vacuum suction head 3.1 can effectively reduce the damage rate of the tea plant.

[0076] refer to Figure 6 This embodiment also provides a method for targeted tea picking, including:

[0077] Step A: Acquire image information of tea plants. In this embodiment, the walking device 5 walks so that the frame 23 sweeps across the surface of different tea plants, and the depth camera 2 acquires image information of the tea plants and transmits the image information signal to the depth camera.

[0078] Step B: Find the coordinates of pixels containing tea leaf features from the image information. In this embodiment, the depth camera 2 transmits the image information signal of the tea plant to the image recognition module 7. The image recognition module 7 identifies pixels containing tea leaf features in the image information and extracts the coordinates of the pixels containing tea leaf features.

[0079] Step C: Measure the depth distance of the coordinates of the pixels containing tea leaf features. In this embodiment, the image recognition module 7 extracts the coordinates of the pixels containing tea leaf features and transmits the signal to the depth camera 2. The image recognition module 7 then controls the depth camera 2 to measure the depth distance of the pixels containing tea leaf features.

[0080] Step D: Normalize the coordinates and depth distance of the pixels containing tea leaf features into three-dimensional coordinates of the tea leaf in the camera coordinate system. In this embodiment, the depth camera 2 transmits the measured depth distance signal to the image recognition module 7. The image recognition module 7 normalizes the coordinates of the pixels containing tea leaf features and their corresponding depth distance into three-dimensional coordinates in the camera coordinate system.

[0081] Step E: Convert the three-dimensional coordinates of the tea leaves in the camera coordinate system to the three-dimensional coordinates of the tea leaves in the world coordinate system. In this embodiment, this step is implemented by the image recognition module 7.

[0082] Step F: Pick the tea leaves according to their three-dimensional coordinates in the world coordinate system. In this embodiment, the image recognition module 7 transmits the three-dimensional coordinate signal of the tea leaves in the world coordinate system to the suction head moving device control module 7. According to the three-dimensional coordinates of the tea leaves in the world coordinate system, the suction head moving device control module 7 controls the three-degree-of-freedom suction head moving device 3 to move the vacuum suction head 3.1 to the side of the tea leaves to be picked. The suction head moving device control module 7 controls the vacuum suction head 3.1 to suck up the tea leaves to be picked.

[0083] After the tea leaves are picked, the walking device 5 drives the tea picking robot to the next tea plant to be picked.

[0084] Compared to existing technologies, the tea-picking robot and tea-positioning picking method provided in this embodiment reduce the interference of ambient light on the depth camera 2 by introducing the light source 1. By introducing a technical solution to obtain three-dimensional coordinates by measuring the depth distance of the tea leaves to be picked, the accuracy of picking is improved. The vacuum suction picking method reduces the loss of tea leaves.

[0085] Example 2

[0086] This embodiment provides a tea-picking robot and a tea-positioning and picking method, which improves and optimizes the tea-picking robot and tea-positioning and picking method provided in Embodiment 1. In addition to the technical solution of Embodiment 1, it also includes:

[0087] refer to Figure 2 and Figure 3In this embodiment, the three-degree-of-freedom suction head moving device 3 includes a front and rear slide rail 18, a lateral slide rail 19 slidably connected to the front and rear slide rail 18, and a vertical slide rail 20 slidably connected to the lateral slide rail 19. The front and rear slide rail 18 are distributed along the walking direction of the walking device 5, and the vertical slide rail 20 is distributed vertically along the ground. The distribution of the lateral slide rail 19, the front and rear slide rail 18, and the vertical slide rail 20 is perpendicular to each other. It also includes a front and rear driving device 18.1 for driving the lateral slide rail 19 to slide on the front and rear slide rail 18, a lateral driving device 19.1 for driving the vertical slide rail 20 to slide on the lateral slide rail 19, and a vertical driving device 20.1 for driving the vacuum suction head 3.1 to slide on the vertical slide rail 20.

[0088] Specifically, a gantry-shaped frame 23 can be used. The legs of the frame 23 are connected to the walking device 5 for support. A front and rear slide rail 18 is provided on each side of the upper inner surface of the frame 23 along the walking direction of the walking device 5. The two front and rear slide rails 18 are parallel to each other and remain horizontal to the ground. A lateral slide rail 19 is connected to front and rear drive devices 18.1 at both ends. The lateral slide rail 19 is also horizontal to the ground and perpendicular to the front and rear slide rails 18. The front and rear drive devices 18.1 on the front and rear slide rails 18 drive the lateral slide rail 19 to move forward and backward. The slide rail 18 moves along the traveling direction of the traveling device 5 to adjust the front and rear position of the vacuum suction head 3.1 along the traveling direction of the traveling device 5; the vertical slide rail 20 is connected to the lateral drive device 19.1, and the vertical slide rail 20 remains perpendicular to the ground and also perpendicular to the front and rear slide rails 18 and the lateral slide rail 19. The lateral drive device 19.1 on the lateral slide rail 19 drives the vertical slide rail 20 to slide along the sides of the frame 23 to adjust the position of the vacuum suction head 3.1 in the lateral directions perpendicular to the traveling direction of the traveling device 5; Reference Figure 4 The vacuum suction head 3.1 is equipped with an air pump mounting plate 10 connected to the vertical drive device 20.1 of the three-degree-of-freedom suction head moving device 3. The vertical drive device 20.1 on the vertical slide rail 20 drives the vacuum suction head 3.1 to move up and down in the vertical direction. It should be noted that the ground in the farmland where the tea is located is rugged. The descriptions of "perpendicular to the ground" and "parallel to the ground" are for better illustration of the technical solution and do not represent the absolute situation in actual use.

[0089] refer to Figure 4The vacuum suction head 3.1 includes an air pump mounting plate 10 connected to the three-degree-of-freedom suction head moving device 3, an air pump 9 connected to the air pump mounting plate 10, and a second L-shaped connecting plate 14. In this embodiment, the air pump mounting plate 10 is connected to the vertical drive device 20.1, and the air pump 9 is installed at the end of the air pump mounting plate 10 facing away from the ground. The normal of the mounting surface 14.1 of the second L-shaped connecting plate 14 is parallel to the vertical slide rail 20, so that the mounting surface 14.1 is parallel to the ground, so that the direction of the suction nozzle 16 is aligned with the tea leaves growing on the top of the tea plant. The suction nozzle 16 generally faces the ground. The mounting surface 14.1 is connected to the connecting chamber 26. The connecting chamber 26 is provided with a suction head air inlet 15 and a suction nozzle 16 facing the tea plant. The connecting chamber 26 connects the suction head air inlet 15 and the suction nozzle 16. Chamber 26 is also equipped with an air exchange valve 17 that controls the opening and closing of the suction nozzle 16. The air exchange valve 17 is connected to the suction head moving device control module 7. The air pump 9 is connected to the connecting chamber 26. The air pump 9 is equipped with an air pump outlet 8 that is connected to the tea storage device. The air pump 9 generates airflow from the connecting chamber 26 to the air pump outlet 8. The tea storage device can be set on the frame 26 and connected to the air pump outlet 8 through the suction pipe so that the tea is finally sent into the tea storage device. When the three-degree-of-freedom suction head moving device 3 drives the vacuum suction head 3.1 to the correct position for sucking up the tea, the air exchange valve 17 opens to suck up the tea. The air pump 9 is normally open, and the suction head air inlet 15 is used to ensure the internal gas flow when the air exchange valve 17 is closed and the tea blocks the vacuum suction head 3.1, so as to ensure that the air pump 9 is not burned out.

[0090] The air pump 9 is connected to the connecting chamber 26 through the first delivery pipe 21. The first delivery pipe 21 is fitted into the drag chain 11. In this way, when the three-degree-of-freedom suction head moving device 3 drives the vacuum suction head 3.1 to move, the drag chain 11 will buffer the deformation and shaking of the first delivery pipe 21. The drag chain 11 is preferably a nylon drag chain 11.1.

[0091] refer to Figure 5The vacuum suction head 3.1 also includes a tea storage tank 22 connected to the air pump mounting plate 10. The tea storage tank 22 is used for tea picking and sampling. The tea storage tank 22 is connected to the air pump 9 and the connecting chamber 26. The tea storage tank 22 and the connecting chamber 26 are connected through a second conveying pipe 24. The second conveying pipe 24 may be slightly larger than the inner diameter of the first conveying pipe 21 to improve throughput. During sampling, the first conveying pipe 21 can be removed or the first tea outlet 25.1 connected to the first conveying pipe 21 on the connecting chamber 26 can be closed, leaving only the second conveying pipe 24 and the second tea outlet 25.1 connected to the second conveying pipe 24 on the connecting chamber 26 open. The second outlet 25.2 is used to draw tea leaves into the tea storage tank 22 under the pressure of the air pump 9. After checking that the tea leaves in the tea storage tank 22 are qualified, the second tea outlet 25.2 is closed or the second delivery pipe 24 is removed from the tea storage tank 22, and the first delivery pipe 21 is reinstalled or the first tea outlet 25.1 connected to the first delivery pipe 21 on the connecting chamber 26 is opened. It should be noted that when only the second delivery pipe 24 or the first delivery pipe 21 is removed, the first tea outlet 25.1 and the second tea outlet 25.2 can be closed or kept open to play a similar role to the suction head air inlet 15.

[0092] refer to Figure 1 The light source 1 is symmetrically distributed around the depth camera 2 along the walking direction of the walking device 5. The light source 1 is preferably long and narrow. The positions of the light source 1 and the depth camera 2 are located on the central axis of the frame 23 along the walking direction of the walking device 5. Preferably, the depth camera 2 is centrally located so that the lighting conditions of the tea leaves within the shooting range of the depth camera 2 are basically consistent.

[0093] refer to Figure 3 The walking device 5 includes a power box 5.1 and track wheels 5.3, which are connected to the power box 5.1 via a transmission chain 5.2. The depth camera 2 can be an Intel RealSense D435 depth camera with a depth field of view of 87°×58° and a depth range of 0.2 to 8m. The light source 1 can be a Lemons KM-BRD49242 light source, symmetrically distributed, illuminating the ground at a 45° angle. The image recognition module 7 can be an industrial control computer 7.1. The second L-shaped connecting plate 14 can be connected to the first connecting plate 13, which is indirectly connected to the air pump 9. The end of the air pump 9 closest to the tea plant and the ground is also equipped with a cylinder head connecting plate 12 for functional expansion. The air exchange valve 17 can be a hand-operated air exchange valve 17.1 with both signal control and manual control.

[0094] In the tea leaf location and picking method of this embodiment, step B:

[0095] The pixels of the acquired tea plant image information are binarized to reduce noise, improve contrast, and facilitate subsequent recognition processing.

[0096] The YOLOX-Nano model network, which uses a convolutional neural network as its backbone, finds pixels containing tea-like features in image information and adds depthwise separable convolutions to significantly reduce the number of convolution parameters, lower computational costs, and maintain accuracy.

[0097] Convolutional neural networks include channel attention mechanisms and spatial attention mechanisms;

[0098] The feature map formula output by the channel attention mechanism is:

[0099] The formula for the feature map output by the spatial attention mechanism is:

[0100] In the formula, F is the input feature matrix, F' is the feature map after processing by the channel attention mechanism, and M... c For channel compression weight matrix, Here, F" is the operator for element-wise multiplication of matrices, F" is the feature matrix output by the spatial attention mechanism, and M is the matrix multiplied by M. s It is a spatially compressed weight matrix.

[0101] Additionally, in step D: based on the intrinsic parameter matrix of depth camera 2, the coordinates and depth distance of pixels containing tea leaf features are normalized to the three-dimensional coordinates of tea leaves in the camera coordinate system.

[0102] The following formula normalizes the coordinates and depth distance of pixels containing tea leaf features to the 3D coordinates of tea leaves in the camera coordinate system:

[0103]

[0104] in:

[0105]

[0106] m = {u, v] T ;

[0107]

[0108] M = [X, Y, Z] T ;

[0109]

[0110] Z = d;

[0111] In the formula, m represents the coordinates of the pixel containing the tea leaf feature, and M represents the three-dimensional coordinates of the tea leaf in the camera coordinate system. Let m be the homogeneous coordinates. Let M be the homogeneous coordinates, K[Rt] be the intrinsic parameter matrix of depth camera 2, K be a third-order matrix, R represent the rotation matrix, t represent the translation matrix, s be the normalized scale factor, α and β be the fusion of focal length and pixel aspect ratio, u and v be the coordinate values ​​of pixels containing tea leaf features, T be the transpose matrix, X, Y and Z be the three-dimensional coordinate values ​​of tea leaves in the camera coordinate system, d be the depth distance, and γ be the radial distortion parameter.

[0112] In step E: based on the pose information of depth camera 2, the three-dimensional coordinates of the tea leaves in the camera coordinate system are converted into three-dimensional spatial coordinates in the world coordinate system;

[0113] Attitude information includes an extrinsic parameter matrix, which is calculated using the following formula:

[0114]

[0115] In the formula, H is the Homographic matrix, and λ is the extrinsic matrix of depth camera 2. The extrinsic matrix describes the position and orientation of depth camera 2 in three-dimensional space, also known as camera pose or camera position. It consists of rotation matrix and translation vector, and is used to align the camera coordinate system with the world coordinate system. Specifically, the extrinsic matrix can transform the camera coordinate system (e.g., pixel coordinate system) with the world coordinate system (e.g., geographic coordinate system).

[0116] The formula for converting the three-dimensional coordinates of tea leaves in the camera coordinate system to three-dimensional spatial coordinates in the world coordinate system is as follows:

[0117] N=[U, V, W]=λ[X, Y, Z];

[0118] In the formula, N represents the three-dimensional spatial coordinates of the tea leaves in the world coordinate system, and U, V, and W represent the three-dimensional spatial coordinate values ​​of the tea leaves in the world coordinate system.

[0119] The suction head moving device control module 6 can be a PLC control cabinet 6.1. The PLC control cabinet 6.1 can control the movement of the three-degree-of-freedom suction head moving device 3 according to the three-dimensional spatial coordinates of the tea leaves in the world coordinate system obtained from the industrial computer 7.1. The communication protocol between the PLC control cabinet 6.1 and the industrial computer 7.1 adopts the Mitsubishi MC communication protocol. Compared with the Modbus TCP protocol, which has unstable transmission limitations, the Mitsubishi MC communication protocol broadens the application scenarios. In step F:

[0120] F1: The industrial computer 7.1 accesses bit D108 of the PLC control cabinet 6.1. When it reads that bit D108 is 1, the industrial computer 7.1 sets bit D100 of the PLC control cabinet 6.1 to 1 and opens the communication connection.

[0121] F2: Send the three-dimensional spatial coordinates of the tea leaves in the world coordinate system to bits D102, D104 and D106 of PLC control cabinet 6.1 respectively, and set bit D108 of PLC control cabinet 6.1 to 0.

[0122] F3: After receiving the three-dimensional spatial coordinates of the tea leaves in the world coordinate system, the PLC control cabinet 6.1 sets the D100 position to 0 and starts controlling the three-degree-of-freedom suction head moving device 3 and the vacuum suction head 3.1 to pick the tender tea buds.

[0123] F4: After harvesting, set D108 to 1 and continue the F1 operation.

[0124] In this embodiment, each component is powered by power supply 4. PLC control cabinet 6.1 and depth camera 2 are connected to industrial computer 7.1 via USB cable. PLC control cabinet 6.1 can be used to detect the status and position of the three-degree-of-freedom suction head moving device 3. Industrial computer 7.1 can realize the functions of autonomously planning the path of walking device 5, processing the compensation data of PLC control cabinet 6.1, and sending the data information of the three-dimensional coordinates of tea leaves in the world coordinate system. Depth camera 2 can be a 3D camera.

[0125] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0126] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "connected," "linked," "located in," "equipped with," "located in," "installed," "set," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0127] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A tea-picking robot, characterized in that, It includes a walking device (5) for walking along the distribution of tea plantations and a frame (23) for passing over the surface of the tea plantations. The frame (23) is equipped with a light source (1) facing the tea plantations and a depth camera (2). It also includes an image recognition module (7) that is signal-connected to the depth camera (2). It also includes a three-degree-of-freedom suction head moving device (3) connected to the frame (23) and a vacuum suction head (3.1) movably connected to the three-degree-of-freedom suction head moving device (3), and a suction head moving device control module (6) that is signal-connected to the three-degree-of-freedom suction head moving device (3), the vacuum suction head (3.1) and the image recognition module (7). The depth camera (2) is used to acquire image information of tea plants and perform depth distance measurement. The image recognition module (7) is used to identify the coordinates of pixels containing tea plant features and calculate the three-dimensional coordinates of the tea plant. The suction head moving device control module (6) is used to control the three-degree-of-freedom suction head moving device (3) to move the position of the vacuum suction head (3.1) and control the opening and closing of the vacuum suction head (3.1) according to the three-dimensional coordinates of the tea plant. Step A: Acquire image information of tea plants. The walking device (5) walks and makes the frame (23) sweep across the surface of different tea plants. The depth camera (2) acquires image information of tea plants and transmits the image information signal to the image recognition module (7). Step B: Find the coordinates of the pixels containing tea plant features from the image information. The depth camera (2) transmits the image information signal of the tea plant to the image recognition module (7). The image recognition module (7) identifies the pixels containing tea plant features in the image information and extracts the coordinates of the pixels containing tea plant features. The step of extracting the coordinates of pixels containing tea-like features includes: The pixels of the acquired tea plant image information are binarized; The YOLOX-Nano model network, which uses a convolutional neural network as its backbone, finds pixels containing tea-like features from image information and adds depthwise separable convolutions for assistance. Convolutional neural networks include channel attention mechanisms and spatial attention mechanisms; The feature map formula output by the channel attention mechanism is: ; The formula for the feature map output by the spatial attention mechanism is: ; In the formula, F is the input feature matrix. The feature map after processing by the channel attention mechanism. For channel compression weight matrix, This is the operator for multiplying matrix elements in sequence. The feature matrix output by the spatial attention mechanism. This is the spatial compression weight matrix; Step C: Depth distance measurement is performed on the coordinates of pixels containing tea leaf features. The image recognition module (7) extracts the coordinates of pixels containing tea leaf features and transmits the signal to the depth camera (2). The image recognition module (7) controls the depth camera (2) to perform depth distance measurement on the coordinates of pixels containing tea leaf features. Step D: Normalize the coordinates and depth distance of the pixels containing tea leaf features into the three-dimensional coordinates of the tea leaf in the camera coordinate system. The depth camera (2) transmits the measured depth distance signal to the image recognition module (7). The image recognition module (7) normalizes the coordinates of the pixels containing tea leaf features and their corresponding depth distance into the three-dimensional coordinates in the camera coordinate system. Step E: Convert the three-dimensional coordinates of the tea leaves in the camera coordinate system to the three-dimensional coordinates of the tea leaves in the world coordinate system. This step is implemented by the image recognition module (7). Step F: Pick tea leaves according to the three-dimensional coordinates of the tea leaves in the world coordinate system. The image recognition module (7) transmits the three-dimensional coordinate signal of the tea leaves in the world coordinate system to the suction head moving device control module (6). According to the three-dimensional coordinates of the tea leaves in the world coordinate system, the suction head moving device control module (6) controls the three-degree-of-freedom suction head moving device (3) to move the vacuum suction head (3.1) to the side of the tea leaves to be picked. The suction head moving device control module (6) controls the vacuum suction head (3.1) to suck up the tea leaves to be picked.

2. The tea-picking robot according to claim 1, characterized in that, The three-degree-of-freedom suction head moving device (3) includes a front and rear slide rail (18), a lateral slide rail (19) slidably connected to the front and rear slide rail (18), and a vertical slide rail (20) slidably connected to the lateral slide rail (19). The front and rear slide rail (18) are distributed along the walking direction of the walking device (5), and the vertical slide rail (20) is distributed along the vertical direction of the ground. The distribution of the lateral slide rail (19), the front and rear slide rail (18), and the vertical slide rail (20) is perpendicular to each other. It also includes a front and rear drive device (18.1) for driving the lateral slide rail (19) to slide on the front and rear slide rail (18), a lateral drive device (19.1) for driving the vertical slide rail (20) to slide on the lateral slide rail (19), and a vertical drive device (20.1) for driving the vacuum suction head (3.1) to slide on the vertical slide rail (20).

3. The tea-picking robot according to claim 2, characterized in that, The vacuum suction head (3.1) includes an air pump mounting plate (10) connected to the three-degree-of-freedom suction head moving device (3), an air pump (9) connected to the air pump mounting plate (10), and a second L-shaped connecting plate (14). The normal of the mounting surface (14.1) of the second L-shaped connecting plate (14) is parallel to the vertical slide rail (20). The mounting surface (14.1) is connected to the connecting chamber (26). The connecting chamber (26) is provided with a suction head air inlet (15) and a suction nozzle (16) facing the tea plant. The connecting chamber (26) is also provided with an air exchange valve (17) for controlling the opening and closing of the suction nozzle (16). The air exchange valve (17) is signal connected to the suction head moving device control module (6). The air pump (9) is connected to the connecting chamber (26), and the air pump (9) is provided with an air pump outlet (8) connected to the tea storage device. The air pump (9) generates airflow from the connecting chamber (26) to the air pump outlet (8).

4. The tea-picking robot according to claim 3, characterized in that, The air pump (9) is connected to the connecting chamber (26) through a first delivery pipe (21), which is inserted into the cable chain (11).

5. The tea-picking robot according to claim 3, characterized in that, The vacuum suction head (3.1) also includes a tea storage tank (22) connected to the air pump mounting plate (10), and the tea storage tank (22) is connected to the air pump (9) and the connecting chamber (26) respectively.

6. The tea-picking robot according to claim 1, characterized in that, The light source (1) is symmetrically distributed around the depth camera (2) along the walking direction of the walking device (5), and the positions of the light source (1) and the depth camera (2) are located on the central axis of the frame (23) along the walking direction of the walking device (5).

7. A method for tea leaf positioning and harvesting using a tea-picking robot, characterized in that, include: Step A: Obtain image information of the tea plant; Step B: Locate the coordinates of pixels containing tea leaf features from the image information; including: The pixels of the acquired tea plant image information are binarized; The YOLOX-Nano model network, which uses a convolutional neural network as its backbone, finds pixels containing tea-like features from image information and adds depthwise separable convolutions for assistance. Convolutional neural networks include channel attention mechanisms and spatial attention mechanisms; The feature map formula output by the channel attention mechanism is: ; The formula for the feature map output by the spatial attention mechanism is: ; In the formula, F is the input feature matrix. The feature map after processing by the channel attention mechanism. For channel compression weight matrix, This is the operator for multiplying matrix elements in sequence. The feature matrix output by the spatial attention mechanism. This is the spatial compression weight matrix; Step C: Measure the depth distance of the pixels containing tea leaf features; Step D: Normalize the coordinates and depth distance of pixels containing tea leaf features into the three-dimensional coordinates of tea leaves in the camera coordinate system; Step E: Convert the three-dimensional coordinates of the tea leaves in the camera coordinate system to the three-dimensional coordinates of the tea leaves in the world coordinate system; Step F: Harvest the tea leaves according to their three-dimensional coordinates in the world coordinate system.

8. The tea-picking robot's tea-positioning and picking method according to claim 7, characterized in that, Step D: Normalize the coordinates and depth distance of pixels containing tea leaf features into the three-dimensional coordinates of the tea leaves in the camera coordinate system, including: The following formula normalizes the coordinates and depth distance of pixels containing tea leaf features to the 3D coordinates of tea leaves in the camera coordinate system: ； in: K= ； m= ； = ； M= ； ； Z=d; In the formula, m represents the coordinates of the pixel containing the tea leaf feature, and M represents the three-dimensional coordinates of the tea leaf in the camera coordinate system. Let m be the homogeneous coordinates. Let M be the homogeneous coordinates. Let K be the intrinsic parameter matrix of the depth camera, R be the rotation matrix, γ be the radial distortion parameter, t be the translation matrix, and s be the normalized scale factor. and This refers to the fusion of focal length and pixel aspect ratio, respectively. and , where are the coordinates of the pixels containing tea leaf features, T represents the transpose matrix, X, Y and Z are the three-dimensional coordinates of the tea leaf in the camera coordinate system, and d is the depth distance.

9. The tea-picking robot's tea-positioning and picking method according to claim 7, characterized in that, In step E: the three-dimensional coordinates of the tea leaves in the camera coordinate system are converted to three-dimensional spatial coordinates in the world coordinate system, including: Attitude information includes an extrinsic parameter matrix, which is calculated using the following formula: =λ; In the formula, H is the Homographic matrix, and λ is the extrinsic parameter matrix of the depth camera. Let K be the intrinsic parameter matrix of the depth camera, K be a third-order matrix, R be the rotation matrix, and t be the translation matrix. The formula for converting the three-dimensional coordinates of tea leaves in the camera coordinate system to three-dimensional spatial coordinates in the world coordinate system is as follows: ； Z=d; In the formula, N represents the three-dimensional spatial coordinates of the tea leaves in the world coordinate system, U, V, and W represent the three-dimensional spatial coordinate values ​​of the tea leaves in the world coordinate system, X, Y, and Z represent the three-dimensional coordinate values ​​of the tea leaves in the camera coordinate system, and d represents the depth distance.

Images