Method for acquiring optimal grabbing pose of plug seedling

Abstract

The invention discloses a method for acquiring an optimal grabbing pose of a plug seedling. The method depends on a two-finger clamping jaw of graded transplanting equipment. The two-finger clamping jaw is positioned above the hole tray; the method comprises the following steps: 1, representing a grabbing pose: # imgabs0 # 2, constructing a data set: shooting a hole tray image from a vertically downward visual angle; the shot images are segmented and zoomed according to plug grids, and pixel-level capture pose labeling is carried out on each image; 3, training a neural network: inputting a required original image into the real-time grabbing detection network, and outputting the grabbing confidence degree, the grabbing angle and the grabbing width of each pixel point in the image by the real-time grabbing detection network; according to the method, the optimal grabbing pose of each seedling in the plug is obtained by fusing machine vision and deep learning, the seedlings are grabbed based on the optimal grabbing poses, leaves and stalks of the seedlings can be effectively avoided, the damage rate in the plug seedling grabbing process is reduced, and the survival rate of the transplanted seedlings is increased.

Description

Technical Field

[0001] This invention relates to the field of target pose recognition technology, and in particular to a method for obtaining the optimal grasping pose of seedlings in a seedling tray. Background Technology

[0002] Plug seedling technology is a modern seedling cultivation technique that uses lightweight soilless materials as the seedling substrate, employs mechanized precision sowing, and produces one seed per hole, resulting in a seedling in one go. It is widely used in the cultivation and production of vegetables, fruits, and flowers. The key step in plug seedling technology is sorting and transplanting the seedlings based on their growth within the plug trays, with the aim of improving the survival rate of the seedlings.

[0003] Currently, the sorting and transplanting of plug seedlings has been largely mechanized, as evidenced by invention patents such as CN103636333A ("Intelligent Seedling Sorting and Transplanting Machine"), CN117322258A ("Plug Seedling Grading and Transplanting System and Method"), CN104798513A ("Plug Seedling Sorting and Transplanting Machine"), and CN108184466A ("An Integrated Machine for Transplanting and Parallel Grafting of Plug Seedlings").

[0004] During operation, the grading and transplanting equipment uses mechanical grippers to grab the seedlings in the holes of the seedling tray. Specifically, the mechanical grippers are inserted into the substrate (the soil clump that surrounds the seedling roots) of the target hole in the seedling tray, and then lifted or moved upwards, thereby separating the substrate along with the whole seedling from the hole.

[0005] The mechanical grippers of the grading transplanting equipment may cause mechanical damage to the seedling leaves during the process of grasping the seedlings in the holes of the seedling tray (the mechanical grippers fall from directly above the seedling leaves, causing the leaves to break or even the stem to break), which in turn affects the survival rate of the seedlings after transplanting.

[0006] Current research on reducing transplanting damage to seedlings in plug trays mainly focuses on the design of mechanical grippers. For example, before grasping the seedlings, the young leaves are first "aggregated" to reduce transplanting damage. Therefore, designing a seedling grasping method based on the technical route of "mechanical grippers avoiding the seedling leaves when performing the grasping operation" is of great significance. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for obtaining the optimal grasping pose of seedlings in plug trays. It integrates machine vision and deep learning methods to obtain the optimal grasping pose of each seedling in the plug tray. Based on the optimal grasping pose, the seedlings can be grasped, which can effectively avoid the leaves and stems of the seedlings, reduce the damage rate during the grasping process of the plug tray seedlings, and improve the survival rate of the seedlings after transplanting.

[0008] The technical solution of this invention is: a method for obtaining the optimal grasping posture of seedlings in a seedling tray, relying on the two-finger gripper of a grading transplanting device; the two-finger gripper is located above the seedling tray; the lower end of the two-finger gripper is provided with two claws for inserting into the substrate, and the opening degree of the two claws is adjusted by opening or closing the action; the two-finger gripper has translational and rotational action modes; the two-finger gripper moves to directly above any position in the seedling tray by translational action, and the two-finger gripper rotates around the vertical axis at any angle by rotational action;

[0009] The method is as follows:

[0010] S01 represents the grasping pose:

[0011] The grasping pose includes coordinates, angle, and width. The coordinates are the coordinates of the grasping center point, which are equivalent to the coordinates of the midpoint of the line connecting the lower ends of the two claws. The angle is the angle between the line connecting the lower ends of the two claws and the horizontal baseline. The width is the length of the line segment formed by the line connecting the lower ends of the two claws.

[0012] Grasping pose in image coordinate system As shown in Equation 1;

[0013] Formula 1:

[0014] Where [u, v] are the coordinates of the pixel in the image coordinate system. The angle in the grasping pose corresponding to the pixel. The width parameter in the grasping pose corresponding to the pixel;

[0015] The grabbing pose g in the world coordinate system is shown in Equations 2 and 3.

[0016] Formula 2:

[0017] Formula 3:

[0018] Equation 2 is obtained from Equation 1 through coordinate transformation, and Equation 3 is obtained from Equation 2 through coordinate transformation, t WC Let t be the transformation matrix between the world coordinate system and the camera coordinate system. CI This is the mapping matrix between the image coordinate system and the camera coordinate system; [x, y, z] are the coordinates of the midpoint of the line connecting the lower endpoints of the two claw bodies in the world coordinate system. The angle between the line connecting the lower ends of the two claws and the horizontal baseline is denoted by , and w is the length of the line segment formed by the line connecting the lower ends of the two claws.

[0019] Since z is a known quantity, equation 3 is simplified to equation 4;

[0020] Formula 4:

[0021] S02, Dataset Construction:

[0022] The camera is fixed directly above the seedling tray, and images of the tray are taken from a vertically downward perspective. The images include each seedling hole in the tray and the seedlings growing from each hole. The captured images are then segmented and scaled according to the seedling tray grid to obtain multiple images of the same resolution to be labeled. Each image to be labeled contains one and only one complete seedling tray grid. Pixel-level grasping pose annotations are performed on each image. The annotation content includes the grasping center, grasping angle, and grasping width. The annotation principles are as follows: I. The grasping center is marked at the junction of the seedling leaf and stem; II. The grasping angle is perpendicular to the line connecting the centroids of the two largest leaves on the seedling; III. The two endpoints of the grasping width are the claw landing points, which are more than 1 cm away from the edge of the seedling tray grid, and the distance between the two claw landing points is not less than 3 cm.

[0023] S03, Training the neural network:

[0024] All labeled images are divided into training and validation sets, and input into a Real-Time Grasping Detection Network (LRGN) for training and testing. The trained LRGN is capable of pixel-level grasping pose annotation of input images. The original image for which the grasping pose needs to be obtained is input into the LRGN for prediction. The LRGN outputs the grasping confidence, grasping angle, and grasping width for each pixel in the image. The pixel with the highest grasping confidence is selected as the grasping center point. The coordinates [x, y] of the grasping center point and the corresponding grasping angle are also specified. The grasping width w corresponding to the grasping center point is the predicted grasping posture of the seedling in the seedling tray.

[0025] A further technical solution of the present invention is as follows: The Real-Time Capture and Detection Network (LRGN) includes an encoder and a decoder; the encoder includes a feature extraction module, a dilation and thinning module, a pixel attention-guided fusion module, a depth-guided fusion module, and a hybrid attention module; after the original image has its features extracted by the feature extraction module, it outputs feature F; feature F is enhanced with global features by the dilation and thinning module, and a branch F with a relatively small receptive field is output. s and the branch F with a relatively large receptive field l F s and F l Two parallel paths are formed; F s In the path, F s F and deep features, respectively, are input to the pixel attention-guided fusion module to learn semantic information and output features. F and deep features, respectively, are input into the depth-guided fusion module to enrich the feature representation, and the output feature F is... S′, F S The input hybrid attention module weights the importance of the input features and outputs feature F. S ";F l In the path, F l F and deep features, respectively, are input to the pixel attention-guided fusion module to learn semantic information and output features. F and deep features, respectively, are input into the depth-guided fusion module to enrich the feature representation, and the output feature F is... l ′, F l The input hybrid attention module weights the importance of the input features and outputs feature F. l The decoder includes a region output head, an angle output head, and a width output head; the region output head receives the characteristic F. S ", outputs the grasp confidence of each pixel; the angle output head and the width output head respectively receive the feature F l "" outputs the grasping angle and grasping width of each pixel.

[0026] A further technical solution of the present invention is as follows: the implementation process of the dilation and thinning module is as follows: after feature F enters the dilation and thinning module, the number of feature channels is first adjusted to 1024 through a 3×3 convolution to obtain feature F′, and then F′ is evenly divided into four groups of feature maps {F along the channel dimension {F 1 F 2 F 3 F 4 Next, a multi-scale feature fusion method is used to add adjacent branch features pixel by pixel. Then, the four sets of results after adding features are input into convolutions with different dilation rates to extract contextual features, resulting in four sets of feature branches. Finally, execute the following two processes respectively: I. Execute feature branches and By concatenating along the channel dimension and then performing a residual connection with the feature F′, a branch F with a relatively small receptive field is obtained. s II. Feature branches and By concatenating along the channel dimension and then performing a residual connection with the feature F′, a branch F with a relatively large receptive field is obtained. l ;

[0027] In the implementation process of the dilation and refinement module, four sets of feature maps {F 1 F 2 F 3 F 4} to four sets of feature branches The operations performed are shown in equations 5, 6, 7, and 8.

[0028] Formula 5:

[0029] Formula 6:

[0030] Formula 7:

[0031] Formula 8:

[0032] In Equation 5, Conv(·) represents a 3×3 convolution with an inflation rate of 1; in Equation 6, Conv(·) represents a 3×3 convolution with an inflation rate of 3; in Equation 7, Conv(·) represents a 3×3 convolution with an inflation rate of 5; and in Equation 8, Conv(·) represents a 3×3 convolution with an inflation rate of 8.

[0033] A further technical solution of the present invention is: the implementation process of the pixel attention-guided fusion module is as follows: the shallow feature F is adjusted to 128 channels by a 1×1 convolution with a dilation rate of 1, and the deep feature F... s-l After adjusting the channel count to 128 using two 1×1 convolutions with a dilation rate of 1, upsampling is performed. Then, the shallow features after channel adjustment are multiplied pixel-by-pixel with the upsampled deep features. The pixel-by-pixel multiplication results are then summed channel-by-channel, and the importance weights of each pixel in the channel-by-channel summed features are obtained using the Sigmoid function (see Equation 9). Finally, the features are output according to Equation 10. Reference or F s-l Refers to F s or F l ;

[0034] Formula 9:

[0035] Formula 10:

[0036] Where F represents shallow features, F s-l For deep features; σ1 is the Sigmoid function in the pixel attention-guided fusion module, whose value represents the pixel importance weight. The higher the value of σ1, the more trusted the pixel. The lower the σ1 value, the greater the trust. The vector corresponding to the pixel in the deep feature is The vector corresponding to the pixel in the shallow features is The output features of the pixel attention-guided fusion module.

[0037] A further technical solution of the present invention is: the implementation process of the deep guided fusion module is as follows: the shallow feature F is adjusted to 256 channels by a 3×3 convolution with an inflation rate of 1, and the deep feature... After upsampling, the number of channels is adjusted to 256 through a 3×3 convolution with a dilation rate of 1; the shallow features after convolution... and deep features after convolution Feature F is obtained by concatenating along the channel dimension. c Then F c The result after inputting the global max pooling layer and F c The results from the input global average pooling layer are summed, and then the sum is passed through a 1×1 convolution with a dilation rate of 1 to adjust the number of channels to 1. The importance weights of each channel in the convolutional features are then obtained using the Sigmoid function. Finally, these weights are combined with the shallow features from the convolution. Multiply, and then combine the result of the multiplication with the shallow features after convolution. The sums are then added together, and finally the sum is combined with the deep features obtained after convolution. The features are concatenated and output as feature F′, where F′ represents F. S ′ or F l ′;

[0038] Shallow features after convolution and deep features after convolution As shown in Equations 11 and 12 respectively; from the shallow features after convolution and deep features after convolution The operations that the deep guided fusion module undergoes to produce the output feature F′ are shown in Equation 13.

[0039] Formula 11:

[0040] Formula 12:

[0041] Formula 13:

[0042] Among them, Conv 3×3 (·) represents a 2D convolution with 256 output channels, a dilation rate of 1, and a kernel size of 3; UP(·) represents an upsampling operation; · represents feature concatenation; GAP(·) represents global average pooling; GMP(·) represents global max pooling; Conv 1×1 (·) represents a two-dimensional convolution with one output channel, an inflation rate of 1, and a kernel size of 1. σ² represents pixel-level multiplication, and σ² is the Sigmoid function in the depth-guided fusion module, whose value represents the channel importance weight.

[0043] A further technical solution of the present invention is: the input features and output features of the hybrid attention module are F′ and F″, respectively, and the operation from F′ to F″ is as shown in Equation 14;

[0044] Formula 14:

[0045] Where F″ refers to F S "and F l ″, α is an adaptive weight with an initial value of 1.0 (adaptively adjusted during training), CAM(·) represents the coordinate attention module, and CA(·) represents the channel attention module.

[0046] A further technical solution of the present invention is: the implementation process of the region output head is: input feature F S "After two 3×3 convolutions with a dilation rate of 1 to adjust the number of channels to 128, then after a 2×2 deconvolution with a dilation rate of 1 for upsampling, and finally after a 1×1 convolution with a dilation rate of 1 and upsampling operation, a grasping point feature map with a channel number of 1 with the same resolution as the original image is obtained. The map contains the grasping confidence of each pixel."

[0047] A further technical solution of the present invention is: the implementation process of the angle output head is: input feature F l "After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 256. Then, after a 2×2 deconvolution with a dilation rate of 1, upsampling is performed. Finally, after a 1×1 convolution with a dilation rate of 1 and upsampling, a grasping point feature map with k channels and the same resolution as the original image is obtained. The map contains the channel k to which the grasping angle of each pixel belongs. The grasping angle can be calculated based on the value of k."

[0048] A further technical solution of the present invention is: the implementation process of the width output head is: input feature F l "After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 128. Then, after a 2×2 deconvolution with a dilation rate of 1, upsampling is performed. Finally, after a 1×1 convolution with a dilation rate of 1 and upsampling operation, a grasping point feature map with a channel number of 1 with the same resolution as the original image is obtained. The map contains the grasping width of each pixel."

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] 1. It provides a method for obtaining the optimal grasping pose of seedlings in plug trays. It integrates machine vision and deep learning methods to obtain the optimal grasping pose of each seedling in the plug tray. Based on the optimal grasping pose, the seedlings can be grasped, which can effectively avoid the leaves and stems of the seedlings, reduce the damage rate of seedlings in plug trays during the grasping process, and improve the survival rate of seedlings after transplanting.

[0051] 2. The grasping pose detected by the Real-Time Grasp Detection Network (LRGN) can better locate the seedling core, and the landing point of its gripper can avoid the seedling body and the edge of the seed tray, thus achieving low-damage grasping.

[0052] 3. Compared to existing networks, the Real-Time Capture Detection Network (LRGN) achieves extremely high detection accuracy (98.83%) on the seedling capture dataset, with an FPS of up to 113 images / second, a weight size of only 46.3MB (which can significantly reduce model loading time and inference latency), and only 12.76M parameters. It has excellent overall performance, balancing detection accuracy and speed while being relatively easy to port to edge computing devices such as mobile phones, embedded systems, and IoT devices.

[0053] 4. The Real-Time Grasp Detection Network (LRGN) shows that while the accuracy decreases at higher Jaccard coefficients compared to lower Jaccard coefficients as the Jaccard coefficient increases from 25% to 35%, it can still accurately determine the grasp pose in most cases even when the Jaccard index of the ground truth and predicted grasp region is set at 35%, demonstrating strong robustness.

[0054] 5. When the Real-Time Crawling Detection Network (LRGN) is applied to the Cornell Crawling Dataset (excluding seedling trays), it achieves high accuracy rates of 98.96% and 98.30% in both image segmentation and object segmentation datasets, demonstrating its excellent model discrimination and generalization capabilities.

[0055] The present invention will be further described below with reference to the figures and embodiments. Attached Figure Description

[0056] Figure 1 To capture the overall structure diagram of the Real-Time Detection Network (LRGN);

[0057] Figure 2 This is a structural diagram of the Dilatation Refinement Module (DRM).

[0058] Figure 3 A structural diagram of the Pixel Attention Guided Fusion (PAG) module;

[0059] Figure 4 This is a structural diagram of the Deep Guided Fusion Module (DGFM).

[0060] Figure 5 This is a structural diagram of the Hybrid Attention Module (MAM).

[0061] Figure 6 This is a schematic diagram of the decoder structure;

[0062] Figure 7 To capture the camera image in step S02 of the pose acquisition method;

[0063] Figure 8 To capture the segmented image to be labeled in step S02 of the pose acquisition method;

[0064] Figure 9 The image for capturing the pose is annotated in step S02 of the pose acquisition method;

[0065] Figure 10 This is a visualization of some test results of the Real-Time Capture Detection Network (LRGN) and other networks on the seedling tray dataset. Detailed Implementation

[0066] Example 1:

[0067] like Figure 1-9 As shown, the optimal gripping posture for seedlings in seedling trays is achieved using a two-finger gripper on a grading and transplanting device. The two-finger gripper is positioned above the seedling tray, which is placed horizontally on the device. The lower end of the gripper has two claw-like parts for inserting the substrate. The opening of these claw-like parts is adjusted by opening and closing. The gripper has translational and rotational movement modes. Translational movement allows it to move directly above any position within the seedling tray, while rotational movement allows it to rotate at any angle around a vertical axis.

[0068] The method is as follows:

[0069] S01 represents the grasping pose:

[0070] The grasping pose includes coordinates, angle, and width. The coordinates are the coordinates of the grasping center point, which are equivalent to the coordinates of the midpoint of the line connecting the lower endpoints of the two claws. The angle is the angle between the line connecting the lower endpoints of the two claws and the horizontal baseline. The width is the length of the line segment formed by the line connecting the lower endpoints of the two claws.

[0071] Grasping pose in image coordinate system As shown in Equation 1;

[0072] Formula 1:

[0073] Where [u, v] are the coordinates of the pixel in the image coordinate system. The angle in the grasping pose corresponding to the pixel. The width parameter in the grasping pose corresponding to the pixel;

[0074] The grabbing pose g in the world coordinate system is shown in Equations 2 and 3.

[0075] Formula 2:

[0076] Formula 3:

[0077] Equation 2 is obtained from Equation 1 through coordinate transformation, and Equation 3 is obtained from Equation 2 through coordinate transformation, t WC Let t be the transformation matrix between the world coordinate system and the camera coordinate system. CI This is the mapping matrix between the image coordinate system and the camera coordinate system; [x, y, z] are the coordinates of the midpoint of the line connecting the lower endpoints of the two claw bodies in the world coordinate system. The angle between the line connecting the lower ends of the two claws and the horizontal baseline is denoted by , and w is the length of the line segment formed by the line connecting the lower ends of the two claws.

[0078] Since z is a known quantity, equation 3 is simplified to equation 4;

[0079] Formula 4:

[0080] Where [x, y] are the coordinates in the grasping pose. To capture the angle in the pose, w is the width in the captured pose.

[0081] S02, Dataset Construction:

[0082] The camera is fixed directly above the seedling tray, and images of the tray are taken from a vertically downward perspective. These images include the individual seedling holes and the seedlings emerging from them. The captured images are then segmented and scaled according to the seedling tray grid to obtain multiple images of the same resolution (350×350) to be labeled. Each image contains exactly one complete seedling tray grid. Pixel-level grasping pose annotations (manual annotation) are performed on each image. The annotations include the grasping center (corresponding to the coordinates in the grasping pose), the grasping angle (corresponding to the angle in the grasping pose), and the grasping position. Width (corresponding to the width in the grasping pose); the marking principles are as follows: Ⅰ. The grasping center is marked at the junction of the seedling leaf and the stem (i.e., with the seedling stem as the center); Ⅱ. The grasping angle is perpendicular to the line connecting the centers of gravity of the two largest leaves on the seedling (i.e., avoiding the two largest leaves of the seedling); Ⅲ. The two ends of the grasping width are the landing points. The landing points are more than 1cm away from the edge of the seedling tray (if less than 1cm, the substrate edge is prone to collapse or fall off), and the distance between the two landing points is not less than 3cm (if less than 3cm, the seedling root system is more likely to be damaged).

[0083] S03, Training the neural network:

[0084] All labeled images are divided into training and validation sets in a 4:1 ratio and input into a Real-Time Grasping Detection Network (LRGN) for training and testing. The trained LRGN will then be capable of pixel-level grasping pose annotation on the input image. The original image for which the grasping pose needs to be obtained is input into the LRGN for prediction. The LRGN outputs the grasping confidence, grasping angle, and grasping width for each pixel in the image. The pixel with the highest grasping confidence is selected as the grasping center point. The coordinates [x, y] of the grasping center point and the corresponding grasping angle are also specified. The grasping width w corresponding to the grasping center point is the predicted grasping posture of the seedling in the seedling tray.

[0085] The Real-Time Capture and Detection Network (LRGN) consists of an encoder and a decoder. The encoder includes a feature extraction module (MobileNet), a dilation and thinning module (DRM), a pixel-guided attention fusion module (PAG), a depth-guided fusion module (DGFM), and a hybrid attention module (MAM). After the feature extraction module extracts features from the original image, it outputs feature F. Feature F is then enhanced with global features by the dilation and thinning module, outputting a branch F with a relatively smaller receptive field. s and the branch F with a relatively large receptive field l F s and F l This creates two parallel paths. s In the path, F s F and deep features, respectively, are input to the pixel attention-guided fusion module to learn semantic information and output features. F and deep features, respectively, are input into the depth-guided fusion module to enrich the feature representation, and the output feature F is... S ′, F S The input hybrid attention module weights the importance of the input features and outputs feature F. S ".F l In the path, F l F and deep features, respectively, are input to the pixel attention-guided fusion module to learn semantic information and output features. F and deep features, respectively, are input into the depth-guided fusion module to enrich the feature representation, and the output feature F is... l ′, F l The input hybrid attention module weights the importance of the input features and outputs feature F. l The decoder includes a region output head, an angle output head, and a width output head. The region output head receives characteristic F. S", outputs the capture confidence score for each pixel. The angle output head and width output head respectively receive the feature F l "" outputs the grasping angle and grasping width of each pixel.

[0086] The implementation process of the dilation and thinning module is as follows: After feature F enters the dilation and thinning module, it first adjusts the number of feature channels to 1024 through a 3×3 convolution to obtain feature F′. Then, F′ is divided into four groups of feature maps {F′} along the channel dimension. 1 F 2 F 3 F 4 Next, a multi-scale feature fusion method is used to add adjacent branch features pixel by pixel. Then, the four sets of results after adding features are input into convolutions with different dilation rates to extract contextual features, resulting in four sets of feature branches. Finally, execute the following two processes respectively: I. Execute feature branches and By concatenating along the channel dimension and then performing a residual connection with the feature F′, a branch F with a relatively small receptive field is obtained. s II. Feature branches and By concatenating along the channel dimension and then performing a residual connection with the feature F′, a branch F with a relatively large receptive field is obtained. l ;

[0087] In the implementation process of the dilation and refinement module, four sets of feature maps {F 1 F 2 F 3 F 4} to four sets of feature branches The operations performed are shown in equations 5, 6, 7, and 8.

[0088] Formula 5:

[0089] Formula 6:

[0090] Formula 7:

[0091] Formula 8:

[0092] In Equation 5, Conv(·) represents a 3×3 convolution with an inflation rate of 1; in Equation 6, Conv(·) represents a 3×3 convolution with an inflation rate of 3; in Equation 7, Conv(·) represents a 3×3 convolution with an inflation rate of 5; and in Equation 8, Conv(·) represents a 3×3 convolution with an inflation rate of 8.

[0093] The implementation process of the pixel attention-guided fusion module is as follows: the shallow feature F is processed by a 1×1 convolution with a dilation rate of 1 to adjust the number of channels to 128, and the deep feature F... s-l After adjusting the channel count to 128 using two 1×1 convolutions with a dilation rate of 1, upsampling is performed. Then, the shallow features after channel adjustment are multiplied pixel-by-pixel with the upsampled deep features. The pixel-by-pixel multiplication results are then summed channel-by-channel, and the importance weights of each pixel in the channel-by-channel summed features are obtained using the Sigmoid function (see Equation 9). Finally, the features are output according to Equation 10. Reference or F s-l Refers to F s or F l ;

[0094] Formula 9:

[0095] Formula 10:

[0096] Where F represents shallow features, F s-l For deep features; σ1 is the Sigmoid function in the pixel attention-guided fusion module, whose value represents the pixel importance weight. The higher the value of σ1, the more trusted the pixel. The lower the σ1 value, the greater the trust. The vector of the corresponding pixel in the deep feature (the corresponding pixel refers to the pixel at the same position in both the deep and shallow features) is: The vector of the corresponding pixel in the shallow feature (the corresponding pixel refers to the pixel at the same position in both the deep and shallow features) is: The output features of the pixel attention-guided fusion module.

[0097] The implementation process of the deep guided fusion module is as follows: the shallow features F are processed by a 3×3 convolution with a dilation rate of 1 to adjust the number of channels to 256, and the deep features... After upsampling, the number of channels is adjusted to 256 through a 3×3 convolution with a dilation rate of 1; the shallow features after convolution... and deep features after convolution Feature F is obtained by concatenating along the channel dimension. c Then F c The result after inputting the global max pooling layer and F c The results from the input global average pooling layer are summed, and then the sum is passed through a 1×1 convolution with a dilation rate of 1 to adjust the number of channels to 1. The importance weights of each channel in the convolutional features are then obtained using the Sigmoid function. Finally, these weights are combined with the shallow features from the convolution. Multiply, and then combine the result of the multiplication with the shallow features after convolution. The sums are then added together, and finally the sum is combined with the deep features obtained after convolution. The features are concatenated and output as feature F′, where F′ represents F. S ′ or F l ′;

[0098] Shallow features after convolution and deep features after convolution As shown in Equations 11 and 12 respectively; from the shallow features after convolution and deep features after convolution The operations that the deep guided fusion module undergoes to produce the output feature F′ are shown in Equation 13.

[0099] Formula 11:

[0100] Formula 12:

[0101] Formula 13:

[0102] Among them, Conv 3×3 (·) represents a 2D convolution with 256 output channels, a dilation rate of 1, and a kernel size of 3; UP(·) represents an upsampling operation; · represents feature concatenation; GAP(·) represents global average pooling; GMP(·) represents global max pooling; Conv 1×1 (·) represents a two-dimensional convolution with one output channel, an inflation rate of 1, and a kernel size of 1. σ² represents pixel-level multiplication, and σ² is the Sigmoid function in the depth-guided fusion module, whose value represents the channel importance weight.

[0103] The implementation process of the hybrid attention module is as follows: the input features and output features of the deep guided fusion module are F′ and F″, respectively, and the operation from F′ to F″ is shown in Equation 14.

[0104] Formula 14:

[0105] Where F″ refers to F S "and F l ″, α is an adaptive weight with an initial value of 1.0 (adaptively adjusted during training), CAM(·) represents the coordinate attention module, and CA(·) represents the channel attention module.

[0106] The implementation process of the region output head is as follows: Input feature F S"After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 128. Then, after a 2×2 deconvolution with a dilation rate of 1, upsampling is performed. Finally, after a 1×1 convolution with a dilation rate of 1 and upsampling operation, a grasping point feature map with a channel count of 1 with the same resolution as the original image (the input image at the front end of the entire neural network) is obtained. The map contains the grasping confidence of each pixel."

[0107] The implementation process of the angle output head is as follows: Input feature F l "After two 3×3 convolutions with a dilation rate of 1 to adjust the number of channels to 256, a 2×2 deconvolution with a dilation rate of 1 is performed for upsampling. Finally, a 1×1 convolution with a dilation rate of 1 and upsampling operation are performed to obtain a grasping point feature map with k channels, which has the same resolution as the original image (the input image at the front end of the entire neural network). The map contains the channel k to which the grasping angle of each pixel belongs. The grasping angle can be calculated based on the value of k."

[0108] The implementation process of the width output header is as follows: Input feature F l "After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 128. Then, after a 2×2 deconvolution with a dilation rate of 1, upsampling is performed. Finally, after a 1×1 convolution with a dilation rate of 1 and upsampling operation, a grasping point feature map with a channel count of 1 with the same resolution as the original image (the input image at the front end of the entire neural network) is obtained. The map contains the grasping width of each pixel."

[0109] Performance verification of the Real-Time Capture and Detection Network (LRGN):

[0110] I. Experimental Conditions

[0111] The Real-Time Capture and Detection Network (LRGN) was built and trained using the PyTorch (2.0.1) deep learning framework and Python (3.10) as the programming language. The computer used in the experiment ran Windows 10, with an NVIDIA GeForce RTX 3080 GPU and a processor... Core TM The system uses an i9-10900X processor at 3.70GHz with 32GB of RAM. During training, the Adam optimizer is used to optimize the network, with a batch size of 16, 200 epochs, an initial learning rate of 0.001, and a learning rate that decays by 0.5 every 50 epochs.

[0112] II. Evaluation Indicators

[0113] The rectangular metric evaluation method is used to evaluate the crawling and detection results of the Real-time Crawling and Detection Network (LRGN). It is defined as follows: if the predicted crawl box G...p and the labeled capture box G t Predictive crawling is accurate when both of the following conditions are met.

[0114] (1) The difference between the predicted gripping angle and the labeled gripping angle is less than 30°;

[0115] (2) Predict the capture box G p With the annotation capture box G t The Jaccard coefficient is greater than 0.25, and the expression for the Jaccard coefficient is shown in Equation 15.

[0116] Formula 15:

[0117] Among them, J(G p G t ) is the predicted capture box G p With the annotation capture box G t Jaccard coefficient, G p ∩G t To predict the area of ​​the intersection of the capture bounding box and the labeled capture bounding box, |G p ∩G t |To predict the area of ​​the union of the capture bounding box and the labeled capture bounding box; G p To predict the grab bounding box, the predicted grab pose line segment is used as the long side of a rectangle, and then translated 15 pixels to each side to form a short rectangle with a width of 30 pixels. This results in the bounding box being G. t To create a bounding box, the line segment containing the labeled gripping pose is used as the long side of the rectangle. The rectangle is then translated 15 pixels to each side to form a short rectangle with a width of 30 pixels.

[0118] III. Experimental Results of Scraping the Cornell Dataset

[0119] LRGN was trained and evaluated on the Cornell grasp dataset [Lenz I, Lee H, Saxena A. Deep learning for detecting robotic grasps[J].The International Journal of Robotics Research,2015,34(4-5):705-724]. The Cornell grasp dataset contains 885 RGB and depth images of 240 real objects at different positions and poses, with each image having a resolution of 640×480. Due to the small size of the Cornell dataset, to prevent overfitting during training, the data was augmented by cropping, random rotation, and random scaling. After augmentation, the dataset resolution was adjusted to 350×350 to meet the input image requirements of the Real-Time Grasping Detection Network (LRGN).

[0120] The Cornell Grasp dataset was divided into two subsets based on image splitting and object splitting to validate the model's discrimination and generalization abilities. Image splitting randomly divided the dataset to test the model's ability to discriminate objects from different angles and positions. Object splitting separated images of the same object from different angles into separate training and test sets, ensuring that objects in the test set did not appear in the training set, thus validating the network's generalization ability when grasping unknown objects. This experiment used only RGB images as input, with a training-to-test set ratio of 4:1. Annotations were performed using the method described in the paper [High-Performance Pixel-Level Grasp Detection Based on AdaptiveGrasping and Grasp-Aware Network], including the grasp center (corresponding to coordinates in the grasp pose), grasp angle (corresponding to the angle in the grasp pose), and grasp width (corresponding to the width in the grasp pose).

[0121] Since the Real-Time Grasp Detection Network (LRGN) is a pixel-level grasp detection algorithm, it generates a predicted grasp box for each pixel of the input image. During testing, it only selects the predicted grasp box G with the highest probability of successful grasping. p (Select the pixel with the highest capture confidence as the capture center point, and define the coordinates [x, y] of the capture center point and the corresponding capture angle.) The grasping width *w* corresponding to the grasping center point determines the line segment where the grasping pose is located. This line segment is used as the long side of a rectangle, and it is translated 15 pixels to both sides to form a short rectangle with a width of 30 pixels. The resulting bounding box is the predicted grasping box *G* with the highest probability of successful grasping. pThe Real-Time Crawling Detection Network (LRGN) was used for evaluation. The experimental results are compared with some existing representative crawling detection algorithms, and are shown in Table 1.

[0122] Table 1:

[0123]

[0124] GG-CNN2 comes from: Morrison D, Corke P, Leitner J. Learning robust, real-time, reactive robotic grasping[J]. The International Journal of Robotics Research, 2020, 39(2-3):183-201;

[0125] AlexNet comes from: Redmon J, Angelova A. Real-time grasp detection using convolutional neural networks [C]. 2015IEEE international conference onrobotics and automation (ICRA), 2015: 1316-1322;

[0126] ResNet-50×2 comes from: Kumra S, Kanan C. Robotic grasp detection using deepconvolutional neural networks [C]. 2017IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS), 2017:769-776;

[0127] GraspNet from: Asif U, Tang J, Harrer S.GraspNet: An EfficientConvolutional Neural Network for Real-time Grasp Detection for Low-poweredDevices[C].IJCAI,2018:4875-4882;

[0128] SGDN Source: Wang D. SGDN: Segmentation-based grasp detection network for unsymmetrical three-finger gripper[J]. arXiv preprint arXiv:2005.08222, 2020;

[0129] TsGNet Source: Yu Y, Cao Z, Liu Z, et al. A two-stream CNN with simultaneous detection and segmentation for robotic grasping[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 52(2): 1167-1181;

[0130] GKNet Source: Xu R, Chu F-J, Vela P A. GKNet: Grasp keypoint network for grasp candidates detection[J]. The International Journal of Robotics Research, 2022, 41(4): 361-389;

[0131] AFFGA-Net(ResNet101) Source: Wang D, Liu C, Chang F, et al. High-performance pixel-level grasp detection based on adaptive grasping and grasp-aware network[J]. IEEE transactions on industrial electronics, 2021, 69(11): 11611-11621.

[0132] The results show that while the GG-CNN2 network has fewer parameters, its grasping detection accuracy is very low, with accuracies of only 73.00% and 69.00% in image segmentation and object segmentation, respectively, significantly impacting the grasping success rate. The TsGNet network also has fewer parameters and achieves an FPS of 147 frames per second, demonstrating strong real-time performance, but its grasping detection accuracy still lags behind the latest grasping detection networks. AFFGA-Net boasts the highest accuracy, with accuracies of 99.09% and 98.64% in image segmentation and object segmentation, respectively; however, its large parameter count limits its real-time grasping detection performance. The Real-Time Crawl Detection Network (LRGN) achieves high accuracy in both image segmentation and object segmentation datasets, at 98.96% and 98.30% respectively, demonstrating its excellent discriminative and generalization capabilities. Its inference speed of 113 images / second is significantly higher than most existing crawling detection algorithms, proving its good real-time performance. With only 12.67M parameters, significantly fewer than other high-accuracy crawling networks (AFFGA-Net and GKNet), it is easily portable to edge computing devices such as mobile phones, embedded systems, and IoT devices.

[0133] IV. Experimental Results of Seedling Tray Dataset

[0134] The Real-Time Grasping Detection Network (LRGN) was trained and evaluated on the collected plug seedling dataset, which was composed in accordance with step S02 of the optimal grasping pose acquisition method for plug seedlings.

[0135] The real-time capture detection network (LRGN) was compared with different networks (AFFGA-Net, GG-CNN2, and GR-ConvNet) in terms of capture detection accuracy, floating-point number, number of parameters, weight size, and FPS. The results are shown in Table 2.

[0136] Table 2:

[0137]

[0138] AFFGA-Net (ResNet101), AFFGA-Net (Xception), AFFGA-Net (DRN), and AFFGA-Net (MobileNet) are all from: Wang D, Liu C, Chang F, et al.

[0139] High-performance pixel-level grasp detection based on adaptive grasping and grasp-aware network[J]. IEEE transactions on industrialelectronics, 2021, 69(11):11611-11621;

[0140] GG-CNN2 comes from: Morrison D, Corke P, Leitner J. Learning robust, real-time, reactive robotic grasping[J]. The International Journal of Robotics Research, 2020, 39(2-3):183-201;

[0141] GR-ConvNet comes from: Kumra S, Joshi S, Sahin F. Antipodal robotic grasping using generative residual convolutional neural network [C]. 2020 IEEE / RSJInternational Conference on Intelligent Robots and Systems (IROS), 2020: 9626-9633.

[0142] The results show that the Real-Time Grab Detection Network (LRGN) achieves the highest detection accuracy of 98.83% on the seedling grab dataset, with an FPS of 113 images / second, second only to GG-CNN2. Its trained weights are only 46.3MB, significantly reducing model loading time and inference latency. It balances detection accuracy and speed while being easily portable to edge computing devices such as mobile phones, embedded systems, and IoT devices.

[0143] The test results of the Real-Time Capture Detection Network (LRGN) and other networks on the seedling capture dataset are visualized, such as... Figure 10 As shown in the figure, the grasping pose detected by the Real-Time Grasp Detection Network (LRGN) can better locate the seedling core, and the landing point of its grippers can avoid the seedling body and the edge of the seed tray, thus achieving low-damage grasping.

[0144] V. Network Validation

[0145] To further verify the effectiveness of the proposed network, the Jaccard coefficient was increased from the initial 25% to 35% in 5% increments. The larger the Jaccard coefficient, the more stringent the crawling conditions, and the more difficult it is to evaluate a successful crawl. The crawling detection accuracy under different Jaccard coefficients is shown in Table 3.

[0146] Table 3:

[0147]

[0148] The results show that although the accuracy of the proposed network is lower under high Jaccard coefficients than under low Jaccard coefficients, even when the Jaccard coefficients of the ground truth and predicted grasping regions are set at 35%, the network can still accurately determine the grasping pose in most cases and has strong robustness.

[0149] VI. Ablation Experiment

[0150] To verify the role of each module in the Real-Time Capture and Detection Network (LRGN), ablation experiments were conducted on each module. The experimental environment and training details were consistent with those described above, and the results are shown in Table 4. The baseline network in Table 4 is the one after the Feature Extraction module (MobileNet) with a decoder added directly. For example, the first row in Table 4 has a decoder added directly after MobileNet, the second row has a decoder added directly after the Dilation and Refinement Module (DRM), the third row has a decoder added directly after the Pixel Attention Guided Fusion Module (PAG), the fourth row has a decoder added directly after the Depth Guided Fusion Module (DGFM), and the fifth row has a decoder added directly after the Hybrid Attention Module (MAM).

[0151] Table 4:

[0152] baseline DRM PAG DGFM MAM accuracy / % √ 94.74 √ √ 95.91 √ √ √ 96.94 √ √ √ √ 98.25 √ √ √ √ √ 98.83

[0153] The results show that, compared to the baseline network, adding the DRM module improved the crawling detection accuracy by 1.17%; adding the PAG module improved it by 1.03%; adding the DGFM module improved it by 1.31%; and adding the MAM module improved it by 0.58%. This demonstrates that adding these modules significantly improves the detection accuracy on the seedling crawling dataset while having minimal impact on the detection speed, thus further enhancing the overall performance of the algorithm.

Claims

1. A method for obtaining the optimal grasping posture of seedlings in a plug tray is based on a two-finger clamp of a graded transplanting device; the two-finger clamp is located above the plug tray; two claws for inserting the substrate are provided at the lower end of the two-finger clamp, and the two claws are opened or closed to adjust the opening degree; the two-finger clamp has translation and rotation action modes, and the two-finger clamp moves to the top of any position in the plug tray through translation, and the two-finger clamp rotates around the vertical axis to any angle through rotation; Its characteristics are: Here’s how: S01, characterizing the grasping posture: The grasping posture includes coordinates, angles and widths, wherein the coordinates are the coordinates of the grasping center point, which is equivalent to the coordinates of the midpoint of the line connecting the lower end points of the two claws; wherein the angle is the angle formed by the line connecting the lower end points of the two claws and the horizontal reference line, wherein the width is the length of the line segment formed by the line connecting the lower end points of the two claws; Grasping pose in image coordinate system As shown in formula 1; Formula 1: Among them, [u, v] is the coordinate of the pixel point in the image coordinate system, is the angle in the grasping posture corresponding to the pixel point, is the width parameter in the grasping posture corresponding to the pixel point; The grasping posture g in the world coordinate system is shown in Equations 2 and 3; Formula 2: Formula 3: Among them, Formula 2 is obtained by coordinate transformation of Formula 1, and Formula 3 is obtained by coordinate transformation of Formula 2. WC is the transformation matrix between the world coordinate system and the camera coordinate system, t CI is the mapping matrix between the image coordinate system and the camera coordinate system; [x, y, z] is the coordinate of the midpoint of the line connecting the lower end points of the two claws in the world coordinate system, is the angle between the line connecting the lower end points of the two claws and the horizontal reference line, and w is the length of the line segment formed by the line connecting the lower end points of the two claws; Since z is a known quantity, equation 3 is simplified to equation 4; Formula 4: S02, dataset construction: The camera is fixed above the plug tray, and the plug tray image is taken from a vertical downward perspective. The image includes each hole in the plug tray and the seedlings grown in each hole. The captured image is segmented and scaled according to the plug tray grid to obtain multiple images with the same resolution to be marked. Each image to be marked has one and only one complete plug tray grid. Each image is annotated with pixel-level grasping posture. The annotation content includes grasping center, grasping angle and grasping width. The annotation principles are as follows: Ⅰ. The grasping center is marked at the intersection of the seedling leaves and stems; Ⅱ. The grasping angle is perpendicular to the center of gravity of the two largest leaves on the seedling; Ⅲ. The two end points of the grasping width are the landing points, which are more than 1 cm away from the edge of the plug tray grid, and the distance between the two landing points is not less than 3 cm; S03, training neural network: All labeled images are divided into training set and validation set, and input into real-time grasping detection network for training and testing. The trained real-time grasping detection network has the ability to annotate the input image with pixel-level grasping posture. The original image for which the grasping posture needs to be obtained is input into the real-time grasping detection network for prediction. The real-time grasping detection network outputs the grasping confidence of each pixel in the image, the grasping angle of each pixel, and the grasping width of each pixel. The pixel with the highest grasping confidence is selected as the grasping center point. The coordinates [x, y] of the grasping center point and the grasping angle corresponding to the grasping center point are obtained. The grasping width w corresponding to the grasping center point is the predicted grasping posture of the seedling in the tray.

2. The method for obtaining the optimal grasping posture of seedlings in a plug tray as claimed in claim 1, characterized in that: The real-time grasping detection network includes an encoder and a decoder; the encoder includes a feature extraction module, an expansion and refinement module, a pixel attention guided fusion module, a depth guided fusion module and a hybrid attention module; after the original image is extracted by the feature extraction module, the feature F is output, and the feature F is enhanced by the expansion and refinement module to enhance the global feature and output the branch F with a relatively small receptive field s and branch F with relatively large receptive field l , F s and F l Form two parallel paths; F s In the path, F s and F are used as deep features and shallow features respectively, and the pixel attention guides the fusion module to learn semantic information and output features and F are respectively used as deep features and shallow features and are jointly input into the depth-guided fusion module to enrich the feature representation, and the output feature F S ′,F S ′ Input hybrid attention module to weight the importance of input features and output feature F S ″; F l In the path, F l and F are used as deep features and shallow features respectively, and the pixel attention guides the fusion module to learn semantic information and output features and F are respectively used as deep features and shallow features and are jointly input into the depth-guided fusion module to enrich the feature representation, and the output feature F l ′,F l ′ Input hybrid attention module to weight the importance of input features and output feature F l ″; The decoder includes a region output head, an angle output head, and a width output head; the region output head receives the feature F S ″, outputs the grasp confidence of each pixel; the angle output head and width output head receive the feature F l ″, output the grab angle and grab width of each pixel.

3. The method for obtaining the optimal grasping posture of seedlings in a plug tray as claimed in claim 2, characterized in that: The implementation process of the expansion and refinement module is as follows: after the feature F enters the expansion and refinement module, a 3×3 convolution is performed to adjust the number of feature channels to 1024 to obtain the feature F′, and then F′ is evenly divided into four groups of feature maps along the channel dimension {F 1 , F 2 , F 3 , F 4 }, then use the multi-scale feature fusion method to add adjacent branch features pixel by pixel, and then input the four groups of results after adding features into convolutions with different void rates to extract context features, and obtain four groups of feature branches Finally, the following two processes are performed respectively: Ⅰ. Branch the feature and Splicing along the channel dimension, and then performing residual connection with feature F′, to obtain a branch F with a relatively small receptive field s ; Ⅱ. Branch the feature and Splicing along the channel dimension, and then performing residual connection with feature F′, to obtain a branch F with a relatively large receptive field l ; In the implementation process of the expansion and refinement module, four sets of feature maps {F 1 , F 2 , F 3 , F 4 } to four sets of feature branches The operations experienced are shown in Equations 5, 6, 7, and 8; Formula 5: Formula 6: Formula 7: Formula 8: Among them, Conv(·) in Formula 5 represents a 3×3 convolution with a dilation rate of 1, Conv(·) in Formula 6 represents a 3×3 convolution with a dilation rate of 3, Conv(·) in Formula 7 represents a 3×3 convolution with a dilation rate of 5, and Conv(·) in Formula 8 represents a 3×3 convolution with a dilation rate of 8.

4. The method for obtaining the optimal gripping posture of seedlings in a plug tray as claimed in claim 3, characterized in that: The implementation process of the pixel attention guided fusion module is as follows: the shallow feature F is adjusted to 128 channels through a 1×1 convolution with a dilation rate of 1, and the deep feature F s-l After two 1×1 convolutions with a dilation rate of 1, the number of channels is adjusted to 128 and then upsampled; Then, the shallow features after adjusting the number of channels are multiplied pixel by pixel with the deep features after upsampling, and then the pixel-by-pixel multiplication results are summed channel by channel, and then the importance weight of each pixel in the features after channel-by-channel summation is obtained by Sigmoid function. Here, the Sigmoid function refers to formula 9, and finally the output feature is referred to formula 10 Reference or F s-l Refers to F s or F l ; Formula 9: Formula 10: Among them, F is the shallow feature, F s-l is the deep feature; σ1 is the Sigmoid function in the pixel attention guidance fusion module, and its value represents the pixel importance weight. The higher the σ1 value, the more trustworthy it is. The lower the σ1 value, the more trust The vector corresponding to the pixel in the deep feature is The vector corresponding to the pixel in the shallow feature is Output features of the pixel attention guided fusion module.

5. The method for obtaining the optimal grasping posture of seedlings in a plug tray as claimed in claim 4, characterized in that: The implementation process of the deep guided fusion module is as follows: the shallow feature F is adjusted to 256 channels through a 3×3 convolution with a dilation rate of 1, and the deep feature After upsampling, the number of channels is adjusted to 256 through a 3×3 convolution with a dilation rate of 1; the shallow features after convolution And the deep features after convolution After concatenation in the channel dimension, we get feature F c , then F c The result after inputting the global maximum pooling layer and F c The results after the global average pooling layer are added, and then the number of channels is adjusted to 1 through a 1×1 convolution with a dilation rate of 1. Then, the importance weight of each channel in the convolution feature is obtained through the Sigmoid function, and the weight value is added to the shallow feature after convolution. Multiply, and then add the multiplication result to the shallow feature after convolution Add, and finally add the result to the deep features after convolution Concatenate and output feature F′, where F′ refers to F S ′ or F l ′; Shallow features after convolution And the deep features after convolution As shown in Equation 11 and Equation 12 respectively; from the shallow features after convolution And the deep features after convolution The operations that the output feature F′ of the deep guided fusion module undergoes are shown in Equation 13; Formula 11: Formula 12: Formula 13: Among them, Conv 3×3 (·) represents a two-dimensional convolution with 256 output channels, a dilation rate of 1, and a kernel size of 3. UP(·) represents an upsampling operation. · represents feature concatenation. GAP(·) represents global average pooling. GMP(·) represents global maximum pooling. Conv 1×1 (·) represents a 2D convolution with 1 output channel, dilation rate 1, and kernel size 1. represents pixel-level multiplication, σ2 is the Sigmoid function in the depth-guided fusion module, and its value represents the channel importance weight.

6. The method for obtaining the optimal grasping posture of seedlings in a plug tray as claimed in claim 5, characterized in that: The input features and output features of the hybrid attention module are F′ and F″ respectively. The operations from F′ to F″ are shown in Equation 14; Formula 14: Among them, F″ refers to F S ″ and F l ″, α is the adaptive weight with an initial value of 1.0 (adaptively adjusted during training), CAM(·) represents the coordinate attention module, and CA(·) represents the channel attention module.

7. The method for obtaining the optimal gripping posture of seedlings in a plug tray as claimed in claim 6, characterized in that: The implementation process of the regional output head is: input feature F S After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 128, and then upsampled by a 2×2 deconvolution with a dilation rate of 1. Finally, after a 1×1 convolution and upsampling operation with a dilation rate of 1, a grasping point feature map with a channel number of 1 that is consistent with the original image resolution is obtained. The map contains the grasping confidence of each pixel.

8. The method for obtaining the optimal gripping posture of seedlings in a plug tray as claimed in claim 7, characterized in that: The implementation process of the angle output head is: Input feature F l "After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 256, and then up-sampled through a 2×2 deconvolution with a dilation rate of 1. Finally, after a 1×1 convolution and up-sampling operation with a dilation rate of 1, a grasping point feature map with a channel number k consistent with the original image resolution is obtained. The map contains the channel k to which the grasping angle of each pixel belongs. The grasping angle can be calculated according to the value of k.

9. The method for obtaining the optimal grasping posture of seedlings in a plug tray as claimed in claim 8, characterized in that: The implementation process of the width output head is: Input feature F l After two 3×3 convolutions with a dilation rate of 1, the number of channels is adjusted to 128, and then up-sampled by a 2×2 deconvolution with a dilation rate of 1. Finally, after a 1×1 convolution and up-sampling operation with a dilation rate of 1, a grab point feature map with a channel number of 1 and the same resolution as the original image is obtained. The map contains the grab width of each pixel.

Images