A ginger orientation and spacing control method and system based on instance segmentation

Abstract

The present application belongs to the field of agricultural image recognition and intelligent direction adjustment, and discloses a ginger direction adjustment and plant spacing control method and system based on instance segmentation, which comprises the following steps: training the improved YOLO11-seg by collecting the stacked ginger data set, and obtaining the mask information of ginger blocks and ginger sprouts. Combined with the depth image after using the repair algorithm, the three-dimensional coordinates of each target ginger in the camera coordinate system are obtained. Based on the multi-factor scoring function, the priority of multiple ginger targets is sorted, and the relative position relationship between the ginger sprouts and the ginger blocks is used to solve the direction adjustment angle and the plant spacing correction distance of the ginger placement. This method realizes the accurate identification and acquisition of the direction adjustment parameters of the stacked ginger in the box, and can be used in intelligent seeding and other agricultural scenes to replace manual seeding, reduce labor costs, improve seeding efficiency and quality, and meet the needs of precision agriculture.

Description

Technical Field

[0001] This invention belongs to the field of agricultural image recognition and intelligent orientation technology, specifically involving a method and system for ginger orientation and plant spacing control based on instance segmentation, which is applicable to automated sowing, orientation, and precise placement of ginger. Background Technology

[0002] In mechanized ginger cultivation, the orientation of ginger sprouts towards the sun at the time of sowing significantly impacts subsequent germination rate, growth direction, and field management effectiveness. Currently, orientation adjustment mainly relies on manual visual judgment and rotation, which is labor-intensive, inefficient, and inconsistent. Existing visual recognition methods mostly focus on target detection, which is ineffective for irregularly shaped, complex, and heavily occluded crops like ginger, especially in distinguishing between ginger rhizomes and sprouts, thus hindering automatic orientation adjustment. The lack of an effective method to accurately identify each ginger instance and differentiate between sprouts and rhizomes also limits seed collection and autonomous orientation adjustment capabilities. Therefore, there is an urgent need for an automatic recognition method integrating instance segmentation, 3D coordinate positioning, and orientation decision-making to improve the intelligence and automation level of ginger cultivation. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a method and system for controlling the orientation and spacing of ginger plants based on instance segmentation. This method enables high-precision identification and three-dimensional positioning in stacked ginger scenarios, allowing for the adjustment of the ginger seed angle and correction of the plant spacing, thereby providing technical support for intelligent sowing and planting operations.

[0004] To achieve the above objectives, the present invention provides the following solution: A method for controlling ginger orientation and plant spacing based on instance segmentation, the method comprising: S1. Let a point at the bottom of the white plastic box be the origin of the world coordinate system, perform coordinate calibration, and obtain the transformation matrix from the camera coordinate system to the world coordinate system. S2. Stack ginger seeds in a white plastic box, use a depth camera to collect RGB image datasets of the stacked ginger seeds, preprocess and label the images, and construct a ginger seed dataset for training. S3. Based on the improved network structure instance segmentation model and ginger seed dataset, a ginger seed instance segmentation model is trained to obtain the ginger seed instance segmentation model; S4. Use the ginger seed instance segmentation model to segment the ginger seed in the image, obtain the mask regions of ginger pieces and ginger sprouts, and calculate their pixel centroid coordinates respectively. S5. Based on the pixel centroid coordinates of ginger pieces and ginger sprouts, a depth map repair algorithm is used to fill the holes in the depth map. Combined with the repaired depth map and the mask area, the three-dimensional world coordinates of the ginger pieces are calculated. S6. Based on the three-dimensional world coordinates of ginger pieces, a multi-factor sorting strategy based on a scoring function is adopted to prioritize the ginger varieties with the highest scores for seed extraction. S7. By calculating the relative direction and distance between ginger sprouts and ginger rhizomes in world coordinates, the orientation angle and plant spacing correction at the time of sowing are determined.

[0005] Preferably, the specific steps of S1 include: Step 101: Obtain the camera intrinsic parameter K and transform the pixel coordinate system to the camera coordinate system; Step 102: Select 4 points at the bottom and 4 points at the top of the white plastic box for SVD calibration to obtain the matrix for transforming the camera coordinate system to the world coordinate system.

[0006] Preferably, the specific steps of S2 include: Step 201: Fill the edges of the acquired RGB images and filter RGB images whose resolution meets the preset requirements; Step 202: Label the selected RGB images, use polygons to mark the outlines of ginger seeds and ginger sprouts in each image, record the labels, and save them as txt files; Step 203: Divide the ginger seed dataset into a training set and a validation set in a 7:3 ratio.

[0007] Preferably, the specific steps of S3 include: Step 301: Using the YOLO11-seg instance segmentation model as the base model, the single Backbone structure is improved into a dual-branch structure. The original module is named RGB_Backbone, and Depth_Backbone is introduced to train the depth map. Step 302: Introduce the SimAM and SGE attention mechanism modules into C3Ghost in Depth_Backbone; Step 303: Expand the original head structure of the YOLO11-seg instance segmentation model into four heads; Step 304: Iteratively train the instance segmentation network on the training set to obtain the improved ginger seed instance segmentation model.

[0008] Preferably, the specific steps of S4 include: Step 401: Use the trained ginger instance segmentation model to infer the real-time acquired images and output the mask region for each ginger piece and ginger sprout. Step 402: Using the spatial moment formula, calculate the zeroth and first moments of the mask region to obtain the pixel geometric center coordinates of each ginger piece and ginger sprout.

[0009] Preferably, the specific steps of S5 include: Step 501: Use the biharmonic equation algorithm to repair holes in the depth map; Step 502: In the repaired depth map, calculate the average depth around the pixel geometric center coordinates of ginger pieces and ginger sprouts to obtain the depth of ginger pieces; combine the pixel coordinates of the geometric center points of ginger pieces and ginger sprouts with the depth of ginger pieces, and transform them into the world coordinate system through camera intrinsic parameters and a matrix for transforming the camera coordinate system to the world coordinate system to obtain the three-dimensional coordinates of the geometric center point of ginger pieces as the ginger seed picking point.

[0010] Preferably, the specific steps of S6 include: Step 601: Use the confidence scores of ginger sprouts and ginger pieces identified by the ginger seed instance segmentation model to represent the confidence factor of the scoring function; Step 602: Use the world coordinate height of the geometric center point of the ginger piece to represent the distance factor of the scoring function; Step 603: Combine the confidence factor and distance factor into a weighted scoring function, sort all identified ginger pieces, and select the target with the highest score as the priority picking object.

[0011] Preferably, the specific steps of S7 include: Step 701: Calculate the deflection angle of the ginger sprout relative to the ginger piece. The deflection angle is used for the adjustment angle. Step 702: Calculate the Euclidean distance between the ginger sprout and the ginger rhizome. The Euclidean distance is used for plant spacing correction. Step 703: Use the deflection angle and Euclidean distance between the ginger sprout and the ginger rhizome as input parameters for subsequent control of the robotic arm or planting mechanism to achieve orientation adjustment and sowing.

[0012] The present invention also provides a ginger orientation and plant spacing control system based on instance segmentation. The system is used to implement the aforementioned method and includes: a transformation module, a preprocessing module, a construction module, a segmentation module, a calculation module, a selection module, and an adjustment module. The transformation module is used to set a point at the bottom of the white plastic box as the origin of the world coordinate system, perform coordinate calibration, and obtain the transformation matrix from the camera coordinate system to the world coordinate system. The preprocessing module is used to stack ginger seeds in a white plastic box, use a depth camera to collect RGB image datasets of the stacked ginger seeds, preprocess and label the images, and construct a ginger seed dataset for training. The building module is used to train a ginger seed instance segmentation model based on an instance segmentation model with an improved network structure and a ginger seed dataset. The segmentation module is used to segment ginger seeds in the image using a ginger seed instance segmentation model, obtain the mask regions of ginger pieces and ginger sprouts, and calculate their pixel centroid coordinates respectively. The calculation module is used to fill the holes in the depth map using a depth map repair algorithm based on the pixel centroid coordinates of the ginger pieces and ginger sprouts, and to calculate the three-dimensional world coordinates of the ginger pieces by combining the repaired depth map and the mask area. The selection module is used to select the ginger variety with the highest score for seed extraction based on the three-dimensional world coordinates of the ginger piece and a multi-factor sorting strategy based on a scoring function. The adjustment module is used to determine the orientation angle and plant spacing correction during sowing by calculating the relative direction and distance between ginger sprouts and ginger pieces in world coordinates.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention discloses a ginger orientation and plant spacing control method based on instance segmentation. It trains an improved YOLO11-seg algorithm by collecting a dataset of stacked ginger seeds to obtain mask information for ginger pieces and sprouts. Combined with depth images processed using a repair algorithm, the 3D coordinates of each target ginger seed in the camera coordinate system are obtained. Multiple ginger targets are prioritized based on a multi-factor scoring function, and the orientation angle and plant spacing correction distance for controlling ginger seed placement are calculated using the relative positional relationship between ginger sprouts and pieces. This method achieves accurate identification of stacked ginger in a box and obtains orientation and placement parameters. It can be used in agricultural scenarios such as intelligent sowing to replace manual sowing, reduce labor costs, improve sowing efficiency and quality, and meet the needs of precision agriculture. Attached Figure Description

[0014] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the calibration method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the image acquisition platform according to an embodiment of the present invention; Figure 3 This is a diagram of the improved YOLO11-seg network structure according to an embodiment of the present invention; Figure 4 This is a diagram showing the pixel centroid coordinates obtained by solving the mask for ginger sprouts and ginger chunks in an embodiment of the present invention. Figure 5 This is a comparison image showing the depth map repair effect of an embodiment of the present invention; Figure 6 This is a diagram illustrating the method for solving the orientation angle and plant spacing correction in an embodiment of the present invention; Figure 7This is a flowchart illustrating a ginger orientation and plant spacing control method based on instance segmentation, according to an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Example 1 like Figure 7 As shown, the specific steps of the ginger seed orientation and plant spacing control method described in this embodiment are as follows: Step 1: Set a point at the bottom of the white plastic box as the origin of the world coordinate system, perform coordinate calibration, and obtain the transformation matrix; Step 2: Stack the ginger seeds in a white plastic box, use a depth camera to collect RGB image datasets of the stacked ginger seeds, preprocess and label the images, and construct a ginger seed dataset for training. Step 3: Improve the network structure based on the instance segmentation model, and train the improved ginger seed instance segmentation model using the ginger seed dataset; Step 4: Use the improved ginger seed instance segmentation model to segment ginger seeds in the image, identify the mask regions of ginger pieces and ginger sprouts in each ginger instance, and calculate their pixel centroid coordinates respectively. Step 5: Use a depth map repair algorithm to fill holes in the depth map. Combine the depth map with the masked areas of the ginger sprouts and pieces to calculate the 3D coordinates of the ginger piece's pickup location. Step 6: Use a multi-factor ranking strategy based on a scoring function to prioritize the ginger varieties with the highest scores; Step 7: Calculate the direction and distance of the ginger sprout and ginger rhizome in world coordinates to determine the orientation angle and plant spacing correction distance of the ginger at the time of sowing.

[0019] Combination Figure 1 This implementation method describes step 1, which involves determining the origin of the world coordinate system, performing coordinate calibration, and obtaining the transformation matrix. The specific steps are as follows: Step 101: Obtain the camera intrinsic parameter matrix Used for converting pixel coordinates to camera coordinates; Step 102: Let the bottom left corner of the white plastic box be the origin of the world coordinate system. Select four points at the bottom and four points at the top of the white plastic box for calibration. Solve the transformation matrix between these eight pixel coordinates and the actual physical coordinates using the SVD decomposition method, thereby obtaining the transformation matrix between the camera coordinate system and the world coordinate system. This enables the mapping from two-dimensional images to three-dimensional physical coordinates; Combination Figure 2 This embodiment describes step 2, which involves preprocessing and labeling the RGB image data acquired by the depth camera. The specific steps are as follows: Step 201: Use a plastic box with a base length of 40cm, width of 30cm, and height of 20cm to hold the ginger seeds. The placement is secured with aluminum profiles, which aids in subsequent image processing and positioning. Stack multiple ginger seeds naturally in the white plastic box. Step 202: The depth camera device used in this embodiment is an Intel RealSense D435i binocular camera. It is fixed above the ginger seed box using an aluminum profile, offset horizontally by 4cm, so that the center lines of the two depth cameras are aligned with the center line of the box. The camera then captures images of the stacked ginger seeds vertically downwards, obtaining an RGB image dataset. Step 203: During the data collection process, the dataset is enriched by gently shaking the box and manually and randomly placing the ginger seeds. Step 204: Fill the edges of the acquired RGB image with white to ensure a resolution of 640x640, which is beneficial for size normalization during model training; Step 205: Label the processed RGB images, use polygons to label the outlines of ginger seeds and ginger sprouts in each image, record the labels, and save them as txt files; Step 206: Divide the ginger seed dataset into a training set and a validation set in a 7:3 ratio.

[0020] Combination Figure 3 This implementation method describes step 3, which involves improving the network structure and training the instance segmentation model. The specific steps are as follows: Step 301: Use the YOLO11-seg instance segmentation model as the base model. The original YOLO11 module is named RGB_Backbone. Depth_Backbone is introduced to train the depth map and perform feature fusion. Depth_Backbone is a replacement of the original YOLO11 RGB_Backbone module, replacing Conv with GhostConv and C3K2 with the C3Ghost-SS module. Step 302: Introduce SimAM and SGE attention mechanism modules into the C3Ghost module to construct the C3Ghost-SS module, which enhances the expressive power of the feature map and improves the boundary details and accuracy of instance segmentation. Step 303: Expand the original head structure to four heads to improve the recognition rate and positioning accuracy of small targets, which is beneficial for multi-target recognition in complex scenes. Step 304: After the above improvements, iteratively train the training set until the model reaches the convergence criterion on the validation set, and finally obtain the optimized ginger seed instance segmentation model.

[0021] Combination Figure 4 This implementation method describes step 4, where the instance segmentation model identifies the masked regions of ginger pieces and ginger sprouts and calculates their pixel centroid coordinates. The specific steps are as follows: Step 401: Use the trained ginger instance segmentation model to perform inference on the real-time acquired images. The model outputs the mask region for each ginger piece and ginger sprout. Step 402: Using the space moment formula , Representing an image Space moments of order; Let be non-negative integers, representing the moments in... Order in direction; Indicates the x and y coordinates of the pixels in the output mask; Represents coordinate points The mask value at the specified location; calculate the zeroth moment of the mask region. , and the first moment and This allows us to obtain the pixel geometric center coordinates of each ginger piece and ginger sprout. , Combination Figure 5 In this implementation method, the first column is the heatmap of the original depth map, the second column is the heatmap of the depth map processed by the camera's built-in filtering algorithm, and the third column is the heatmap of the depth map processed using this method. In step 5 of this implementation method, the depth map is filled with holes using a depth map repair algorithm. Combining the depth map with the masked areas of the ginger sprouts and pieces, the three-dimensional coordinates of the ginger piece's pickup location are calculated. The specific steps are as follows: Step 501: A depth map inpainting algorithm based on the biharmonic equation is used to fill in the holes. Mathematically, in digital images, the Laplacian operator is usually discretized using a 3×3 convolution kernel as follows: ; And the two Laplace nuclei: ; in, Indicates the image in The depth value at that location; This represents the Laplace operator (the first-order Laplace on discrete). This represents the discrete biharmonic operator (the second-order Laplace over discrete). L represents the commonly used 3×3 discrete Laplace kernel; K represents the equivalent 5×5 double Laplacian convolution kernel obtained by convolving L with itself in two dimensions. P is a 3×3 depth image region centered at (x,y). Q represents a 5×5 depth image region centered at (x,y), thereby improving the continuity and reliability of depth information. The average depth around the pixel geometric center coordinates of the ginger pieces and shoots is calculated in the repaired depth image. ,in, Indicates the coordinates of the current pixel; Represents the coordinates of any pixel within the neighborhood; Indicates the neighborhood radius, used to determine the surrounding coordinate range; , Represented by pixels Centered on, with radius The neighborhood set, that is, containing all those that satisfy... pixels , for The average depth at that location, Represented by pixels Centered on, with radius The neighborhood set, that is, containing all those that satisfy... pixels , for The number of pixels in the neighborhood For pixels The depth value at that location is obtained through the camera intrinsic parameter matrix. And the transformation matrix from camera coordinate system to world coordinate system Find the seeding point Finally, the three-dimensional position coordinates of the geometric center point of the ginger piece in the world coordinate system were obtained. This serves as the location for obtaining the seed of this ginger variety.

[0022] This implementation method executes step 6, a multi-factor ranking strategy based on a scoring function, prioritizing the ginger variety with the highest score. The specific steps are as follows: Step 601: Set the distance factor, which is determined by the height of the ginger block in the world coordinate system. The higher the position, the easier it is to collect seeds. Normalize the distance from the bottom to the top of the box. , Step 602: Set the confidence factor, which is the confidence level of the ginger segmentation model in recognizing this ginger piece. , Step 603: The two are weighted and combined to form a scoring function. The score of the identified ginger piece is given by the formula. Calculation, where , All identified ginger pieces are sorted, and the highest-scoring target is selected as the priority picking object. Combination Figure 6 This implementation method describes step 7, which involves calculating the direction and distance in world coordinates between the ginger sprout and the ginger rhizome to determine the orientation angle and plant spacing correction distance of the ginger at the time of sowing. The specific steps are as follows: Step 701: Based on the obtained 3D centroids of the ginger piece and ginger sprout in the world coordinate system, calculate their direction vectors and distance. The coordinates of the ginger piece are... The coordinates of the ginger sprout are , Step 702: Adjustment angle. Calculate the deflection angle of the ginger sprout center relative to the ginger piece center in the horizontal plane using the arctangent function. This is used to determine the orientation angle during sowing; the formula is: , Step 703, and the Euclidean distance between the ginger sprout and the ginger piece. This can be used to correct the planting spacing. When planting ginger in a row, the y and z coordinates of the ginger seed placement position are fixed in an ideal state. The X coordinate of each ginger seed placement position is corrected, i.e., xd, to ensure that the ginger sprout positions are consistent.

[0023] Step 704: Adjust the deflection angle between the ginger sprout and the ginger piece. Euclidean distance As input parameters for subsequent control of robotic arms or planting mechanisms, it enables automated orientation and precise seeding.

[0024] Example 2 The present invention also provides a ginger orientation and plant spacing control system based on instance segmentation. The system is used to implement the aforementioned method and includes: a transformation module, a preprocessing module, a construction module, a segmentation module, a calculation module, a selection module, and an adjustment module. The transformation module is used to set a point at the bottom of the white plastic box as the origin of the world coordinate system, perform coordinate calibration, and obtain the transformation matrix from the camera coordinate system to the world coordinate system. The preprocessing module is used to stack ginger seeds in a white plastic box, use a depth camera to collect RGB image datasets of the stacked ginger seeds, preprocess and label the images, and build a ginger seed dataset for training. A module is built to train a ginger seed instance segmentation model based on an instance segmentation model with an improved network structure and a ginger seed dataset. The segmentation module is used to segment ginger seeds in an image using a ginger seed instance segmentation model, obtain the mask regions of ginger pieces and ginger sprouts, and calculate their pixel centroid coordinates respectively. The calculation module is used to fill the holes in the depth map based on the pixel centroid coordinates of ginger pieces and ginger sprouts, and calculate the three-dimensional world coordinates of ginger pieces by combining the repaired depth map and the mask area. The selection module is used to select ginger varieties with the highest scores based on the three-dimensional world coordinates of ginger pieces. It employs a multi-factor sorting strategy based on a scoring function to prioritize the selection of ginger varieties for seed extraction. The adjustment module is used to determine the orientation angle and plant spacing correction during sowing by calculating the relative direction and distance between ginger sprouts and ginger rhizomes in world coordinates.

[0025] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for controlling ginger orientation and plant spacing based on instance segmentation, characterized in that, The method includes: S1. Let a point at the bottom of the white plastic box be the origin of the world coordinate system, perform coordinate calibration, and obtain the transformation matrix from the camera coordinate system to the world coordinate system. S2. Stack ginger seeds in a white plastic box, use a depth camera to collect RGB image datasets of the stacked ginger seeds, preprocess and label the images, and construct a ginger seed dataset for training. S3. Based on the improved network structure instance segmentation model and ginger seed dataset, a ginger seed instance segmentation model is trained to obtain the ginger seed instance segmentation model; S4. Use the ginger seed instance segmentation model to segment the ginger seed in the image, obtain the mask regions of ginger pieces and ginger sprouts, and calculate their pixel centroid coordinates respectively. S5. Based on the pixel centroid coordinates of ginger pieces and ginger sprouts, a depth map repair algorithm is used to fill the holes in the depth map. Combined with the repaired depth map and the mask area, the three-dimensional world coordinates of the ginger pieces are calculated. S6. Based on the three-dimensional world coordinates of ginger pieces, a multi-factor sorting strategy based on a scoring function is adopted to prioritize the ginger varieties with the highest scores for seed extraction. S7. By calculating the relative direction and distance between ginger sprouts and ginger rhizomes in world coordinates, the orientation angle and plant spacing correction at the time of sowing are determined. The method for segmenting ginger seeds in an image using a ginger seed instance segmentation model, obtaining the mask regions of ginger pieces and ginger sprouts, and calculating their pixel centroid coordinates includes: Step 401: Use the trained ginger instance segmentation model to perform inference on the real-time acquired images. The model outputs the mask region for each ginger piece and ginger sprout. Step 402: Using the space moment formula , Representing an image Space moments of order; Let be non-negative integers, representing the moments in... Order in direction; Indicates the x and y coordinates of the pixels in the output mask; Represents coordinate points The mask value at the specified location; calculate the zeroth moment of the mask region. , and the first moment and This allows us to obtain the pixel geometric center coordinates of each ginger piece and ginger sprout. , Based on the pixel centroid coordinates of ginger pieces and ginger sprouts, a depth map inpainting algorithm is used to fill in depth map holes. Combining the inpainted depth map with the masked area, the method for calculating the 3D world coordinates of the ginger pieces includes: Step 501: A depth map inpainting algorithm based on the biharmonic equation is used to fill in the hole regions. In the digital image, the Laplacian operator is discretized using a 3×3 convolution kernel as follows: ； And the two Laplace nuclei: ； in, Indicates the image in The depth value at that location; Represents the Laplace operator; Represents the discrete biharmonic operator; L represents the commonly used 3×3 discrete Laplace kernel; K represents the equivalent 5×5 double Laplacian convolution kernel obtained by convolving L with itself in two dimensions; P is a 3×3 depth image region centered at (x,y). Let Q be a 5×5 depth image region centered at (x,y). The average depth around the pixel geometric center coordinates of the ginger pieces and shoots in the repaired depth image is calculated. ,in, Indicates the coordinates of the current pixel; Represents the coordinates of any pixel within the neighborhood; Indicates the neighborhood radius, used to determine the surrounding coordinate range; , Represented by pixels Centered on, with radius The neighborhood set, that is, containing all those that satisfy... pixels , for The average depth at that location, Represented by pixels Centered on, with radius The neighborhood set, that is, containing all those that satisfy... pixels , for The number of pixels in the neighborhood For pixels The depth value at that location is obtained through the camera intrinsic parameter matrix. And the transformation matrix from camera coordinate system to world coordinate system Find the seeding point Finally, the three-dimensional position coordinates of the geometric center point of the ginger piece in the world coordinate system were obtained. This serves as the location for obtaining the seed of this ginger variety.

2. The method according to claim 1, characterized in that, The specific steps of S1 include: Step 101: Obtain the camera intrinsic parameter K and transform the pixel coordinate system to the camera coordinate system; Step 102: Select 4 points at the bottom and 4 points at the top of the white plastic box for SVD calibration to obtain the matrix for transforming the camera coordinate system to the world coordinate system.

3. The method according to claim 1, characterized in that, The specific steps of S2 include: Step 201: Fill the edges of the acquired RGB images and filter RGB images whose resolution meets the preset requirements; Step 202: Label the selected RGB images, use polygons to mark the outlines of ginger seeds and ginger sprouts in each image, record the labels, and save them as txt files; Step 203: Divide the ginger seed dataset into a training set and a validation set in a 7:3 ratio.

4. The method according to claim 1, characterized in that, The specific steps of S3 include: Step 301: Using the YOLO11-seg instance segmentation model as the base model, the single Backbone structure is improved into a dual-branch structure. The original module is named RGB_Backbone, and Depth_Backbone is introduced to train the depth map. Step 302: Introduce the SimAM and SGE attention mechanism modules into C3Ghost in Depth_Backbone; Step 303: Expand the original head structure of the YOLO11-seg instance segmentation model into four heads; Step 304: Iteratively train the instance segmentation network on the training set to obtain the improved ginger seed instance segmentation model.

5. The method according to claim 1, characterized in that, The specific steps of S4 include: Step 401: Use the trained ginger instance segmentation model to infer the real-time acquired images and output the mask region for each ginger piece and ginger sprout. Step 402: Using the spatial moment formula, calculate the zeroth and first moments of the mask region to obtain the pixel geometric center coordinates of each ginger piece and ginger sprout.

6. The method according to claim 1, characterized in that, The specific steps of S5 include: Step 501: Use the biharmonic equation algorithm to repair holes in the depth map; Step 502: In the repaired depth map, calculate the average depth around the pixel geometric center coordinates of ginger pieces and ginger sprouts to obtain the depth of ginger pieces; combine the pixel coordinates of the geometric center points of ginger pieces and ginger sprouts with the depth of ginger pieces, and transform them into the world coordinate system through camera intrinsic parameters and a matrix for transforming the camera coordinate system to the world coordinate system to obtain the three-dimensional coordinates of the geometric center point of ginger pieces as the ginger seed picking point.

7. The method according to claim 1, characterized in that, The specific steps of S6 include: Step 601: Use the confidence scores of ginger sprouts and ginger pieces identified by the ginger seed instance segmentation model to represent the confidence factor of the scoring function; Step 602: Use the world coordinate height of the geometric center point of the ginger piece to represent the distance factor of the scoring function; Step 603: Combine the confidence factor and distance factor into a weighted scoring function, sort all identified ginger pieces, and select the target with the highest score as the priority picking object.

8. The method according to claim 1, characterized in that, The specific steps of S7 include: Step 701: Calculate the deflection angle of the ginger sprout relative to the ginger piece. The deflection angle is used for the adjustment angle. Step 702: Calculate the Euclidean distance between the ginger sprout and the ginger rhizome. The Euclidean distance is used for plant spacing correction. Step 703: Use the deflection angle and Euclidean distance between the ginger sprout and the ginger rhizome as input parameters for subsequent control of the robotic arm or planting mechanism to achieve orientation adjustment and sowing.

9. A ginger orientation and plant spacing control system based on instance segmentation, the system being used to implement the method described in any one of claims 1-8, characterized in that, The system includes: a transformation module, a preprocessing module, a construction module, a segmentation module, a calculation module, a selection module, and an adjustment module; The transformation module is used to set a point at the bottom of the white plastic box as the origin of the world coordinate system, perform coordinate calibration, and obtain the transformation matrix from the camera coordinate system to the world coordinate system. The preprocessing module is used to stack ginger seeds in a white plastic box, use a depth camera to collect RGB image datasets of the stacked ginger seeds, preprocess and label the images, and construct a ginger seed dataset for training. The building module is used to train a ginger seed instance segmentation model based on an instance segmentation model with an improved network structure and a ginger seed dataset. The segmentation module is used to segment ginger seeds in the image using a ginger seed instance segmentation model, obtain the mask regions of ginger pieces and ginger sprouts, and calculate their pixel centroid coordinates respectively. The calculation module is used to fill the holes in the depth map using a depth map repair algorithm based on the pixel centroid coordinates of the ginger pieces and ginger sprouts, and to calculate the three-dimensional world coordinates of the ginger pieces by combining the repaired depth map and the mask area. The selection module is used to select the ginger variety with the highest score for seed extraction based on the three-dimensional world coordinates of the ginger piece and a multi-factor sorting strategy based on a scoring function. The adjustment module is used to determine the orientation angle and plant spacing correction during sowing by calculating the relative direction and distance between ginger sprouts and ginger pieces in world coordinates.

Images