Paddy field early-stage seedling row accurate acquisition method based on leaf sheath identification

Abstract

The invention discloses a method for accurately acquiring early rice field seedling rows based on leaf sheath identification. The method comprises the following steps: constructing a field image data enhancement set of early rice field seedlings; constructing a target detection model used for positioning seedling leaf sheath key points; identifying seedling sheath key points in the to-be-identified seedling image by using the target detection model; performing perspective correction and clustering fitting on the seedling sheath key points in the to-be-identified seedling image; and mapping the extracted seedling rows back to a coordinate system on the seedling image to be identified. The accurate acquisition method for the early-stage rice seedling row in the rice field can effectively restrain the influence of wind disturbance and complex background on the acquisition accuracy of the early-stage soft and weak rice seedling row, provides reliable navigation for accurate operation of early-stage facilities in the rice field, and is particularly suitable for low-power-consumption embedded equipment.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent agricultural machinery and equipment, and particularly relates to a method for accurately acquiring early rice seedling rows in a rice field based on leaf sheath recognition. Background Art

[0002] As the application of intelligent agricultural machinery and equipment in rice field cultivation and management continues to deepen, the precision of automatic identification of rice seedling rows is becoming increasingly demanding for the refined operation of facilities. Traditional rice seedling row detection methods rely on image processing techniques such as color segmentation and edge detection, which are easily interfered with by weeds, algae, and leaf movement, resulting in insufficient accuracy. Semantic segmentation methods based on deep learning have high computational complexity and are difficult to meet real-time requirements. While target detection methods are highly efficient, traditional models have limited ability to extract features from slender, curved seedlings. Especially for early-stage rice seedlings in rice fields, existing leaf positioning methods suffer from significant errors under wind disturbances, and the large number of model parameters makes them difficult to deploy on the low-power devices of intelligent agricultural machinery.

[0003] For example, in the early days of robotic weeding in rice fields, the seedlings had a weak ability to resist bending, and their leaves were easily displaced by the wind. This resulted in a large error between the traditional whole-plant detection method and the ideal navigation line (i.e., the central axis between seedling rows) when generating the navigation line. Figure 3 d. Furthermore, as rice seedlings progress through the growth stages, their leaf extension direction, morphology, and structural characteristics undergo dynamic changes, further increasing the difficulty of detecting leaf stability. However, compared to leaves, the contact point between the seedling sheath and the soil is more stable and less affected by environmental disturbances and changes in growth stage. Summary of the Invention

[0004] The purpose of the invention is to provide a method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath recognition. By accurately extracting the key feature points of the seedling leaf sheaths, the impact of wind disturbances and changes in growth stages on positioning accuracy is reduced, thereby improving the navigation accuracy and operation stability of the weeding robot in complex field environments.

[0005] Technical solution: The method for accurately obtaining early rice seedling rows in rice fields based on leaf sheath identification according to the present invention comprises the following steps:

[0006] Step 1: construct an enhanced field image data set of early rice seedlings;

[0007] Step 2: Build a target detection model for locating key points of rice seedling sheaths and train the target detection model using the field image data enhancement set;

[0008] Step 3: read the image of the seedling to be identified, establish a coordinate system on the image of the seedling to be identified, and then use the target detection model to identify the key points of the seedling sheath in the image of the seedling to be identified, and output the key points of the seedling sheath in the image of the seedling to be identified;

[0009] Step 4: Perform perspective correction and cluster fitting on the key points of the seedling sheaths in the seedling image to be identified, and extract the seedling rows in the seedling image to be identified;

[0010] Step 5: Map the extracted rows of rice seedlings back to the coordinate system of the rice seedling image to be identified, and use the rice seedling image to be identified together with the rows of rice seedlings as the final detection result.

[0011] Furthermore, in step 1, the specific steps for constructing the enhanced field image data set of early rice seedlings are as follows:

[0012] Step 1.1, collect images of early rice seedlings in rice fields under different meteorological conditions, at different locations, and at different locations according to the set image resolution;

[0013] Step 1.2: Data enhancement processing is performed on the collected images of early rice seedlings in the rice fields, including cropping, rotation, brightness adjustment, and Gaussian noise injection.

[0014] In step 1.3, the data enhancement processing of each early rice seedling image in the rice field is used to construct a field image data enhancement set, and the field image data enhancement set is divided into a training set, a validation set, and a test set according to a set ratio value.

[0015] Furthermore, in step 2, the specific steps of constructing a target detection model for locating key points of seedling sheaths are as follows:

[0016] Step 2.1, setting the positioning range of the seedling sheath key point to the seedling area where the leaf ring extends to above the soil surface;

[0017] Step 2.2, build an improved YOLOv8 network model as the target detection model;

[0018] In step 2.3, the target detection model is trained using the training set, and then the trained target detection model is verified using the validation set. After the verification is passed, the target detection model is tested using the test set. After the test is passed, the target detection model is constructed.

[0019] Furthermore, in step 2.2, the constructed improved YOLOv8 network model includes a backbone network module, a neck module and a head module; the backbone network module includes the first Conv layer, the second Conv layer, the first C2f layer, the third Conv layer, the second C2f layer, the fourth Conv layer, the third C2f layer, the fifth Conv layer, the DSC-C2f layer, the CBAM layer, the SPPF layer and the C3Ghost layer connected in series in sequence; the neck module includes the first Up sample layer, the first Concat layer, the fourth C2f layer, the second Up sample layer, the second Concat layer, the fifth C2f layer, the sixth Conv layer, the third Concat layer, the sixth C2f layer, the seventh Conv layer, the fourth Concat layer and the seventh C2f layer connected in series in sequence; the head module includes the first Detect layer, the second Detect layer and the third Detect layer;

[0020] The first Conv layer, the second Conv layer, the third Conv layer, the fourth Conv layer, the fifth Conv layer, the sixth Conv layer and the seventh Conv layer are all used for feature extraction; the first C2f layer, the second C2f layer, the third C2f layer, the fourth C2f layer, the fifth C2f layer, the sixth C2f layer and the seventh C2f layer are all used to extract features and fuse information on the input feature map to obtain a higher level of feature representation; the DSC-C2f layer is used to enhance the feature extraction capability through adaptive deformation convolution; the CBAM layer is used to suppress background noise interference; the SPPF layer is used to capture multi-scale information; the C3Ghost layer is used to reduce the number of network parameters and computational complexity through redundant feature generation strategies; the first Up sample layer and the second Up sample layer are both used for upsampling; the first Concat layer, the second Concat layer, the third Concat layer and the fourth Concat layer are all used for feature fusion; the first Detect layer, the second Detect layer and the third Detect layer are all used to predict bounding boxes and categories;

[0021] The input end of the first Conv layer is used to access the seedling image to be identified, the output end of the second C2f layer is also connected to the input end of the second Concat layer, and the output end of the third C2f layer is also connected to the input end of the first Concat layer; the output end of the C3Ghost layer is respectively connected to the input end of the second Up sample layer and the input end of the fourth Concat layer; the output end of the fifth C2f layer is also connected to the input end of the first Detect layer; the input end of the sixth C2f layer is also connected to the input end of the second Detect layer; the input end of the seventh C2f layer is connected to the input end of the third Detect layer; the output ends of the first Detect layer, the second Detect layer and the third Detect layer respectively output seedling images marked with seedling sheath key points.

[0022] Furthermore, the DSC-C2f layer includes an eighth Conv layer, a Split layer, n DSC-Bneck layers, a fifth Concat layer, and a ninth Conv layer connected in series in sequence, and the output ends of the eighth Conv layer, the Split layer, and the n DSC-Bneck layers are also connected to the input end of the fifth Concat layer; the DSC-Bneck layer includes a tenth Conv layer, a DySnakeConv layer, an eleventh Conv layer, and an Add layer connected in series in sequence, and the input end of the tenth Conv layer is also connected to the input end of the Add layer; the DySnakeConv layer includes a twelfth Conv layer, a DSConv layer, and a sixth Concat layer, the input end of the twelfth Conv layer serves as the input end of the DySnakeConv layer, the output end of the twelfth Conv layer is connected to the input end of the DSConv layer, the output end of the DSConv layer is connected to the input end of the sixth Concat layer, and the output end of the sixth Concat layer serves as the output end of the DySnakeConv layer;

[0023] The eighth Conv layer, the ninth Conv layer, the tenth Conv layer, the eleventh Conv layer and the twelfth Conv layer are all used for feature extraction; the Split layer is used to segment the input tensor; the fifth Concat layer and the sixth Concat layer are both used for feature fusion; the Add layer is used to add the input tensor element by element; the DSConv layer is used to decompose the standard convolution into two steps: depthwise convolution and point-by-point convolution, reducing the amount of calculation and the number of parameters.

[0024] Furthermore, in step 4, the specific steps of extracting the rows of seedlings in the seedling image to be identified are:

[0025] Step 4.1, locating the vanishing point of the seedling in the seedling image to be identified based on the key points of the seedling sheath;

[0026] Step 4.2, performing top view correction on the seedling image to be identified and the key points of the seedling sheath according to the seedling vanishing point, to obtain a corrected top view of the seedling and each corrected key point;

[0027] Step 4.3: Perform row fitting on each correction key point to obtain the corrected seedling row.

[0028] Furthermore, in step 4.1, the specific steps of locating the vanishing point of the seedling in the seedling image to be identified based on the key points of the seedling sheath are as follows:

[0029] Step 4.1.1: Set the diameter of the dense area circle and the point count threshold. The diameter of the dense area circle is 8 to 12 pixels in length, and the point count threshold is 2 to 4.

[0030] Step 4.1.2, generating initial line segments for each row on the image of the seedling to be identified based on the obtained seedling sheath key points, so that each seedling sheath key point is distributed adjacently along the corresponding initial line segment;

[0031] Step 4.1.3, representing each generated row of initial line segments in the form of two-dimensional homogeneous coordinates, and extending each row of initial line segments toward the far end;

[0032] Step 4.1.4, obtain the intersection points of the extended initial line segments of each row, and then establish a regional circle on the image of the seedlings to be identified based on the diameter of the regional circle. When the number of intersection points in the regional circle reaches the maximum and exceeds the point threshold, stop moving, and use the regional circle at this time as the intersection-dense area, and then set the center point of the intersection-dense area as the seedling vanishing point.

[0033] Furthermore, in step 4.2, the specific steps of performing top view correction on the seedling image to be identified and the seedling sheath key points according to the seedling vanishing point are as follows:

[0034] Step 4.2.1: Establish a homography matrix H that describes the mapping relationship between the image of the seedling to be identified and the rectified top view of the seedling. Then, solve the homography matrix H based on the coordinates of the seedling vanishing point and the measured seedling row spacing.

[0035] Step 4.2.2, establishing a coordinate projection relationship between pixel coordinates in the seedling image to be identified and pixel coordinates in the rectified seedling top view according to the homography matrix H;

[0036] Step 4.2.3, correcting the seedling sheath key points in the seedling image to be identified into the corrected key points in the seedling top view according to the coordinate projection relationship;

[0037] Step 4.2.4, correct the initial line segment of the row to the corrected line segment in the top view of the seedling according to the coordinate projection relationship.

[0038] Furthermore, in step 4.3, the specific steps of performing row fitting on each correction key point to obtain the corrected seedling row are as follows:

[0039] Step 4.3.1, perform vertical ground projection on each correction key point in the corrected seedling top view to obtain each seedling projection point, and then put each seedling projection point adjacent to the same correction line segment into the same point set to obtain each projection point set;

[0040] Step 4.3.2: Set the reference search point, search angle, and search radius. Create two vertically diagonally opposite sector-shaped areas with the reference search point as the center, using the search angle and search radius. The bisectors of the central angles of the sector-shaped areas must be parallel to the correction line segments adjacent to the reference search point.

[0041] Step 4.3.3: Select a set of projection points, take a seedling projection point as the reference search point, establish two fan-shaped areas of the seedling projection point based on the search angle and search radius, then translate the correction line segment adjacent to the reference search point, and draw two boundary lines when translating to the two ends of the arc line of the fan-shaped area. The area between the two boundary lines is used as the search range, and then each seedling projection point within the search range is used as the search result of the same type as the current seedling projection point;

[0042] Step 4.3.4, determining whether all the seedling projection points in the current projection point set have obtained similar search results. If so, proceed to step 4.3.5; otherwise, return to step 4.3.3;

[0043] Step 4.3.5, comparing the sizes of the various similar search results corresponding to the current set of projection points, finding any similar search result with the largest number of seedling projection points, and performing a linear fit on each seedling projection point in the similar search result using the least squares method to obtain a fitting line corresponding to the current set of projection points;

[0044] Step 4.3.6, determine whether each set of projection points has obtained a fitted straight line. If no fitted straight line is obtained, return to step 4.3.3. If the corresponding fitted straight lines have been obtained, use each fitted straight line as the corrected seedling row.

[0045] Furthermore, in step 5, the specific steps of mapping the extracted seedling rows back to the coordinate system of the seedling image to be identified are:

[0046] Step 5.1, calculate the inverse homography matrix H of the homography matrix H -1 ;

[0047] Step 5.2, calculate the linear equations corresponding to each row of seedlings after correction, and then use the inverse homography matrix H -1Map each straight line equation back to the coordinate system on the seedling image to be identified.

[0048] Compared with the prior art, the present invention has the following beneficial effects: (1) the present invention makes full use of the growth characteristics of early seedlings, that is, the leaf sheath is more resistant to wind disturbance than the leaf in terms of tissue structure, so that the early seedling row acquisition method based on leaf sheath recognition can significantly reduce the wind disturbance error by about 70%, thereby improving the detection accuracy and computational efficiency; (2) the improved YOLOv8 model proposed in the present invention integrates the dynamic snake convolution (DSC-C2F), CBAM attention mechanism and C3Ghost lightweight module, and improves the mAP@50 index by 2.9%, and the model parameters are 0. The detection time of a single frame is less than 30ms, which significantly improves the detection accuracy and computational efficiency. (3) Compared with the traditional K-Means and DBSCAN algorithms, the VP-PSE strategy proposed in this paper reduces the lateral distance error by 46.3% (from 6.17 pixels to 3.31 pixels) and the angle error by 33.8% (from 5.76° to 3.82°). The single-frame fitting time is 29.18ms, which is suitable for low-power embedded devices of intelligent agricultural machinery and equipment, and provides a more stable seedling row detection solution for field automation operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Flowchart of the present invention;

[0050] Figure 2 This is a schematic diagram of the image of early rice seedlings under interference scenarios such as algae, weeds, and wind;

[0051] Figure 3 To obtain the autonomous navigation diagram of early rice seedling rows based on leaf recognition and sheath recognition;

[0052] Figure 4 This is the improved YOLOv8 model architecture diagram;

[0053] Figure 5 for Figure 4 The DSC-C2F module structure diagram in the figure;

[0054] Figure 6 for Figure 4 Block diagram of the CBAM channel attention module in

[15] ;

[0055] Figure 7 for Figure 4 Block diagram of the spatial attention module of CBAM in

[15] ;

[0056] Figure 8 Schematic diagram of the seedling row clustering and fitting method based on VP-PSE;

[0057] Figure 9Comparison of target detection results for leaf sheath recognition in interference scenarios such as algae and wind;

[0058] Figure 10 Comparison of results of seedling row detection methods for leaf sheath and leaf blade identification;

[0059] Figure 11 Error comparison of seedling row detection methods for leaf sheath and leaf blade identification. DETAILED DESCRIPTION

[0060] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0061] like Figure 1 As shown, the method for accurately obtaining early rice seedling rows in a rice field based on leaf sheath identification disclosed in the present invention includes the following steps:

[0062] Step 1: construct an enhanced field image data set of early rice seedlings;

[0063] Step 2: Build a target detection model for locating key points of rice seedling sheaths and train the target detection model using the field image data enhancement set;

[0064] Step 3: read the image of the seedling to be identified, establish a coordinate system on the image of the seedling to be identified, and then use the target detection model to identify the key points of the seedling sheath in the image of the seedling to be identified, and output the key points of the seedling sheath in the image of the seedling to be identified;

[0065] Step 4: Based on the vanishing point and perspective geometry (VP-PSE) strategy, perspective correction and cluster fitting are performed on the key points of the seedling sheaths in the seedling image to be identified, and the seedling rows in the seedling image to be identified are extracted;

[0066] Step 5: Map the extracted rows of rice seedlings back to the coordinate system of the rice seedling image to be identified, and use the rice seedling image to be identified together with the rows of rice seedlings as the final detection result.

[0067] Normally growing seedlings such as Figure 3 As shown in a, by accurately extracting the key feature points of the seedling sheath, the influence of wind disturbance and growth stage changes on positioning accuracy is reduced. Even if the outline boundary of the rice leaf is blurred in the image, this method can still accurately identify the leaf sheath position, ensuring the stability and reliability of the detection, thereby improving the navigation accuracy and operation stability of the weeding robot in complex field environments. Figure 3 In the horizontal direction, the positioning is adjusted according to the maximum tilt amplitude of the leaf sheath in the image; in the vertical direction, the detection range extends downward from the leaf ring (the junction between the leaf sheath and the leaf blade) to cover the seedling part above the soil surface, as shown in Figure 5. Figure 3 As shown in c.

[0068] Furthermore, in step 1, the specific steps for constructing the enhanced field image data set of early rice seedlings are as follows:

[0069] Step 1.1, collect images of early rice seedlings in rice fields under different meteorological conditions, at different locations, and at different locations according to the set image resolution, e.g. Figure 2 As shown in the figure, to evaluate and verify the adaptability of the subsequent detection model to complex environments, images of early rice seedlings in rice fields were collected under various weather conditions, including sunny, cloudy, and overcast days, with wind speeds ranging from 0 to 2 m / s. Dynamic deformation of leaves caused by wind was obvious. A total of 3,300 images were collected, covering scenes such as algae, weeds, and wind, with a resolution of 640×640. During this period, the images were acquired at different times and locations to ensure that there were no overlapping areas in the images.

[0070] Step 1.2: Perform data enhancement processing on the collected images of early rice seedlings in the rice fields, including cropping, rotation, brightness adjustment, and Gaussian noise injection. Data enhancement can enhance the recognition ability of the detection model in complex field environments.

[0071] Step 1.3: Use the data augmentation processed early rice seedling images to construct a field image data augmentation set, and divide the field image data augmentation set into a training set, a validation set, and a test set according to the set ratio value. Through cropping, rotation, brightness adjustment, and Gaussian noise injection, the expanded 3,300 images are divided into a training set, a validation set, and a test set in a ratio of 7:2:1;

[0072] Furthermore, in step 2, the specific steps of constructing a target detection model for locating key points of seedling sheaths are as follows:

[0073] Step 2.1: Set the positioning range of the seedling sheath key points to the seedling area where the leaf ring extends to the soil surface, such as Figure 3 As shown in the figure, to address the problem of early weak seedlings being easily disturbed by wind, the key point positioning range of the seedling sheath is located in the area where the leaf ring extends above the soil surface, avoiding positioning deviation caused by disturbance and improving the detection accuracy in the initial growth stage of the seedling row;

[0074] Step 2.2: Build an improved YOLOv8 network model as the target detection model, such as Figure 4 As shown in the figure, given the high real-time requirements for early precision operation equipment in rice fields, in order to achieve lightweight models and efficient processing, an improved YOLOv8 was constructed based on the YOLOv8n and YOLOv8s network models, taking into account both real-time and accuracy requirements;

[0075] In step 2.3, the target detection model is trained using the training set, and then the trained target detection model is verified using the validation set. After the verification is passed, the target detection model is tested using the test set. After the test is passed, the target detection model is constructed.

[0076] Furthermore, in step 2.3, when the target detection model is trained using the training set, the Focal Loss loss function is used to alleviate the category imbalance problem between seedlings and weeds. The initial learning rate (0.001) is dynamically adjusted in combination with cosine annealing learning rate scheduling, and mixed precision training (FP16 / FP32) is used to accelerate the calculation. The batch size is set to 32. The trained target detection model can accurately output the key points of the seedling sheath, providing reliable input for subsequent seedling row extraction.

[0077] Further, such as Figure 4 As shown, in step 2.2, the improved YOLOv8 network model constructed includes a backbone network module, a neck module and a head module, which enhances the feature extraction capability of the curved leaf sheath structure through adaptive deformation convolution; the backbone network module includes the first Conv layer, the second Conv layer, the first C2f layer, the third Conv layer, the second C2f layer, the fourth Conv layer, the third C2f layer, the fifth Conv layer, the DSC-C2f layer, the CBAM layer, the SPPF layer and the C3Ghost layer connected in series in sequence; the neck module includes the first Up sample layer, the first Concat layer, the fourth C2f layer, the second Up sample layer, the second Concat layer, the fifth C2f layer, the sixth Conv layer, the third Concat layer, the sixth C2f layer, the seventh Conv layer, the fourth Concat layer and the seventh C2f layer connected in series in sequence; the head module includes the first Detect layer, the second Detect layer and the third Detect layer;

[0078] The first Conv layer, the second Conv layer, the third Conv layer, the fourth Conv layer, the fifth Conv layer, the sixth Conv layer and the seventh Conv layer are all used for feature extraction, and each Conv layer is a standard convolution layer; the first C2f layer, the second C2f layer, the third C2f layer, the fourth C2f layer, the fifth C2f layer, the sixth C2f layer and the seventh C2f layer are all used to extract features and fuse information from the input feature map to obtain a higher level of feature representation; the DSC-C2f layer is a dynamic snake-shaped convolution layer, which is used to enhance the feature extraction capability through adaptive deformation convolution; the CBAM layer is a convolution block attention layer, which is used to suppress background noise interference; the SPPF (Spatial Pyramid Pooling Fast, spatial pyramid pooling module) layer is used to capture multi-scale information; the C3Ghost layer is used to reduce the amount of network parameters and computation through redundant feature generation strategy; the first Up The sample layer and the second Upsample layer are used for upsampling; the first Concat layer, the second Concat layer, the third Concat layer, and the fourth Concat layer are used for feature fusion; the first Detect layer, the second Detect layer, and the third Detect layer are used to predict bounding boxes and categories;

[0079] The DSC-C2f layer can enhance the network's ability to capture multi-scale features in rice seedling row recognition. By introducing a serpentine convolution structure, the DSC-C2f layer can adaptively adjust the path of the convolution kernel according to different areas of the input image to adapt to targets of different shapes and sizes. This structure effectively improves the recognition accuracy of subtle differences between the complex background of the rice field and the rice seedlings, especially in the case of cluttered background or dense seedling rows, and can more accurately locate the position of the seedling rows. The CBAM layer can automatically adjust the weight distribution of different channels and spatial positions in the feature map to highlight the key areas of the leaf sheath in the image. The C3Ghost layer can reduce the number of network parameters and computational complexity through a redundant feature generation strategy, and compress the total number of parameters to less than 5M through a single repeated structure, significantly reducing the amount of floating-point operations and achieving real-time inference efficiency with a single-frame detection time of less than 30ms, providing precision agricultural equipment with a detection solution that is both lightweight, high-precision and low-latency.

[0080] The input end of the first Conv layer is used to access the seedling image to be identified, the output end of the second C2f layer is also connected to the input end of the second Concat layer, and the output end of the third C2f layer is also connected to the input end of the first Concat layer; the output end of the C3Ghost layer is respectively connected to the input end of the second Up sample layer and the input end of the fourth Concat layer; the output end of the fifth C2f layer is also connected to the input end of the first Detect layer; the input end of the sixth C2f layer is also connected to the input end of the second Detect layer; the input end of the seventh C2f layer is connected to the input end of the third Detect layer; the output ends of the first Detect layer, the second Detect layer and the third Detect layer respectively output seedling images marked with seedling sheath key points.

[0081] The CBAM (Convolutional Block Attention Module) layer includes a channel attention module and a spatial attention module. The channel attention layer includes a maximum pooling layer, an average pooling layer, and a shared multi-layer perceptron, such as Figure 6 As shown; the spatial attention layer includes the maximum pooling layer, the average pooling layer, the convolution layer and the Sigmoid activation function, as shown Figure 7 As shown in the figure; the maximum pooling layer is used to perform maximum pooling on each channel of the input feature map, extract the most significant features of each channel, and form global information; the average pooling layer is used to perform average pooling on each channel of the input feature map to generate global average information for each channel; the multilayer perceptron is used to perform nonlinear transformation on the pooled features through the fully connected layer, further capture the dependencies between channels, and generate weights for each channel. MLP is usually composed of one or more fully connected layers and is processed by an activation function (ReLU); the convolution operation is used to process the pooled features through the convolution layer (1x1 convolution) to generate a spatial attention map. These convolution layers help the model learn the relationship between different spatial regions in the image; the Sigmoid activation function is used to process the convolution output through the Sigmoid activation function to limit the value of the spatial attention map to between [0,1], indicating the importance of each spatial position.

[0082] Further, such as Figure 5As shown, the DSC-C2f (dynamic snake convolution module) layer includes the eighth Conv layer, the Split layer, the n DSC-Bneck layers, the fifth Concat layer and the ninth Conv layer connected in series in sequence, and the output ends of the eighth Conv layer, the Split layer and the n DSC-Bneck layers are also connected to the input end of the fifth Concat layer; the DSC-Bneck layer includes the tenth Conv layer, the DySnakeConv layer, the eleventh Conv layer and the Add layer connected in series in sequence, and the input end of the tenth Conv layer is also connected to the input end of the Add layer; the DySnakeConv layer includes the twelfth Conv layer, the DSConv layer and the sixth Concat layer, the input end of the twelfth Conv layer serves as the input end of the DySnakeConv layer, the output end of the twelfth Conv layer is connected to the input end of the DSConv layer, the output end of the DSConv layer is connected to the input end of the sixth Concat layer, and the output end of the sixth Concat layer serves as the output end of the DySnakeConv layer;

[0083] The eighth Conv layer, the ninth Conv layer, the tenth Conv layer, the eleventh Conv layer and the twelfth Conv layer are all used for feature extraction; the Split layer is used to segment the input tensor; the fifth Concat layer and the sixth Concat layer are both used for feature fusion; the Add layer is used to add the input tensor element by element; the DSConv (dynamic snake convolutional network) layer is used to decompose the standard convolution into two steps: depthwise convolution and point-by-point convolution, reducing the amount of calculation and the number of parameters.

[0084] Furthermore, in step 4, perspective correction and cluster fitting are performed on the key points of the seedling sheaths in the seedling image to be identified based on the vanishing point and perspective geometry (VP-PSE) strategy, and the specific steps for extracting the seedling rows in the seedling image to be identified are as follows:

[0085] Step 4.1, locating the vanishing point of the seedling in the seedling image to be identified based on the key points of the seedling sheath;

[0086] Step 4.2, performing top view correction on the seedling image to be identified and the key points of the seedling sheath according to the seedling vanishing point, to obtain a corrected top view of the seedling and each corrected key point;

[0087] Step 4.3: Perform row fitting on each correction key point to obtain the corrected seedling row.

[0088] Furthermore, in step 4.1, the specific steps of locating the vanishing point of the seedling in the seedling image to be identified based on the key points of the seedling sheath are as follows:

[0089] Step 4.1.1: Set the diameter of the dense area circle and the point count threshold. The diameter of the dense area circle is 8 to 12 pixels in length, and the point count threshold is 2 to 4. The diameter of the dense area circle is preferably set to 10 pixels in length, and the point count threshold is preferably set to 3.

[0090] Step 4.1.2, according to the key points of the seedling sheath {(x i ,y i )} Generate initial line segments for each row on the seedling image to be identified, and the key points of the seedling sheath {(x i ,y i )}like Figure 8 As shown in a, the generated initial line segment is as follows Figure 8 As shown in b, the key points of each seedling sheath are distributed close to each other along the initial line segment of the corresponding row;

[0091] Step 4.1.3, generate each initial line segment in two-dimensional homogeneous coordinates l i =(a i ,b i ,c i ) T , and extend the initial line segments of each row to the far end, such as Figure 8 As shown in c;

[0092] Step 4.1.4, obtain the intersection points of the extended initial line segments of each row, and then establish a regional circle on the image of the seedlings to be identified based on the diameter of the regional circle. When the number of intersection points in the regional circle reaches the maximum and exceeds the point threshold, stop moving, and use the regional circle at this time as the intersection-dense area, and then set the center point of the intersection-dense area as the seedling vanishing point.

[0093] Furthermore, in step 4.2, the specific steps of performing top view correction on the seedling image to be identified and the seedling sheath key points according to the seedling vanishing point are as follows:

[0094] Step 4.2.1: Establish a homography matrix H, which is a 3×3 matrix, to describe the mapping relationship between the image of the seedling to be identified and the rectified top view of the seedling. The homography matrix H is then solved based on the coordinates of the seedling vanishing point and the measured seedling row spacing.

[0095] Step 4.2.2: Establish pixel coordinates p = (x, y, 1) in the image of the seedling to be identified based on the homography matrix H. T The pixel coordinates of the corrected seedling top view p′=(x′,y′,1) T The coordinate projection relationship p′~Hp, the top view of the seedling is as follows Figure 8 As shown in d;

[0096] Step 4.2.3, according to the coordinate projection relationship, the key points of the seedling sheath {(x i ,y i )} is corrected to the correction key point {(x′ i ,y′ i )};

[0097] Step 4.2.4, correct the initial line segment of the row to the corrected line segment in the top view of the seedling according to the coordinate projection relationship.

[0098] Furthermore, in step 4.3, the specific steps of performing row fitting on each correction key point to obtain the corrected seedling row are as follows:

[0099] Step 4.3.1, perform vertical ground projection on each correction key point in the corrected seedling top view to obtain each seedling projection point, and then put each seedling projection point adjacent to the same correction line segment into the same point set to obtain each projection point set;

[0100] Step 4.3.2, set the reference search point, search angle, and search radius. The search angle is preferably 5°, and the search radius is preferably 5 pixels long. Based on the search angle and search radius, establish two vertically diagonal fan-shaped areas with the reference search point as the center. The bisector of the central angle of the fan-shaped area is parallel to the correction line segment adjacent to the reference search point, as shown in the following example: Figure 8 d and 8e;

[0101] Step 4.3.3: Select a set of projection points, take a seedling projection point as the reference search point, establish two fan-shaped areas of the seedling projection point based on the search angle and search radius, then translate the correction line segment adjacent to the reference search point, and draw two boundary lines when translating to the two ends of the arc line of the fan-shaped area. The area between the two boundary lines is used as the search range, and then each seedling projection point within the search range is used as the search result of the same type as the current seedling projection point;

[0102] Step 4.3.4, determining whether all the seedling projection points in the current projection point set have obtained similar search results. If so, proceed to step 4.3.5; otherwise, return to step 4.3.3;

[0103] Step 4.3.5, comparing the sizes of the various similar search results corresponding to the current set of projection points, finding any similar search result with the largest number of seedling projection points, and performing a linear fit on each seedling projection point in the similar search result using the least squares method to obtain a fitting line corresponding to the current set of projection points;

[0104] Step 4.3.6, determine whether each set of projection points has obtained a fitted straight line. If no fitted straight line is obtained, return to step 4.3.3. If the corresponding fitted straight lines have been obtained, use each fitted straight line as the corrected seedling row.

[0105] Furthermore, in step 5, the specific steps of mapping the extracted seedling rows back to the coordinate system of the seedling image to be identified are:

[0106] Step 5.1, calculate the inverse homography matrix H of the homography matrix H -1 , for the corrected coordinate p′, the corresponding original image coordinates satisfy p~H -1 p′;

[0107] Step 5.2, calculate the linear equations corresponding to each row of seedlings after correction, and then use the inverse homography matrix H -1 Map each straight line equation back to the coordinate system of the seedling image to be identified, such as Figure 8 As shown in f.

[0108] The method for accurately obtaining early rice seedling rows based on leaf sheath identification disclosed in the present invention was verified by experiments, taking algae and wind as two interference environments as examples. The experimental results are as follows: Figures 9-11 shown.

[0109] In order to verify the detection performance of the model of the method for accurately acquiring early rice seedling rows based on leaf sheath recognition in different field environments, experiments were conducted under two interference environments: algae and wind. The results are shown in Figure 2. Figure 9 In an algae-disturbed environment, such as Figure 9 As shown in a, although this model can accurately detect most seedlings, the identification of individual seedlings still has certain challenges, such as Figure 9 As shown in the white circle in the middle, this may be because the algae are similar in color to the leaves or the morphology of the dead leaves interferes with the model's ability to extract features from the sheath area. Figure 9 As shown in Figure f, the visibility of some seedlings is reduced due to leaf swaying, which affects the model's detection accuracy. This shows that the model's detection accuracy is somewhat affected in both algae and wind interference environments, but overall recognition performance remains good. This shows that the leaf sheath recognition method proposed in this paper demonstrates strong robustness and good detection performance across a variety of field environments.

[0110] In order to verify the accuracy of the early seedling row fitting based on VP-PSE, the manually annotated seedling rows were used as a benchmark to calculate the average lateral distance and angle between the fitting results and the manually annotated rows. The search angle was set to 5° and the search radius was set to 5 pixel array lengths. Assuming that K represents the accuracy of fitting the early seedling rows, represents the average lateral distance error, represents the average angle error, then K, and It can be calculated by formula (1).

[0111]

[0112] Where N is the number of images that correctly fit the early seedling rows, M is the total number of images participating in the test, U is the total number of point pairs selected at the same horizontal position on the two straight lines, and L is the total number of points selected at the same horizontal position on the two straight lines. n Represents the horizontal distance of the nth point pair, α i is the angle error of the i-th image.

[0113] The early seedling row fitting test dataset contains 120 images of seedlings captured in complex scenarios, including missing seedlings, inter-row interspersed plants, and irregular seedling rows. These complex scenarios do not appear independently; they often alternate or coexist within the same seedling row, significantly increasing the difficulty of seedling row fitting. The VP-PSE algorithm, combined with typical clustering algorithms (K-Means and DBSCAN) and the least squares method, was used to obtain early seedling rows. The results were compared with manual labeling, and are shown in Table 1.

[0114] Table 1 Comparison of detection accuracy results of VP-PSE algorithm, K-Means algorithm and DBSCAN algorithm

[0115]

[0116] As shown in Table 1, the VP-PSE algorithm achieved a fitting accuracy of 93.33% in 120 test images, with 112 images successfully fitted, an average lateral distance of 3.31, and an average angular error of 3.82°, which are significantly lower than those of the K-Means and DBSCAN algorithms (6.17 and 4.83, 4.67° and 5.76°, respectively). This shows that the VP-PSE algorithm is superior in fitting accuracy (K) and average angular error. The VP-PSE algorithm outperforms the K-Means and DBSCAN algorithms in all aspects. The fitting time for a single image is similar between the VP-PSE algorithm (29.18ms) and the K-Means algorithm (28.33ms). The DBSCAN algorithm takes 49.27ms, the longest of the three algorithms. This demonstrates that the VP-PSE algorithm not only provides highly accurate fitting results but also boasts high computational efficiency.

[0117] Under two disturbance environments, algae and wind, the results of obtaining early seedling rows based on leaf sheath and leaf recognition were compared, such as Figure 10 and 11 shown. Figure 10In the figure, the yellow lines are the manually marked early seedling rows, the red lines are the early seedling rows identified based on leaf sheaths, and the white lines are the early seedling rows identified based on leaves.

[0118] When faced with algae disturbing the environment, Figure 11 It can be seen that the errors of obtaining early seedling rows based on leaf sheath and leaf recognition are 3.26 and 6.73 respectively; the reason why the error of early seedling rows based on leaf recognition is too large is that the visible part of the leaves in the edge area is reduced and the angle of the leaves changes, which increases the difficulty of row recognition. Figure 10 a It was also found that, especially as the seedling rows approached the edge of the image, the differences between rows gradually increased.

[0119] When facing wind interference environment, Figure 11 It can be seen that the errors of obtaining early seedling rows based on leaf sheath and leaf recognition are 3.38 and 11.57, respectively. Obviously, the influence of wind interference environment on the errors of obtaining early seedling rows based on leaf sheath and leaf recognition is greater than that of algae interference environment, especially the influence on the errors of obtaining early seedling rows based on leaf recognition is more significant.

[0120] It can be seen that the error changes in obtaining early seedling rows based on leaf sheath recognition under the two interference environments of algae and wind are not obvious. This recognition method can accurately identify the relatively stable key points of early seedling leaf sheaths, effectively suppress the interference of seedling leaves, and can well solve the problem of accurate identification of early seedling rows.

[0121] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for accurately obtaining early rice seedling rows in rice fields based on leaf sheath recognition, characterized in that: The following steps are involved: Step 1: construct an enhanced field image data set of early rice seedlings; Step 2: Build a target detection model for locating key points of rice seedling sheaths and train the target detection model using the field image data enhancement set; Step 3: read the image of the seedling to be identified, establish a coordinate system on the image of the seedling to be identified, and then use the target detection model to identify the key points of the seedling sheath in the image of the seedling to be identified, and output the key points of the seedling sheath in the image of the seedling to be identified; Step 4: Perform perspective correction and cluster fitting on the key points of the seedling sheaths in the seedling image to be identified, and extract the seedling rows in the seedling image to be identified; Step 5: Map the extracted rows of rice seedlings back to the coordinate system of the rice seedling image to be identified, and use the rice seedling image to be identified together with the rows of rice seedlings as the final detection result.

2. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath identification according to claim 1, characterized in that: In step 1, the specific steps for constructing the enhanced field image data set of early rice seedlings are as follows: Step 1.1, collect images of early rice seedlings in rice fields under different meteorological conditions, at different locations, and at different locations according to the set image resolution; Step 1.2: Data enhancement processing is performed on the collected images of early rice seedlings in the rice fields, including cropping, rotation, brightness adjustment, and Gaussian noise injection. In step 1.3, the data enhancement processing of each early rice seedling image in the rice field is used to construct a field image data enhancement set, and the field image data enhancement set is divided into a training set, a validation set, and a test set according to a set ratio value.

3. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath identification according to claim 2, characterized in that: In step 2, the specific steps for constructing a target detection model for locating key points of seedling sheaths are as follows: Step 2.1, setting the positioning range of the seedling sheath key point to the seedling area where the leaf ring extends to above the soil surface; Step 2.2, build an improved YOLOv8 network model as the target detection model; In step 2.3, the target detection model is trained using the training set, and then the trained target detection model is verified using the validation set. After the verification is passed, the target detection model is tested using the test set. After the test is passed, the target detection model is constructed.

4. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath identification according to claim 3, characterized in that: In step 2.2, the constructed improved YOLOv8 network model includes a backbone network module, a neck module and a head module; the backbone network module includes the first Conv layer, the second Conv layer, the first C2f layer, the third Conv layer, the second C2f layer, the fourth Conv layer, the third C2f layer, the fifth Conv layer, the DSC-C2f layer, the CBAM layer, the SPPF layer and the C3Ghost layer connected in series in sequence; the neck module includes the first Up sample layer, the first Concat layer, the fourth C2f layer, the second Up sample layer, the second Concat layer, the fifth C2f layer, the sixth Conv layer, the third Concat layer, the sixth C2f layer, the seventh Conv layer, the fourth Concat layer and the seventh C2f layer connected in series in sequence; the head module includes the first Detect layer, the second Detect layer and the third Detect layer; The first Conv layer, the second Conv layer, the third Conv layer, the fourth Conv layer, the fifth Conv layer, the sixth Conv layer and the seventh Conv layer are all used for feature extraction; the first C2f layer, the second C2f layer, the third C2f layer, the fourth C2f layer, the fifth C2f layer, the sixth C2f layer and the seventh C2f layer are all used to extract features and fuse information on the input feature map to obtain a higher level of feature representation; the DSC-C2f layer is used to enhance the feature extraction capability through adaptive deformation convolution; the CBAM layer is used to suppress background noise interference; the SPPF layer is used to capture multi-scale information; the C3Ghost layer is used to reduce the amount of network parameters and computation through redundant feature generation strategy; the first Up sample layer and the second Up sample layer are both used for upsampling; The first Concat layer, the second Concat layer, the third Concat layer, and the fourth Concat layer are all used for feature fusion; the first Detect layer, the second Detect layer, and the third Detect layer are all used to predict bounding boxes and categories; The input end of the first Conv layer is used to access the seedling image to be identified, the output end of the second C2f layer is also connected to the input end of the second Concat layer, and the output end of the third C2f layer is also connected to the input end of the first Concat layer; the output end of the C3Ghost layer is respectively connected to the input end of the second Up sample layer and the input end of the fourth Concat layer; the output end of the fifth C2f layer is also connected to the input end of the first Detect layer; the input end of the sixth C2f layer is also connected to the input end of the second Detect layer; the input end of the seventh C2f layer is connected to the input end of the third Detect layer; the output ends of the first Detect layer, the second Detect layer and the third Detect layer respectively output seedling images marked with seedling sheath key points.

5. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath recognition according to claim 4, characterized in that: The DSC-C2f layer includes an eighth Conv layer, a Split layer, n DSC-Bneck layers, a fifth Concat layer, and a ninth Conv layer connected in series in sequence. The output ends of the eighth Conv layer, the Split layer, and the n DSC-Bneck layers are also connected to the input end of the fifth Concat layer. The DSC-Bneck layer includes a tenth Conv layer, a DySnakeConv layer, an eleventh Conv layer, and an Add layer connected in series in sequence. The input end of the tenth Conv layer is also connected to the input end of the Add layer. The DySnakeConv layer includes a twelfth Conv layer, a DSConv layer, and a sixth Concat layer. The input end of the twelfth Conv layer serves as the input end of the DySnakeConv layer, the output end of the twelfth Conv layer is connected to the input end of the DSConv layer, the output end of the DSConv layer is connected to the input end of the sixth Concat layer, and the output end of the sixth Concat layer serves as the output end of the DySnakeConv layer. The eighth Conv layer, the ninth Conv layer, the tenth Conv layer, the eleventh Conv layer and the twelfth Conv layer are all used for feature extraction; the Split layer is used to segment the input tensor; the fifth Concat layer and the sixth Concat layer are both used for feature fusion; the Add layer is used to add the input tensor element by element; the DSConv layer is used to decompose the standard convolution into two steps: depthwise convolution and point-by-point convolution, reducing the amount of calculation and the number of parameters.

6. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath identification according to claim 3, characterized in that: In step 4, the specific steps of extracting the rows of seedlings in the seedling image to be identified are: Step 4.1, locating the vanishing point of the seedling in the seedling image to be identified based on the key points of the seedling sheath; Step 4.2, performing top view correction on the seedling image to be identified and the key points of the seedling sheath according to the seedling vanishing point, to obtain a corrected top view of the seedling and each corrected key point; Step 4.3: Perform row fitting on each correction key point to obtain the corrected seedling row.

7. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath recognition according to claim 6, characterized in that: In step 4.1, the specific steps for locating the vanishing point of the seedling in the seedling image to be identified based on the key points of the seedling sheath are as follows: Step 4.1.1: Set the diameter of the dense area circle and the point count threshold. The diameter of the dense area circle is 8 to 12 pixels in length, and the point count threshold is 2 to 4. Step 4.1.2, generating initial line segments for each row on the image of the seedling to be identified based on the obtained seedling sheath key points, so that each seedling sheath key point is distributed adjacently along the corresponding initial line segment; Step 4.1.3, representing each generated row of initial line segments in the form of two-dimensional homogeneous coordinates, and extending each row of initial line segments toward the far end; Step 4.1.4, obtain the intersection points of the extended initial line segments of each row, and then establish a regional circle on the image of the seedlings to be identified based on the diameter of the regional circle. When the number of intersection points in the regional circle reaches the maximum and exceeds the point threshold, stop moving, and use the regional circle at this time as the intersection-dense area, and then set the center point of the intersection-dense area as the seedling vanishing point.

8. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath recognition according to claim 7, characterized in that: In step 4.2, the specific steps for performing top view correction on the seedling image to be identified and the key points of the seedling sheath are as follows: Step 4.2.1: Establish a homography matrix H that describes the mapping relationship between the image of the seedling to be identified and the rectified top view of the seedling. Then, solve the homography matrix H based on the coordinates of the seedling vanishing point and the measured seedling row spacing. Step 4.2.2, establishing a coordinate projection relationship between pixel coordinates in the seedling image to be identified and pixel coordinates in the rectified seedling top view according to the homography matrix H; Step 4.2.3, correcting the seedling sheath key points in the seedling image to be identified into the corrected key points in the seedling top view according to the coordinate projection relationship; Step 4.2.4, correct the initial line segment of the row to the corrected line segment in the top view of the seedling according to the coordinate projection relationship.

9. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath recognition according to claim 8, characterized in that: In step 4.3, the specific steps for performing row fitting on each correction key point to obtain the corrected seedling row are as follows: Step 4.3.1, perform vertical ground projection on each correction key point in the corrected seedling top view to obtain each seedling projection point, and then put each seedling projection point adjacent to the same correction line segment into the same point set to obtain each projection point set; Step 4.3.2: Set the reference search point, search angle, and search radius. Create two vertically diagonally opposite sector-shaped areas with the reference search point as the center, using the search angle and search radius. The bisectors of the central angles of the sector-shaped areas must be parallel to the correction line segments adjacent to the reference search point. Step 4.3.3: Select a set of projection points, take a seedling projection point as the reference search point, establish two fan-shaped areas of the seedling projection point based on the search angle and search radius, then translate the correction line segment adjacent to the reference search point, and draw two boundary lines when translating to the two ends of the arc line of the fan-shaped area. The area between the two boundary lines is used as the search range, and then each seedling projection point within the search range is used as the search result of the same type as the current seedling projection point; Step 4.3.4, determining whether all the seedling projection points in the current projection point set have obtained similar search results. If so, proceed to step 4.3.5; otherwise, return to step 4.3.3; Step 4.3.5, comparing the sizes of the various similar search results corresponding to the current set of projection points, finding any similar search result with the largest number of seedling projection points, and performing a linear fit on each seedling projection point in the similar search result using the least squares method to obtain a fitting line corresponding to the current set of projection points; Step 4.3.6, determine whether each set of projection points has obtained a fitted straight line. If no fitted straight line is obtained, return to step 4.3.

3. If the corresponding fitted straight lines have been obtained, use each fitted straight line as the corrected seedling row.

10. The method for accurately acquiring early rice seedling rows in rice fields based on leaf sheath recognition according to claim 8, characterized in that: In step 5, the specific steps of mapping the extracted seedling rows back to the coordinate system of the seedling image to be identified are: Step 5.1, calculate the inverse homography matrix H of the homography matrix H -1 ; Step 5.2, calculate the linear equations corresponding to each row of seedlings after correction, and then use the inverse homography matrix H -1 Map each straight line equation back to the coordinate system on the seedling image to be identified.

Images