A deep learning-based rice panicle length measurement method

Abstract

The application discloses a rice panicle length measurement method based on deep learning, belongs to the technical field of deep learning posture estimation and image skeleton extraction, and specifically comprises the following steps: firstly, key points are labeled on an original rice panicle picture set, a panicle neck joint dataset and a branch dataset are constructed, a key point detection model is trained respectively, and the key point detection model is used for detecting panicle neck joint key points and all branch tip key points of a to-be-measured rice panicle picture; a skeleton of the to-be-measured rice panicle picture is extracted, all key points are bound on the skeleton, a shortest path connecting a panicle neck joint binding point and each branch tip binding point is searched, and a path with the longest integral length is taken as the rice panicle length. The application adopts a strategy of "key point guided skeleton search", can effectively solve the measurement difficulty problem caused by the bending and overlapping of rice panicles, and has obvious automation and accuracy advantages when a large number of breeding sample data processing is involved.

Description

Technical Field

[0001] This invention belongs to the field of deep learning pose estimation and image skeleton extraction technology, specifically relating to a deep learning-based method for measuring the length of rice spikes. Background Technology

[0002] Rice panicle length is an important agronomic trait for measuring rice yield, quality, and lodging resistance. Accurate measurement of panicle length is crucial for rice breeding, phenotypic analysis, and variety improvement. Traditional measurement methods mainly rely on manual operation using rulers or measuring tapes. This method is not only time-consuming and labor-intensive, but also extremely inefficient. Furthermore, it is difficult to accurately measure curved panicles, leading to significant measurement errors. In addition, manual measurement is susceptible to subjective factors and cannot handle large batches of samples quickly and without damage.

[0003] Existing computer vision-based measurement methods typically use edge detection or simple skeleton extraction. However, when dealing with organisms like rice panicles, which have complex branching structures (including panicle neck nodes, branches, and grains), these methods often struggle to accurately determine the starting point (panicle neck node) and ending point (panicle tip) of the skeleton, leading to incorrect skeleton paths that fail to accurately reflect the true panicle length. Summary of the Invention

[0004] To address the measurement challenges posed by traditional methods and existing image processing techniques when dealing with curved and complex rice panicles, this invention provides a deep learning-based method for measuring rice panicle length, which offers significant advantages in automation and accuracy when processing large amounts of breeding sample data.

[0005] The technical solution adopted in this invention is as follows:

[0006] A deep learning-based method for measuring rice panicle length includes the following steps:

[0007] Step 1: Obtain the original set of rice panicle images and label each rice panicle image with the bounding rectangle of the panicle, the panicle neck node, the panicle node of all branches, the starting point of the last panicle awn at the end of the branch, and the bounding rectangle of the branch; the bounding rectangle of the branch needs to cover the entire branch area.

[0008] Step 2: Take the original rice panicle image set with the bounding rectangle of the panicle and the panicle neck node in the rice panicle image as the panicle neck node dataset; take the original rice panicle image set with the panicle node of each branch, the starting point of the last panicle awn at the end of the branch, and the bounding rectangle of the branch as the branch dataset.

[0009] Step 3: Construct key point detection models. Train key point detection models using the ear neck dataset and branch dataset respectively to obtain the corresponding ear neck detection model and branch tip detection model.

[0010] Step 4: Extract the foreground from the rice ear image to be tested. After binarization, the obtained foreground image is converted into a black and white mask image to obtain the rice ear mask. Extract the skeleton of the rice ear image to be tested based on the rice ear mask, and convert the coordinates of each skeleton point into tensors.

[0011] Step 5: Input the images of the rice panicles to be tested into the panicle neck detection model and the branch tip detection model respectively, and output the key points of the panicle neck and all the key points of the branch tips.

[0012] Find the skeleton point in the skeleton that is closest to the key point of the ear neck node, and use it as the ear neck node binding point;

[0013] Find the skeleton point in the skeleton that is closest to the key point at the end of each branch and use it as the corresponding binding point for the branch end;

[0014] Step 6: Construct multiple root node-leaf node groups, with the binding point of the spike neck node as the root node of the path and the binding point of each branch tip as the leaf node of the path.

[0015] For each root node-leaf node group, find the shortest path connecting the root node and the leaf node in the skeleton (the skeleton is acyclic) as a candidate spike length path;

[0016] Step 7: Calculate the integral length of each candidate panicle length path, and take the candidate panicle length path with the longest integral length as the panicle length of the rice panicle image to be tested.

[0017] Furthermore, in step 1, the bounding rectangle of the branch is obtained by marking at least one auxiliary point in the branch.

[0018] Furthermore, the key point detection model mentioned in step 3 is specifically the RTMPose model.

[0019] Furthermore, in step 3, the RTMPose-s model with a small number of parameters is trained using the ear neck dataset to obtain the ear neck detection model.

[0020] Furthermore, in step 3, the RTMPose-m model with a large number of parameters is trained using the branch dataset to obtain the branch tip detection model.

[0021] Furthermore, in step 4, the Bi-RefNet network is used for foreground extraction.

[0022] Furthermore, in step 4, after smoothing the black and white mask image, a pure rice ear mask is obtained.

[0023] Furthermore, in step 4, a median filtering algorithm is used for smoothing.

[0024] Furthermore, in step 4, the skeletonization algorithm is used to extract the skeleton.

[0025] Furthermore, step 6 uses the BFS (Breadth-First Search) algorithm to find the shortest path.

[0026] Furthermore, step 7 includes a smoothing process for the candidate spike length path before calculating the integral length.

[0027] Furthermore, the smoothing process in step 7 specifically involves: downsampling the candidate ear length path, using an interpolation algorithm to interpolate and fit the downsampled candidate ear length path to generate a smooth approximate curve, and calculating the integral length of the approximate curve as the integral length of the corresponding candidate ear length path.

[0028] Furthermore, the interpolation algorithm is the CubicSpline interpolation algorithm.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] 1) This invention relies on deep learning technology to realize intelligent measurement of ear length, which is convenient and efficient, eliminating the need for various instruments in traditional manual measurement, and saving a lot of repetitive and tedious manual labor.

[0031] 2) Compared with traditional ear length measurement schemes that rely solely on image processing, this invention has the advantages of high robustness and high accuracy. Traditional ear length measurement methods based on image processing technology are overly dependent on the shooting environment and have high requirements for lighting and background selection. Changes in lighting and the purity of the background can greatly affect the measurement error. The method of this invention benefits from the high robustness and high accuracy of deep learning algorithms, which can resist interference from various complex shooting environments while ensuring a low error rate.

[0032] 3) Compared with the traditional method of measuring ear length that relies solely on image processing, the method of this invention can specifically identify the neck node and the end of the branch, without the need for manual calibration of the neck node or manual removal of awns, thus truly realizing fully automated ear length measurement. Attached Figure Description

[0033] Figure 1 This is a schematic diagram showing the annotations of the rice ear image in Example 1;

[0034] Figure 2 This is a comparison image of rice ears before and after foreground extraction in Example 1;

[0035] Figure 3 This is the skeleton diagram of the rice ear image extracted in Example 1;

[0036] Figure 4The key point binding results for the rice ear image to be tested in Example 1;

[0037] Figure 5 This is a diagram of the length of the rice panicle obtained in Example 1. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0039] Example 1

[0040] This embodiment proposes a deep learning-based method for measuring rice panicle length, including the following steps:

[0041] Step 1: Obtain the original set of rice ear images and label each image as follows: Figure 1 The diagram shows the circumscribed rectangle of the rice panicle, the key points of the panicle neck node, and the key points of the panicle nodes (i.e., the starting point of the branch), the key points of the branch ends (i.e., the ending point of the branch), and the circumscribed rectangle of the branch; among them, the key point of the branch ends is specifically the starting point of the awn of the last panicle grain at the end of the branch.

[0042] Step 2: Divide the original dataset into two parts. Specifically, the original rice ear image set with the bounding rectangle of the rice ear and the key points of the ear neck node in the rice ear image is the ear neck node dataset; the original rice ear image set with the key points of the ear node, the key points at the end of the branch, and the bounding rectangle of the branch in the rice ear image is the branch dataset.

[0043] Step 3: The RTMPose model is used as the keypoint detection model. For the spikelet neck dataset (simple structure and obvious features), the RTMPose-s model with fewer parameters is trained to obtain the spikelet neck detection model; for the branch dataset (containing complex branch tip information), the RTMPose-m model with more parameters is trained to obtain the branch tip detection model. This hierarchical training strategy ensures the accuracy of keypoint detection while avoiding redundant computation caused by a single large model.

[0044] Step 4: Use the Bi-RefNet network to extract the foreground from the rice ear image to be tested, such as... Figure 2 As shown, the rice ears are separated from the complex background. The resulting foreground image is binarized and converted into a black-and-white mask. To eliminate noise and jagged edges on the mask, a median filtering algorithm is used to smooth the black-and-white mask, resulting in a clean rice ear mask. The skeletonization algorithm is then used to process the rice ear mask and extract the skeleton of the rice ear image to be tested, as shown below. Figure 3As shown, the (x,y) coordinates of each skeleton point are transformed into an N×2 dimensional NumPy tensor to facilitate subsequent skeleton point binding operations:

[0045] np.

[0046] Where N represents the total number of skeleton points, (x i ,y i ) represents the coordinates of the i-th point on the skeleton.

[0047] Step 5: Input the images of the rice panicles to be tested into the panicle neck detection model and the branch tip detection model respectively, and output the key points of the panicle neck and all the key points of the branch tips.

[0048] To map the key points of the panicle neck and the key points of the branch tips onto the skeleton, tensor operations are used to calculate the Euclidean distance from each key point to all points in the skeleton tensor. Then, the skeleton point closest to the key point of the panicle neck is found in the skeleton and used as the panicle neck binding point. The skeleton point closest to each key point of the branch tip is found in the skeleton and used as the corresponding branch tip binding point.

[0049] like Figure 4 As shown, this step "anchors" the abstract keypoint coordinates to the specific physical skeleton, providing accurate start and end positions for subsequent path search.

[0050] Step 6: Construct multiple root node-leaf node groups, with the binding point of the spike neck node as the root node of the path and the binding point of each branch tip as the leaf node of the path.

[0051] For each root node-leaf node group, the BFS algorithm is used to find the shortest path connecting the root node and the leaf node in the skeleton, which is used as a candidate spike length path.

[0052] Step 7: To eliminate the impact of jagged jitter generated during skeleton extraction on length calculation, the candidate panicle length paths are smoothed. Specifically, the candidate panicle length paths are downsampled (e.g., downsampling rate 1:20), and the CubicSpline interpolation algorithm is used to interpolate and fit the downsampled candidate panicle length paths, generating a smooth approximate curve. The integral length of the approximate curve is calculated as the integral length of the corresponding candidate panicle length path. The candidate panicle length path with the longest integral length is taken as the panicle length of the rice panicle image to be measured. Figure 5 The length of the red dashed line shown.

[0053] It should be noted that this is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for measuring the length of rice panicles based on deep learning, characterized in that, Includes the following steps: Step 1: Obtain the original set of rice panicle images and label each rice panicle image with the bounding rectangle of the panicle, the panicle neck node, the panicle node of all branches, the starting point of the last panicle awn at the end of the branch, and the bounding rectangle of the branch; the bounding rectangle of the branch needs to cover the entire branch area. Step 2: Take the original rice panicle image set labeled with the bounding rectangle of the panicle and the panicle neck node as the panicle neck node dataset; take the original rice panicle image set labeled with the panicle node of each branch, the starting point of the last panicle awn at the end of the branch, and the bounding rectangle of the branch as the branch dataset. Step 3: Construct key point detection models. Train key point detection models using the ear neck dataset and branch dataset respectively to obtain the corresponding ear neck detection model and branch tip detection model. Step 4: Extract the foreground from the rice ear image to be tested. After binarization, the obtained foreground image is converted into a black and white mask image to obtain the rice ear mask. Extract the skeleton of the rice ear image to be tested based on the rice ear mask, and convert the coordinates of each skeleton point into tensors. Step 5: Input the images of the rice panicles to be tested into the panicle neck detection model and the branch tip detection model respectively, and output the key points of the panicle neck and all the key points of the branch tips. Find the skeleton point in the skeleton that is closest to the key point of the ear neck node, and use it as the ear neck node binding point; Find the skeleton point in the skeleton that is closest to the key point at the end of each branch and use it as the corresponding binding point for the branch end; Step 6: Construct multiple root node-leaf node groups, with the binding point of the panicle neck node as the root node of the path and the binding point of each branch tip as the leaf node of the path; for each root node-leaf node group, find the shortest path connecting the root node and the leaf node in the skeleton, as the candidate panicle length path. Step 7: Calculate the integral length of each candidate panicle length path, and take the longest candidate panicle length path as the panicle length of the rice panicle image to be tested.

2. The method for measuring rice panicle length based on deep learning according to claim 1, characterized in that, In step 1, the bounding rectangle of the branch is obtained by marking at least one auxiliary point in the branch.

3. The rice panicle length measurement method based on deep learning according to claim 1, characterized in that, The key point detection model mentioned in step 3 is specifically the RTMPose model.

4. The rice panicle length measurement method based on deep learning according to claim 3, characterized in that, The RTMPose-s model was trained using the spikelet neck dataset to obtain the spikelet neck detection model.

5. The rice panicle length measurement method based on deep learning according to claim 3, characterized in that, The RTMPose-m model was trained using the branch dataset to obtain the branch tip detection model.

6. The rice panicle length measurement method based on deep learning according to claim 1, characterized in that, In step 4, the Bi-RefNet network is used for foreground extraction.

7. The rice panicle length measurement method based on deep learning according to claim 6, characterized in that, In step 4, the black and white mask image is smoothed using a median filtering algorithm to obtain the rice ear mask.

8. The rice panicle length measurement method based on deep learning according to claim 6, characterized in that, In step 4, the skeletonize algorithm is used to extract the skeleton.

9. The rice panicle length measurement method based on deep learning according to claim 1, characterized in that, Step 6 uses the BFS algorithm to find the shortest path.

10. The method for measuring rice panicle length based on deep learning according to claim 1, characterized in that, Step 7, before calculating the integral length, also includes a smoothing process for the candidate ear length path. Specifically, the candidate ear length path is downsampled, and the CubicSpline interpolation algorithm is used to interpolate and fit the downsampled candidate ear length path to generate a smooth approximate curve. The integral length of the approximate curve is then calculated as the integral length of the corresponding candidate ear length path.

Images