A method for automatically and non-destructively extracting soybean plant phenotypes

Abstract

This invention discloses a method for automatically and non-destructively extracting soybean plant phenotypes, comprising: acquiring soybean plant images; inputting the soybean plant images into a pre-trained YOLOv5s-S neural network model to identify pod regions and stem regions; obtaining coordinate information of pods and stems; obtaining the number of pods based on the pod coordinate information; and using the trained U... 2 The Net neural network model performs semantic segmentation on soybean plant images to obtain pseudo-color segmentation maps. The soybean plant images are then binarized to obtain binarized images. The pseudo-color segmentation maps and binarized images are superimposed, and an A*-based path search algorithm is used to process the superimposed image to obtain plant height, number of effective branches, and branch length. This invention's method for automatically and non-destructively extracting soybean plant phenotypes can calculate the number of pods, identify all effective branches, and measure plant height and branch length in a single scan, achieving satisfactory accuracy.

Description

Technical Field

[0001] This invention relates to the field of agronomy, and in particular to an automated and non-destructive method for extracting soybean plant phenotypes. Background Technology

[0002] Soybeans are one of the world's most important legume crops and a major source of high-quality protein for humans. The nutritional value of soybeans helps prevent heart disease and diabetes to some extent. Given the vital role soybeans play in the modern economic environment, cultivating high-quality, high-yield soybean varieties has become a focus for breeding experts. Phenotypic analysis of soybeans is crucial for screening superior varieties and is an important process in crop breeding, as well as a key step in cultivating high-quality soybeans. Obtaining stem-related phenotypic data has always been a challenge in soybean plant phenotypic extraction. Previously, stem-related phenotypic data were mainly obtained manually, requiring significant manpower and resources, and incurring high time costs, making it difficult to meet the needs of breeders. Furthermore, the accuracy and reliability of manual observation are easily affected by subjectivity.

[0003] A Chinese invention patent with application number CN202210870626.2 and publication number CN115205834A discloses an automatic method for obtaining stem-related phenotypes of soybean plants. This method combines target detection and directional search to identify the effective number of branches, branch angles, main stem length, and stem curvature in mature plants. However, this method may make errors in extracting phenotypes from plants with many branches or branches that bend downwards. Furthermore, this method obtains soybean plant phenotypes by connecting stem nodes with straight lines, which cannot reflect the true stem vein pattern. In actual production, plants often have dense pods, severely obscured stems, many branches, and secondary branches, which increases the difficulty of phenotype extraction. Currently, there is no complete method to obtain the phenotype of branch length in soybean plants. Summary of the Invention

[0004] This invention proposes an SPP-extractor (Soybean Plant Phenotype Extractor) algorithm, which aims to automatically and non-destructively extract pod-related and stem-related phenotypic features from densely growing and intact soybean plants. This not only improves the efficiency of soybean plant phenotypic extraction but also provides high-precision phenotypic extraction results, which is of great significance for soybean breeding research.

[0005] To achieve the above objectives, the present invention provides a method for automatically and non-destructively extracting soybean plant phenotypes, the method comprising the following steps:

[0006] Acquire images of soybean plants;

[0007] The soybean plant image is input into a pre-trained YOLOv5s-S neural network model to identify the pod region and stem region in the soybean plant image. Based on the identification results, the coordinate information of the pod and the coordinate information of the stem are obtained respectively.

[0008] The number of bean pods is obtained based on the coordinate information of the bean pods;

[0009] Using the trained U 2 The -Net neural network model performs semantic segmentation on the soybean plant image to obtain a pseudo-color segmentation map;

[0010] The soybean plant image is binarized to obtain a binarized image;

[0011] The pseudo-color segmentation image and the binarized image are superimposed, and the superimposed image is processed using a path search algorithm based on A* search to obtain the phenotypic features related to soybean plant stems.

[0012] Preferably, before the step of inputting the soybean plant image into a pre-trained YOLOv5s-S neural network model to identify the pod region and stem region in the soybean plant image, the method further includes:

[0013] Images of soybean plants were acquired, and the pods and stems of the soybean plants were labeled using the LabelImg image annotation tool. The tip area of ​​the pods was labeled as the pod category, and the continuous area on the stem was labeled as the stem category, thus obtaining the target detection dataset.

[0014] The target detection dataset is input into the initial YOLOv5 neural network model based on the SE attention mechanism for iterative training to obtain the pre-trained YOLOv5s-S neural network model.

[0015] Preferably, the training U is used 2 The steps of using the -Net neural network model to perform semantic segmentation on the soybean plant image to obtain a pseudo-color segmentation map specifically include: using the trained U... 2 The -Net neural network model performs semantic segmentation on the soybean plant image, classifying each pixel in the image into different semantic object categories, and extracting features of the main stem and all branches based on the semantic object categories to obtain a pseudo-color segmentation map of the stem region.

[0016] Preferably, the training U is used 2 Before the step of semantic segmentation of the soybean plant image using the Net neural network model, the method further includes:

[0017] Images of soybean plants were acquired, and the stem regions of the soybean plants were labeled using the LabelMe image annotation tool to obtain the semantic segmentation task dataset.

[0018] Input the semantic segmentation task dataset into the initial U 2 - The Net neural network model is iteratively trained to obtain the trained U. 2 -Net neural network model.

[0019] Preferably, the estimated cost function of the path search algorithm based on A* search is:

[0020] f(n) = g(n) + h(n)

[0021] Where f(n) represents the estimated cost from the starting point through node n to the ending point, g(n) represents the cost of moving from the starting point to node n, and h(n) represents the estimated cost from node n to the ending point.

[0022] Preferably, the soybean plant stem-related phenotypic features include plant height. Accordingly, the step of obtaining the soybean plant stem-related phenotypic features specifically includes: taking the highest and lowest points of the stem region in the superimposed image as the start and end points respectively, using an A* search-based path search algorithm to find the shortest path between the start and end points, taking the shortest path as the plant for the main stem, and obtaining the main stem length, which is the plant height.

[0023] Preferably, the step of obtaining the phenotypic features related to the soybean plant stem further includes: when the top of the main stem of the soybean plant bends downward, processing the superimposed image using the MSE algorithm, calculating and extracting the complete main stem path, and obtaining the plant height, a phenotypic feature related to the soybean plant stem, based on the main stem path.

[0024] Preferably, the MSE algorithm includes the following steps:

[0025] S10 uses five digits to represent each bounding box detected by the pre-trained YOLOv5s-S neural network model:

[0026] I = [0, x, y, w, h]

[0027] Where x and y represent the horizontal and vertical coordinates of the center of the rectangle, w and h represent the width and height of the rectangle, and the first number in rectangle I is an index with an initial value of 0. Given the number of stem regions identified in n images, and the images are represented by an n×5 matrix;

[0028] S11, sort all main stem regions in ascending order of y-coordinate;

[0029] S12, change the index values ​​of all regions on the main stem path determined by the path search algorithm based on A* search to 1, i.e., sr(i)∈p main Accordingly,

[0030] I[i]=[1,x,y,w,h],sr(i)∈p main

[0031] S13, continue reading the y-coordinate, find the next stem region sr(i) with the identifier 0, set it as the starting point and find the path from that point to the main stem vertex;

[0032] S14, when a path is found that bypasses the current main stem vertex, set the center of sr(i) as the new starting point of the main stem and extend the main stem along the downward bending part;

[0033] S15, repeat steps S13 and S14, and grow the main stem accordingly, until all stem areas have been checked and a complete main stem path is obtained. The plant height is then calculated based on the main stem path.

[0034] Preferably, the soybean plant stem-related phenotypic features further include the number of effective branches and the branch length. Accordingly, the step of obtaining the soybean plant stem-related phenotypic features further includes: processing the superimposed image using the BIL algorithm to obtain the number of effective branches and the branch length of the soybean plant stem-related phenotypic features, wherein the BIL algorithm performs branch identification and length measurement based on the n×5 matrix image.

[0035] Preferably, the BIL algorithm includes the following steps:

[0036] S20, initialize all rectangles I passing through the main path to an indicator value of 1;

[0037] S21, sequentially scan the main stem region where the indicator value of rectangle I is 0;

[0038] S22, Given sr(j), use an A*-based path search algorithm to find the path p from sr(j) to the end of the main stem. j ;

[0039] S23, when p is found j With the main path p main If they intersect, then set the intersection point as p. j The endpoint, and calculate p based on it. j The length of path p j The branch length is obtained from the length of the rectangle I;

[0040] S24, along p j All stem regions are updated with the indicator value of j to indicate that they belong to the j-th branch of the plant;

[0041] S25. Repeat steps S21 to S24 until all stem regions have non-zero indicator values ​​to obtain the effective number of branches and the length of all branches.

[0042] The beneficial effects of this invention are as follows: The method for automatically and non-destructively extracting soybean plant phenotypes combines object detection, semantic segmentation, image processing, and heuristic search to automatically and non-destructively obtain important phenotypic traits of soybean plants. First, to address the problem of pod occlusion, the standard YOLOv5s model was improved by adding an SE attention mechanism for object detection. The resulting model can accurately identify pods and stems and calculate the number of pods in a single scan. Second, considering that mature branches are usually curved and covered by pods, a branch identification and length measurement module of SPP-extractor was designed, combining image processing, object detection, semantic segmentation, and heuristic search. Experimental results on real plants show that, compared with manual measurement, SPP-extractor achieves better performance in terms of pod number, effective branch number, plant height, and branch length. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a method for automatically and non-destructively removing soybean plant phenotypic traits according to the present invention.

[0045] Figure 2 This invention annotates soybean plants as follows: (a) target detection data annotation; (b) semantic segmentation task annotation;

[0046] Figure 3 This is the network structure of the SE attention mechanism in this invention;

[0047] Figure 4 This is a diagram of the YOLOv5s-S neural network model structure used in this invention;

[0048] Figure 5 This is the result of comparing the number of pods and stems with the number of pods identified using the YOLOv5s-S neural network model in this invention;

[0049] Figure 6 These are the segmented images output by the present invention for identifying stems and branches: (a) is the original image, and (b,c) and (d,e) are generated by U-Net and U... 2- The result of Net semantic segmentation, (c,e) without superimposing the original image;

[0050] Figure 7 This invention compares the use of only binarized images and superimposed semantic segmentation images when extracting branches.

[0051] Figure 8 (a) is the original image of the soybean plant, (b) is the binarized image, (c) is the pseudo-color segmentation image, and (d) is the composite map used for pathfinding.

[0052] Figure 9 This is a diagram of the main stem extraction process of the present invention: (a) determining the starting point and ending point of the main stem, (b) determining the path, and (c) an enlarged view of the identified path marked with dashed lines;

[0053] Figure 10 The present invention uses the MSE algorithm to identify the downward-curving main stem in the following steps: (a) initial incomplete main stem path, (b) path expansion process, and (f) final complete main stem path.

[0054] Figure 11 This is the pseudocode of the MSE algorithm of this invention;

[0055] Figure 12 This invention describes the working process of using the BIL algorithm for branch identification and length measurement.

[0056] Figure 13 This is the pseudocode of the BIL algorithm of this invention;

[0057] Figure 14 The comparison is between manual measurements of various soybean plants and the values ​​obtained by the present invention using the SPP-extractor: (a) with downward-curving branches, (b) with secondary branches. Detailed Implementation

[0058] The technical solution of the present invention will now be clearly and completely described with reference to specific embodiments. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] See Figure 1 , Figure 1 The diagram shows a flowchart of a method for automatically and non-destructively extracting soybean plant phenotypes according to the present invention. In this embodiment, the method includes the following steps:

[0060] S1, Obtain an image of a soybean plant;

[0061] S2, input the soybean plant image into the pre-trained YOLOv5s-S neural network model to identify the pod region and stem region in the soybean plant image, and obtain the coordinate information of the pod and the coordinate information of the stem according to the identification results.

[0062] S3, obtain the number of bean pods based on the coordinate information of the bean pods;

[0063] S4, using the trained U 2 The -Net neural network model performs semantic segmentation on soybean plant images to obtain pseudo-color segmentation maps;

[0064] S5. Perform binarization processing on the soybean plant image to obtain a binarized image;

[0065] S6. The pseudo-color segmentation image and the binarized image are superimposed, and the superimposed image is processed by a path search algorithm based on A* search to obtain the phenotypic features related to soybean plant stems.

[0066] It should be noted that the SPP-extractor of this invention aims to automatically and non-destructively extract pod-related and stem-related phenotypic features from densely grown and intact soybean plants. To this end, this embodiment deconstructs the task into three sub-tasks and designs three modules accordingly: pod and stem region identification, stem and branch extraction, and branch length calculation. Figure 1 The overall structure of SPP-extractor is described. This invention's method for automatically and non-destructively extracting soybean plant phenotypic traits combines object detection, semantic segmentation, image processing, and heuristic search to automatically and non-destructively acquire important phenotypic traits of soybean plants. First, to address the problem of pod occlusion, the standard YOLOv5s model is improved by adding an SE attention mechanism for object detection. The resulting model can accurately identify pods and stems and calculate the number of pods in a single scan. Second, considering that mature branches are often curved and covered by pods, a branch recognition and length measurement module of SPP-extractor is designed, combining image processing, object detection, semantic segmentation, and heuristic search. Experimental results on real plants show that, compared to manual measurement, SPP-extractor achieves better performance in terms of pod number, effective branch number, plant height, and branch length.

[0067] Furthermore, prior to step S2, the method further includes a step of training the initial YOLOv5 neural network model, specifically including:

[0068] Images of soybean plants were acquired, and the pods and stems of the soybean plants were labeled using the LabelImg image annotation tool. The tip area of ​​the pods was labeled as the pod category, and the continuous area on the stem was labeled as the stem category, thus obtaining the target detection dataset.

[0069] The object detection dataset is input into the initial YOLOv5 neural network model based on the SE attention mechanism for iterative training to obtain the pre-trained YOLOv5s-S neural network model.

[0070] Furthermore, prior to step S4, the method further includes processing the initial U. 2 The training process of the -Net neural network model specifically includes:

[0071] Images of soybean plants were acquired, and the stem regions of the soybean plants were labeled using the LabelMe image annotation tool to obtain the semantic segmentation task dataset.

[0072] Input the semantic segmentation task dataset into the initial U 2 - The Net neural network model is iteratively trained to obtain the trained U. 2 -Net neural network model.

[0073] It should be noted that two soybean plant datasets were created in this embodiment, as shown in Table 1, for object detection and semantic segmentation tasks, respectively.

[0074] Table 1 Data Labeling

[0075]

[0076] This invention uses the LabelImg image annotation tool to annotate the pods and stems of soybean plants, saving the output location information in 'txt' format. The LabelMe image annotation tool is used to annotate the stems of soybean plants, saving the output in '.xml' format, as shown in the annotation example. Figure 2 As shown.

[0077] It should be understood that identifying pod and stem regions is the first subtask of the SPP-extractor, providing a basis for subsequent extraction. Object detection identifies instances of a specified object type from the input image. In this embodiment, there are two types of target objects: pods and stems. However, the severe overlap between pods and stems poses a significant challenge to standard object detection models. This embodiment addresses this issue from three aspects:

[0078] First, only the tip of the pod is marked as a substitute indicator, instead of marking the entire pod. This allows shaded pods to be identified as long as part of them is visible. Figure 2(a) shows an example with annotations. Each consists of four preset color dots (i.e., Figure 2 (a) The rectangle formed by the white dots (which could be marked as green dots in practice) marks a target: a pod or a stem. Similarly, considering that the pod remains on the plant, only continuous areas on the stem are marked as positive examples of stem areas.

[0079] Secondly, this embodiment proposes the YOLOv5 neural network as the basic model for this module. The YOLOv5 model is the fifth generation of the YOLO (You Only Look Once) series of object detection algorithms, and it has achieved a significant improvement in accuracy.

[0080] Finally, to improve the model's predictability for both dense and sparse regions, this embodiment adds an SE (Squeeze and Activate Network) attention mechanism to the YOLOv5s architecture, resulting in the YOLOv5s-S model. Unlike traditional convolutional pooling, which assigns equal weights to each channel of the feature map, the SE attention mechanism learns the importance of each channel and the task at hand, and weights each channel based on its relevance to the final task. Figure 3 The diagram illustrates this process: each channel of the feature map starts with equal importance and then acquires different weights after the squeezing and excitation processes, with different colors representing different weights. The learning process then focuses on the patterns in the channels with higher weights. The SE mechanism is typically integrated into the BottleNeck module in C3, forming SeC3. Figure 4 The structure of the YOLOv5s-S model is described in detail.

[0081] Given an annotated image, a YOLOv5s-S model is trained to identify pod and stem regions in the given image and output their coordinates. The total number of pods is then calculated in a single plant scan. The coordinates of the stem form the basis for subsequently extracting the number of effective branches, plant height, and branch length, as they are necessary for identifying the main stem and branches.

[0082] like Figure 5 As shown, it displays the detection results on very dense soybean plants. Because the stems are identified in segments, the detected stems are of different lengths than the labeled ones, resulting in more or fewer identifications. However, the difference in length does not affect the location of the stems.

[0083] Further, step S4 specifically includes: using the trained U 2 The -Net neural network model performs semantic segmentation on the soybean plant image, classifying each pixel in the image into different semantic object categories, and extracting features of the main stem and all branches based on the semantic object categories to obtain a pseudo-color segmentation map of the stem region.

[0084] It should be noted that stem and branch extraction is the second subtask of the SPP-extractor. Extracting stem-related phenotypic features first requires accurate branch identification. Furthermore, the identified branch regions must be as close as possible to the actual branches to ensure accurate height and length calculations. In densely planted plants, pods can negatively impact the stem area in two ways: first, when they overlap the top of the stem, they can obscure the stem region; second, when two or more pods accidentally connect and form a false stem on a board, they can mislead pathfinding. These factors can easily cause traditional path planning algorithms to deviate from the true stem region, resulting in inaccurate phenotypic data. Therefore, this embodiment first uses U... 2 The -Net neural network model extracts the stem region. Given an input image, semantic segmentation classifies each pixel in the image into one of several semantic object categories, such as stem, pod, and background in this embodiment.

[0085] It should be understood that SPP-extractor uses U 2 -Net neural network model for semantic segmentation. 2 -Net is a novel network architecture based on U-Net, employing an encoder-decoder architecture that has achieved great success in medical image segmentation. The encoder module extracts features from the input image, increasing the receptive domain by acquiring local image features and performing image-level classification. The decoder module performs upsampling by fusing features from several different scales. The overall network structure forms the letter 'U'. 2 -Net improves U-Net by nesting two U-shaped structures to enrich the scale of the included perceptual domain, thereby capturing more contextual information and enhancing segmentation performance.

[0086] In the specific implementation, given an input image, the stem and branch extraction submodule identifies all branches on the plant, including the main stem and short secondary branches growing from it, and marks them with bright green. Using this image, the SPP-extractor then measures the branch length, identifies the main stem and valid branches, and discards unwanted secondary branches. For semantic segmentation of the stem region of soybean plants, this embodiment uses U-Net and U... 2 - A comparative experiment was conducted between two semantic segmentation neural network models, namely -Net. Figure 6 An example of the output segmented image is shown; clearly, through... Figure 6 (c) As can be seen, pod density has a negative impact again; annotated branches are interrupted by pods, resulting in discontinuous branch paths. This affects the subsequent subtask of calculating branch length. Figure 6 (d) It can be seen that this embodiment uses U 2The -Net neural network model produces better segmented images for semantic segmentation.

[0087] Furthermore, branch length extraction is the third subtask of the SPP-extractor. Measuring branch length is typically modeled as finding the shortest path between two specific points on an image raster, such as from the top of the plant to the cotyledon node of the main stem, and from the top of a branch to its cotyledon node intersecting with the main stem of other branches. Generally, there are two options for generating the raster: binarization or semantic segmentation; however, densely planted soybean plants make each option insufficient when used alone. If a binarized image is used alone, pods overlapping with the stem will form unnecessary paths, such as... Figure 7 As shown by the dashed line in (a), the pathfinding process can easily deviate from the actual stem to overlapping pods, resulting in significant differences in path length. If only the segmented image generated by semantic segmentation is used, the path may be interrupted because the stem region is usually discontinuous and also due to occlusion by the pods. Therefore, the SPP-extractor combines the binarized and semantically segmented maps to form a complete map for subsequent pathfinding, such as... Figure 8 As shown.

[0088] In its implementation, given a pixel image with binarization and segmented branches, this invention develops a path search algorithm based on A* (A-Star) search to find the shortest end-to-end path through a stem or branch. The lengths of the stem and branch are measured as plant height and branch length, respectively, and the number of effective branches is counted based on the number of paths found by each branch. The estimated cost function of the A* search-based path search algorithm is:

[0089] f(n) = g(n) + h(n)

[0090] Given node n, f(n) represents the estimated cost of moving from the starting point to the destination via node n, g(n) represents the cost of moving from the starting point to node n, and h(n) represents the estimated cost of moving from node n to the destination. The A* algorithm evaluates the value of each candidate node for the next move and selects the node with the lowest cost for expansion until the target point is reached. First, the input image is compressed to one-eighth of its original width and height to reduce the size of the raster image required for the A* search. After binarization, denoising, and expansion, the input image is read from top to bottom and left to right, represented by a two-dimensional array A. Each element of A represents a pixel in the image; a black pixel is assigned a value of '0' (background), and a pixel that is part of a plant is assigned a value of '1'. The semantic segmentation map is similarly represented by a two-dimensional array B, with values ​​'1' and '0' representing the stem and background regions, respectively. The two arrays are then added element-wise to obtain three distinct values: '0', '1', and '2'. Elements with a value of '0' indicate that the corresponding pixel is identified as background in both maps and is therefore set as an impassable obstacle during the search. Elements with values ​​of '1' and '2' are possible nodes during the search, with movement costs set to 1 and 0.1 respectively. This is essentially designed to relax the restrictions on pathfinding and give higher preference to pixels that are identified as stem regions in both maps.

[0091] Furthermore, the phenotypic features related to soybean plant stems include plant height. Therefore, the first task of the branch length extraction submodule is to identify the main stem and measure its length to obtain the plant height. The specific steps include: taking the highest and lowest points of the stem region in the superimposed image as the start and end points respectively, using an A* search-based path search algorithm to find the shortest path between the start and end points, taking the shortest path as the plant with the given main stem, and obtaining the main stem length, which is the plant height.

[0092] It should be understood that plant height is an important phenotypic feature of soybeans, measuring the length from the cotyledon node to the apical meristem of the main stem. The highest and lowest points in the stem region of the color image obtained from semantic segmentation are used as the start and end points, respectively, and the shortest path between them is found using the A* algorithm described above, representing the plant as the given main stem. Figure 9 (b) The dashed line shows the identified main stem.

[0093] Furthermore, densely planted plants present a challenge: due to the weight of the pods concentrated in one area, the main stem may bend downwards at the top. The algorithm described above no longer applies to this situation, because the starting point of the main stem is not necessarily the highest point of all stem regions. Figure 10 (a) shows such an example. Figure 10 The dark gray line in (b) shows the initially extracted path. To address this issue, this embodiment proposes an algorithm, MSE (Main Stem Extraction), whose pseudocode is as follows: Figure 11 (As shown) to calculate and extract the complete main stem. The MSE algorithm includes the following steps ( Figure 10 (b)-(e) illustrate the process:

[0094] S10 uses five digits to represent each bounding box detected by the pre-trained YOLOv5s-S neural network model:

[0095] I = [0, x, y, w, h]

[0096] Where x and y represent the horizontal and vertical coordinates of the center of the rectangle, w and h represent the width and height of the rectangle, and the first number in rectangle I is an index with an initial value of 0. Given the number of stem regions identified in n images, and the images are represented by an n×5 matrix;

[0097] S11, Sort all main stem regions in ascending order of y coordinate. In this embodiment, the x and y coordinates start from the lower left corner and the upper left corner, respectively. Therefore, the stem regions are classified from the top to the bottom of the plant.

[0098] S12, change the index values ​​of all regions on the main stem path determined by the path search algorithm based on A* search to 1, i.e., sr(i)∈p main p main Indicates the main stem path, and correspondingly,

[0099] I[i]=[1,x,y,w,h],sr(i)∈p main

[0100] S13, continue reading the y-coordinate, find the next stem region sr(i) with the identifier 0, set it as the starting point and find the path from that point to the main stem vertex;

[0101] S14, when a path is found that bypasses the current main stem vertex, that is, the main stem region that bypasses the top of the current main stem path, the center of sr(i) is set as the new starting point of the main stem, and the main stem is extended to the region along the downward bending part;

[0102] S15. If more such stem regions are found, repeat steps S13 and S14 and grow the main stem accordingly until all stem regions have been checked and a complete main stem path is obtained. The plant height is then calculated based on the main stem path.

[0103] Furthermore, the phenotypic features related to soybean plant stems also include the number of effective branches and branch length. Therefore, the second task of the branch length extraction submodule is to identify branches and measure their length. A branch is a lateral branch extending from the main stem, and branch length is defined as the length from the end of the branch to its starting point on the main stem. The number of effective branches and branch length are important phenotypic features considered in plant breeding. In the collected soybean plant dataset, plants can have zero to up to twelve branches, some clustered around the main stem, while others are scattered. Some branches also have secondary branches that need to be discarded from the effective branch count. This embodiment proposes a BIL algorithm, which is designed for branch identification and length measurement in the same n×5 matrix. The pseudocode of the BIL algorithm is as follows: Figure 13 As shown. Figure 12 This explains the working process of the BILS algorithm, which specifically includes:

[0104] S20, In the case of identifying the main stem as described above, all rectangles I passing through the main path are initialized to an indicator value of 1;

[0105] S21, sequentially scan the main stem regions with a value of 0 in the rectangle I indicator. These regions can be the vertices of branches, since all main stem regions are sorted in ascending order of their y-coordinates.

[0106] S22, Given a sr(j), use an A*-based path search algorithm to find the path p from sr(j) to the end of the main stem. j ;

[0107] S23, when p is found j With the main stem path p main If they intersect, then set the intersection point as p. j The endpoint, and calculate p based on it. j Length;

[0108] S24, along p j All stem regions are updated with the indicator value of j to indicate that they belong to the j-th branch of the plant;

[0109] S25. Repeat steps S21 to S24 until all stem regions have non-zero indicator values ​​to obtain the effective number of branches and the length of all branches.

[0110] It should be noted that in real plants, a branch can have one or more smaller branches, i.e., secondary branches. These secondary branches are usually short: less than 3 cm in length, and do not produce inflorescences, so they are not counted as true branches when measured manually. However, the BIL algorithm can still identify and measure these secondary branches. Figure 14 The white lines in (b) show two secondary branches. If the path from sr(j) to the end of the main stem is p... jInstead, BIL distinguishes secondary branches by their base region. To align with manual measurement standards, BIL discards all such branches. Furthermore, considering that the branch originates at the center point of the main stem frame at the branch's top, we add half the diagonal of stem frame I to the planned path p. j The length of the branch is taken as the final branch length.

[0111] SPP-extractor automatically extracts three different stem-related phenotypic features: main stem length, number of branches in the plant, and length of each branch. Figure 14 The manually measured values ​​were compared with the values ​​measured by the algorithm of the present invention on two real samples.

[0112] In summary, SPP-extractor can effectively identify and measure the height of soybean plants, even when they are densely covered and bent by pods. Regarding branch identification and their length, SPP-extractor's BIL algorithm can identify all branches in most cases. More attractively, the algorithms of this invention are also highly efficient; on average, extracting all the aforementioned phenotypic features from a single plant takes only 2.8 seconds, much faster than manual measurement, and requires no human intervention, significantly improving the efficiency of soybean plant phenotypic extraction. The SPP-extractor algorithm of this invention can count the number of pods in a plant, identify all valid branches, and measure the plant height and branch length in a single scan without removing any pods from the plant, with satisfactory accuracy.

Claims

1. A method for automatically and non-destructively extracting soybean plant phenotypes, characterized in that, The method includes the following steps: Acquire images of soybean plants; The soybean plant image is input into a pre-trained YOLOv5s-S neural network model to identify the pod region and stem region in the soybean plant image. Based on the identification results, the coordinate information of the pod and the coordinate information of the stem are obtained respectively. The number of bean pods is obtained based on the coordinate information of the bean pods; The trained U 2 -Net neural network model performs semantic segmentation on the soybean plant image to obtain a pseudo-color segmentation map. The soybean plant image is binarized to obtain a binarized image; The pseudo-color segmentation image and the binarized image are superimposed, and the superimposed image is processed using a path search algorithm based on A* search to obtain the phenotypic features related to soybean plant stems. Before the step of inputting the soybean plant image into a pre-trained YOLOv5s-S neural network model to identify the pod and stem regions in the soybean plant image, the method further includes: Images of soybean plants were acquired, and the pods and stems of the soybean plants were labeled using the LabelImg image annotation tool. The tip area of ​​the pods was labeled as the pod category, and the continuous area on the stem was labeled as the stem category, thus obtaining the target detection dataset. The target detection dataset is input into the initial YOLOv5 neural network model based on the SE attention mechanism for iterative training to obtain the pre-trained YOLOv5s-S neural network model; The step of obtaining the phenotypic features of soybean plant stems further includes: when the top of the main stem of the soybean plant bends downward, the MSE algorithm is used to process the superimposed image, calculate and extract the complete main stem path, and obtain the plant height, a phenotypic feature of soybean plant stems, based on the main stem path. The MSE algorithm includes the following steps: S10 uses five digits to represent each bounding box detected by the pre-trained YOLOv5s-S neural network model: ； Where x and y represent the horizontal and vertical coordinates of the center of the rectangle, and w and h represent the width and height of the rectangle. The first number is an index with an initial value of 0, given... The number of stem regions identified in each image, and used as... A 5x5 matrix represents the image; S11, press Sort all main stem regions in ascending order of coordinates; S12, change the index values ​​of all regions on the main stem path determined by the path search algorithm based on A* search to 1, that is... Accordingly, ； in, Indicates the stem region. Indicates the main path; S13, continue reading Coordinates, find the next stem region with identifier 0. Set it as the starting point and find the path from that point to the top of the main stem; S14, when a path is found that bypasses the current main stem vertex, The center is set as the new starting point of the main stem, and the main stem is extended along the downward-curving part; S15, repeat steps S13 and S14, and grow the main stem accordingly, until all stem areas have been checked and a complete main stem path is obtained. The plant height is then calculated based on the main stem path.

2. The method for automatically and non-destructively extracting soybean plant phenotypes according to claim 1, characterized in that, The trained U 2 The step of performing semantic segmentation on the soybean plant image by the trained U 2 The step of performing semantic segmentation on the soybean plant image by the trained U -Net neural network model classifies each pixel in the image into different semantic object categories, and extracts features of the main stem and all branches according to the semantic object categories to obtain a pseudo-color segmentation map of the stem region.

3. The method for automatically and non-destructively extracting soybean plant phenotypes according to claim 1, characterized in that, The trained U 2 Before the step of semantic segmentation of the soybean plant image using the -Net neural network model, the method further includes: Images of soybean plants were acquired, and the stem regions of the soybean plants were labeled using the LabelMe image annotation tool to obtain the semantic segmentation task dataset. Input the semantic segmentation task dataset into the initial U 2 - The Net neural network model is iteratively trained to obtain the trained U. 2 -Net neural network model.

4. The method for automatically and non-destructively extracting soybean plant phenotypes according to any one of claims 1 to 3, characterized in that, The estimated cost function of the path search algorithm based on A* search is: ； in, This indicates the nodes passed from the starting point. Estimated cost to the destination, This indicates moving from the starting point to a node. The cost, Indicates from node Estimated cost to the destination.

5. The method for automatically and non-destructively extracting soybean plant phenotypes according to claim 4, characterized in that, The soybean plant stem-related phenotypic features include plant height. Accordingly, the steps for obtaining the soybean plant stem-related phenotypic features specifically include: taking the highest and lowest points of the stem region in the superimposed image as the start and end points respectively, using an A* search-based path search algorithm to find the shortest path between the start and end points, taking the shortest path as the plant for the main stem, and obtaining the main stem length, which is the plant height.

6. The method for automatically and non-destructively extracting soybean plant phenotypes according to claim 1, characterized in that, The soybean plant stem-related phenotypic features also include the number of effective branches and branch length. Correspondingly, the step of obtaining the soybean plant stem-related phenotypic features further includes: processing the superimposed image using the BIL algorithm to obtain the number of effective branches and branch length of the soybean plant stem-related phenotypic features, wherein the BIL algorithm is based on the... Branch identification and length measurement are performed on a ×5 matrix image.

7. The method for automatically and non-destructively extracting soybean plant phenotypes according to claim 6, characterized in that, The BIL algorithm includes the following steps: S20 will pass through all rectangles on the main path. All pointer values ​​are initialized to 1; S21, scan the rectangular frame sequentially. Main stem region with an indicator value of 0; S22, given A path search algorithm based on A* search is used to find the path from... Path to the end of the main stem ; S23, when discovered With the main path If they intersect, then set the intersection point as... The endpoint, and calculate accordingly. The length depends on the path. Length and rectangle Obtain the branch length; S24, along All stem regions were used The indicator values ​​are updated to indicate that they belong to the first [number]th ... One branch; S25. Repeat steps S21 to S24 until all stem regions have non-zero indicator values ​​to obtain the effective number of branches and the length of all branches.

Images