Rice root and shoot tracking method and system based on two-stage segmentation and topological feature fusion

Abstract

This invention discloses a method and system for tracking rice roots and shoots based on two-stage segmentation and topological feature fusion. The method first acquires several frames of seed images, then obtains individual seeds through instance segmentation, followed by semantic segmentation to obtain the roots, shoots, and main body of each seed. Finally, topological graph matching is used to track the main body of the seed. Since the roots, shoots, and main body belong to the same individual in the image, root and shoot tracking is achieved simultaneously. This invention enables automated and precise monitoring of the dynamic growth of root and shoot growth in single seeds in dense environments, providing reliable technical support for dynamic monitoring of rice germination and phenotypic patterns.

Description

Technical Field

[0001] This invention relates to rice seed root and shoot tracking, and in particular to a method and system for rice root and shoot tracking based on two-stage segmentation and topological feature fusion. Background Technology

[0002] The seed germination stage is a crucial phase in the rice life cycle, significantly impacting its response to salt stress. This stage involves a dramatic transition from dormancy to metabolic activation. The dynamic responses of roots and shoots during this period contain rich information on salt tolerance and are a core element in the selection of salt-tolerant varieties. Key phenotypic indicators such as germination rate, root length, and shoot length are closely related to varietal salt tolerance. Monitoring these dynamic phenotypes can reveal seed physiological mechanisms, providing important evidence for the breeding of superior rice varieties. Traditional methods for identifying salt tolerance rely on manual observation, which suffers from low efficiency, high subjectivity, poor repeatability, and weak dynamic monitoring capabilities. Furthermore, they often depend on final static indicators, neglecting the spatiotemporal dynamic differences in root and shoot growth, making them unsuitable for large-scale germplasm resource screening.

[0003] With the rapid development of plant phenomics, high-throughput phenotyping techniques based on imaging technologies such as visible light and depth cameras are gradually being applied to crop phenotyping, driving the transformation of phenotypic parameter extraction towards non-destructive and high-throughput methods. Multi-source data fusion and computer vision technologies have also provided new possibilities for seed germination phenotyping analysis. Existing studies have attempted to obtain root and shoot phenotypic characteristics through image analysis, but these have mostly focused on non-occluded environments. Because plant roots have complex branching structures, they are prone to entanglement and occlusion during growth. Current technologies are mainly suitable for analyzing discrete root systems without entanglement or root systems requiring single-seed isolation culture, and are difficult to directly apply to densely packed seed populations.

[0004] Furthermore, existing multi-target tracking methods often rely on target appearance features in dense scenes, but the tracking performance deteriorates significantly when there is occlusion, irregular and similar target shapes. For example, in pedestrian and vehicle tracking scenarios, the target shape is stable and the individual boundaries are clear. Existing methods can rely on rigid motion models or appearance features to achieve tracking, but these characteristics are not suitable for root and shoot tracking. In dense scenes, rice seed roots and shoots are slender, irregular in shape, and have multiple branches and bends. During germination, roots and shoots intertwine, and root and shoot growth is accompanied by nonlinear deformations such as bending, forking, and extension. Therefore, it is difficult to track rice seed roots and shoots in dense scenes, making accurate and efficient monitoring of rice seed germination morphology a challenge that urgently requires technological breakthroughs. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a rice root and shoot tracking method and system based on two-stage segmentation and topological feature fusion, which solves the technical problems of seed individual tracking loss and organ affiliation breakage under dense shading conditions in the prior art.

[0006] Technical solution: The rice root and shoot tracking method based on two-stage segmentation and topological feature fusion described in this invention includes the following steps:

[0007] (1) Collect several frames of images of rice seeds throughout the entire germination stage, with each frame containing several seeds;

[0008] (2) Perform instance segmentation on the image to obtain the region information of each seed individual;

[0009] (3) Based on the regional information of each seed individual, crop the seed individual image, and perform semantic segmentation on the cropped seed individual image to obtain the root, shoot and main body region information of the seed individual;

[0010] (4) Extract the centroid coordinates of the main seed region as spatial nodes, construct an adjacency graph, and establish seed tracking in several frames of images based on the adjacency graph to complete root tracking and bud tracking in several frames of images.

[0011] Furthermore, the method for tracking the seed subject in several frames of images based on the adjacency graph in step (4) includes the following steps:

[0012] In the adjacency graph of two consecutive frames, the correlation difference value between the seed main region in the previous frame and the seed main region in the next frame is calculated, and the pair of seed main regions with the smallest correlation difference value is selected by a greedy strategy to establish a correlation relationship.

[0013] The correlation difference value is calculated by the spatial distance, topological difference, and difference in bud growth direction between the main seed region in the previous frame image and the main seed region in the next frame image.

[0014] Furthermore, the associated difference value ;

[0015] In the previous frame image, the node corresponding to the main seed region is P, and in the next frame image, the node corresponding to the main seed region is Q. This represents the normalized Euclidean distance between P and Q. Let P and Q be the absolute differences between the sum of the distances of P and Q to their respective K nearest neighbors in their adjacency graphs after normalization. The difference in bud growth direction corresponding to P and Q mapped to the interval [0,1] is represented by α, β, and λ, which are coefficients. .

[0016] Furthermore, when the seed corresponding to node Q is in an ungerminated state, the directional difference weight λ is set to zero, and its weight value is proportionally distributed to α and β, i.e., the weights are updated. , ,at this time The ungerminated state is defined as a seed bud length pixel value being less than 50% of the minimum bounding rectangle width pixel value of the seed body.

[0017] Furthermore, in step (2), the iShapeInst network is used for instance segmentation, wherein an EMA attention module is embedded in each BasicBlock of the feature extraction backbone network Stage2~Stage5 in the iShapeInst network.

[0018] Furthermore, in step (3), the DeepLabv3+ network is used for semantic segmentation. The backbone network of the DeepLabv3+ network is MobileNetV2, and an adaptive spatial feature pooling module is inserted into the encoder of the DeepLabv3+ network.

[0019] The rice root and shoot tracking system based on two-stage segmentation and topological feature fusion described in this invention includes:

[0020] The image acquisition unit is used to acquire several frames of images throughout the entire germination stage of rice seeds, with each frame containing several seeds.

[0021] The seed individual segmentation unit is used to perform instance segmentation on the image to obtain the region information of each seed individual;

[0022] The seed individual organ segmentation unit is used to crop the seed individual image based on the regional information of each seed individual, and to perform semantic segmentation on the cropped seed individual image to obtain the root, shoot and main body region information of the seed individual;

[0023] The seed body tracking unit is used to extract the centroid coordinates of the seed body region as spatial nodes, construct an adjacency graph, and establish seed body tracking in several frames of images based on the adjacency graph, thereby completing root tracking and bud tracking in several frames of images.

[0024] The electronic device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the rice root and shoot tracking method based on two-stage segmentation and topological feature fusion.

[0025] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the rice root and shoot tracking method based on two-stage segmentation and topological feature fusion.

[0026] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows: The present invention proposes a three-stage processing architecture based on instance segmentation-semantic segmentation-topology graph matching, which realizes automated monitoring of the root and shoot growth dynamics of single seeds, provides reliable technical support for dynamic monitoring of rice germination and phenotypic growth, and is of great significance for screening superior germplasm resources, shortening the breeding cycle, and improving breeding efficiency. Attached Figure Description

[0027] Figure 1 This is a flowchart of the rice seed root and shoot tracking method of the present invention.

[0028] Figure 2 This is a schematic diagram illustrating the instance segmentation effect at different densities in an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the individual tracking effect of mapping the subject tracking ID to the instance segmentation mask in an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of root and bud tracking in an embodiment of the present invention. Detailed Implementation

[0031] Existing technologies often fail to effectively track the root and shoot germination of rice seeds under dense conditions, making it difficult to adapt to the unique characteristics of rice roots and shoots, such as their slender shape, large root curvature angles, and numerous branches. In dense scenarios where roots and shoots are intertwined and obscuring each other, accurate tracking of individual seed roots and shoots becomes challenging. This invention proposes a three-stage processing architecture based on instance segmentation, semantic segmentation, and topological graph matching. First, an instance segmentation network segments intertwined and obscuring individual rice seeds. Then, a lightweight semantic segmentation model analyzes the roots, shoots, and main organs of a single seed. Finally, a spatial topological relationship is constructed based on the centroid of the seed's main body mask. By integrating a comprehensive association model that considers spatial distance differences, topological structure differences, and shoot growth direction differences, combined with a dynamic weight adjustment mechanism and a greedy matching strategy, accurate tracking of individual rice seed roots and shoots in dense scenarios is achieved. The technical solution of this invention is further described below with reference to the accompanying drawings. The overall process of this invention is as follows: Figure 1 As shown in (a): Input rice seed germination time-series image data, process it through a topological feature fusion tracking algorithm, and output the root and shoot tracking results of each seed in the time dimension. Specifically, it includes the following core stages:

[0032] In the instance segmentation stage, the rice seed instance segmentation network is an improvement upon the iShapeInst network architecture, adding deformable convolutions and EMA attention mechanisms to enhance the network's segmentation of fine structures such as root tips, shoot tips, and boundaries, thereby improving the model's segmentation accuracy for the dense and slender structures during rice germination. The EMA attention mechanism is embedded in each BasicBlock of the feature extraction backbone network (Stages 2-5). Channels are divided into four groups, each retaining original channel information to reduce detail loss due to dimensionality reduction; 1×1 convolutions are used to encode global spatial information along the height and width directions to locate the overall seed outline; 3×3 convolutions are used to capture local seed texture features; and pixel-level attention maps are used to redistribute channel weights to highlight the boundary features between the seed and the background.

[0033] A deformable convolutional layer is embedded before the pyramid pooling module of the network encoder to learn the offset. Make the convolution kernel adapt to the root and shoot bending morphology:

[0034] ;

[0035] in, To output feature map at location The value at that location, For the input feature map, The sampling grid is for a regular convolution kernel. For sampling network The nth offset coordinate in For the convolution kernel at position The weight parameters, This is an adaptive offset for the root-shoot morphology predicted by independent convolutional layers. This deformable convolutional layer is cascaded with a pyramid pooling module to improve the modeling capability of dense, interwoven root-shoot topologies through multi-scale feature optimization.

[0036] like Figure 1 As shown in (b), for each frame of the input temporal image, a pre-trained instance segmentation network generates the pixel region of each seed individual and outputs the pixel-level mask, bounding box coordinates and confidence score of each seed individual. Different colors represent different seed individual mask images.

[0037] II. Semantic segmentation stage: The rice seed organ segmentation network is based on the lightweight semantic segmentation architecture of DeepLabv3+. The backbone network adopts MobileNetV2 and introduces an adaptive spatial feature pooling module to enhance the discrimination ability of small organs such as root tips and coleoptiles.

[0038] like Figure 1As shown in (c), the seed individual mask output by instance segmentation is used to crop the RGB region of the seed individual from the original image using the minimum bounding rectangle; semantic segmentation is performed on this region to segment three types of organs: root (green region), bud (blue region), and seed body (red region).

[0039] III. Graph Matching Stage of Root-Shoot Tracking: To establish the association between seed subjects across different frames, the algorithm constructs and utilizes a topological graph structure. This topological graph matching stage, based on the image region of each seed subject output by semantic segmentation, implements cross-frame seed association according to the following process.

[0040] 1) Adjacency Graph Construction: Calculate the centroid coordinates of the seed subject mask in each seed semantic segmentation and transform them to the original image coordinate system as spatial nodes. Calculate the Euclidean distance between all pairs of nodes, and establish undirected edge connections between each node and only its K nearest other nodes to generate an adjacency graph structure representing the local proximity relationships between seed subjects.

[0041] 2) Cross-frame association calculation: For a specific seed main node P in the previous frame and a candidate seed main node Q in the current frame, the algorithm calculates their comprehensive association difference value by fusing three difference indices. Addressing the irregular positional movement caused by individual growth, global positional offset due to population synchronization displacement, irregular seed morphology, and cross-frame association errors caused by dense occlusion and morphological similarity during rice seed germination, this invention integrates three complementary features—spatial distance difference, topological structure difference, and shoot growth direction difference—to construct a comprehensive association function: spatial distance difference uses normalized Euclidean distance to address small-scale individual displacement interference; topological structure difference utilizes the stability of local adjacency graph relationships to mitigate the impact of population displacement; and shoot growth direction difference distinguishes spatially adjacent seeds based on the directional continuity during germination. The centroid of the seed main mask is chosen as the reference tracking node because the morphological stability of the main mask is superior to other regions during germination.

[0042] (a) Spatial distance difference: Calculate the Euclidean distance between nodes P and Q. This distance, after being normalized by min-max, is used as a measure of spatial difference, denoted as . The smaller the value, the closer the two nodes are in spatial location and the stronger their correlation.

[0043] (b) Topological differences: Calculate the sum of distances between nodes P and Q and their K nearest neighbors (K=5 in this embodiment) in their respective adjacency graphs. , Take the absolute value of the difference between the two. After min-max normalization, it is used as a measure of topological difference, denoted as . The smaller the difference, the higher the similarity of the local neighborhood structure of the two nodes in their respective frames.

[0044] (c) Differences in bud growth direction: Extract the bud contours corresponding to nodes P and Q. Based on the node coordinates, locate the point on the contour closest to the node as the bud base, and the point on the contour farthest from the bud base as the bud tip. Calculate the direction vector from the bud base to the bud tip. and After normalization, the directional differences are mapped to the interval [0,1] through vector dot product and linear transformation, denoted as... The smaller the value, the higher the consistency of bud growth direction between two frames, where the direction is perpendicular. It is 0.5, 0 when they are completely in the same direction, and 1 when they are completely opposite.

[0045] Finally, by combining these three normalized difference measures, a comprehensive correlation difference value is obtained:

[0046] ;

[0047] in Weights are based on spatial distance differences. For topological difference weights, The weights for differences in bud growth direction are given, and the weighting coefficients satisfy the following conditions: (In this embodiment, it is set) , , ).

[0048] If the target seed is in an ungerminated state (the bud length pixel value is less than 50% of the minimum bounding rectangle width pixel value of the seed body), then the orientation difference weight will be adjusted. Set the value to zero, and proportionally redistribute the original directional difference weights to the spatial and topological difference terms:

[0049] ,

[0050] At this point, the overall correlation difference value is updated to:

[0051] .

[0052] 3) Cross-frame matching and trajectory update: After constructing the S-value cost matrix based on all node pairs, the effective node pairs with the smallest S-value are selected iteratively through a greedy strategy to establish a unique matching relationship and dynamically reduce the matrix. Finally, the status labels of the matched pairs and unmatched nodes are output.

[0053] For successfully matched node pairs, their ID (tracking identifier) ​​and corresponding root and shoot data are updated; unmatched nodes in the current frame are marked as new seeds, and unmatched nodes in the previous frame are marked as removed from the field of view, thus realizing the tracking of the entire growth cycle of seed germination.

[0054] Through the graph matching mechanism described above, the spatiotemporal relationship between seed subjects in different frames is effectively established, such as... Figure 1 As shown in (d), cross-frame tracking of the seed body is achieved by calculating spatial distance, topological structure and bud growth direction features. Since the root and bud belong to the same entity in the image, by tracking the seed body, its associated root and bud organs automatically inherit the same tracking ID number, thus achieving synchronous tracking of the root and bud.

[0055] The method described in this invention will be verified through specific experiments below.

[0056] (1) Data collection and dataset construction.

[0057] Seeds of different rice varieties were placed in separate culture boxes for seed germination culture, with each variety's seeds arranged within a 12×12 cm area. Using an indoor high-throughput phenotyping platform equipped with an RGB camera, RGB images of 114 rice varieties throughout their seed germination growth cycle were acquired. For 60 varieties, the density was 16 seeds per image, and for 54 varieties, the density was 25 seeds per image. All images were pre-processed to a resolution of 2880×2880 pixels.

[0058] Instance Segmentation Dataset Construction: A rice seed germination instance segmentation dataset was constructed, containing both virtual and real images. Virtual images were used to train the instance segmentation model, while real images were used to evaluate the model's generalization performance and segmentation accuracy in real-world environments. From this dataset, 360 images covering key germination stages, including 70 varieties (60 with a density of 16 seeds / sheet and 10 with a density of 25 seeds / sheet), were manually labeled. The labeled data was divided into two groups: 250 images (190 with a density of 16 seeds / sheet and 60 with a density of 25 seeds / sheet) were used to construct the virtual dataset, while the remaining 110 images (50 with a density of 16 seeds / sheet and 60 with a density of 25 seeds / sheet) were specifically used for model validation in real-world scenarios. The virtual dataset was constructed as follows: First, individual seed contours were extracted based on labeled masks from 250 real images. These contours were then combined against a blank background to generate seed distribution scenes with different density gradients (36 seeds / image, 25 seeds / image, and 16 seeds / image). To enhance data diversity, random rotation, scaling, and translation operations were applied to each seed. Simultaneously, a physical consistency constraint of 0.3 was set for the overlapping area; when the overlapping area between instances exceeded the threshold, their spatial positions were dynamically adjusted and rearranged. The final result was an instance segmentation dataset containing 7500 virtual images, with 2500 images at each density level (36 seeds / image, 25 seeds / image, and 16 seeds / image). This virtual dataset was divided in an 8:1:1 ratio.

[0059] The dataset consists of a virtual training set (6000 images, 2000 of each of the three densities), a virtual validation set (750 images, 250 of each of the three densities), and a virtual test set (750 images, 250 of each of the three densities). Additionally, 110 real images constitute an independent real test set, which is completely independent of the aforementioned virtual dataset.

[0060] The virtual training set and virtual validation set are used for model training and hyperparameter tuning; the virtual test set is specifically used to evaluate the model's segmentation accuracy on virtual data; and the independent real test set is specifically used to evaluate the model's generalization performance and segmentation effect in real-world scenarios.

[0061] Semantic Segmentation Dataset Construction: A semantic segmentation dataset for rice seeds was constructed to train a semantic segmentation model for segmenting individual organs of rice seeds. Using the trained rice seed instance segmentation network, each seed was segmented from the collected RGB images of seed germination. The maximum bounding box of each seed was extracted, and the bounding box was expanded by 20 pixels. Then, a small image of each seed was cropped according to the coordinates of the expanded bounding box. Blurry samples were removed after quality screening. A total of 2235 rice seed images covering key germination stages such as bud break, white sheath emergence, and coleoptile elongation, with different backgrounds and lighting conditions, were selected. Semantic segmentation annotations were manually performed to construct the semantic segmentation dataset. The dataset was divided into a training set of 1788 images, a validation set of 224 images, and a test set of 223 images in an 8:1:1 ratio.

[0062] (2) Experimental methods.

[0063] To evaluate the accuracy of the method of the present invention, a three-stage evaluation was conducted.

[0064] Instance Segmentation Model Training and Evaluation: To evaluate the performance of the seed instance segmentation model under dense conditions, the model was trained on a virtual instance segmentation dataset using the PyTorch 2.0.1 framework. Model parameters were optimized using a virtual validation set, and the segmentation accuracy on virtual data was quantitatively evaluated using a virtual test set. Subsequently, the trained model was directly applied to real-world scenarios, and its generalization performance and segmentation accuracy in real-world scenarios were evaluated using a real test set. The ADAMW optimizer was used to train the seed instance segmentation model. The training was iterated for 200 epochs on the virtual training set with a batch size of 4. The initial learning rate was 0.001. The Mean Intersection Over Union (mIoU) score was used to evaluate the accuracy of model validation. The best test model was selected based on the highest mIoU achieved on the validation dataset. Experiments were conducted on a Linux server equipped with an NVIDIA A40 GPU (48GB RAM) and a 12-core 2.10 GHz CPU. Model accuracy was evaluated using mIoU, Average Precision (AP), and F1 score, with evaluation metrics ranging from 0 to 100%.

[0065] Semantic Segmentation Model Training and Evaluation: To evaluate the performance of the seed organ segmentation model, a semantic segmentation model was trained on a semantic segmentation dataset using the PyTorch 1.13 framework. Model accuracy was evaluated through validation set optimization and test set quantization. The ADAM optimizer was used to train the seed instance segmentation model. Training lasted 300 iterations with a batch size of 32. The initial learning rate was 0.0005. Experiments were conducted on a Linux server equipped with an NVIDIA V100 GPU (32GB RAM) and a 6-core 2.10 GHz CPU. Model accuracy was evaluated using mIoU, precision (Pre), and recall (Rec), with evaluation metrics ranging from 0 to 100%.

[0066] Tracking Algorithm Evaluation: To evaluate the accuracy of subject tracking, the seed subject mask output from instance segmentation is visualized and mapped to the subject tracking ID number. The same ID is uniquely marked with a color across different frames. The number of tracks that maintain the same ID without any jumps from the initial frame to the final frame is counted as the number of correctly tracked tracks. This number is divided by the total number of tracks to obtain the Trajectory ID Accuracy (TIDA), calculated using the following formula:

[0067] ;

[0068] Where M represents the number of trajectories without ID errors throughout the entire process, and N represents the total number of trajectories. The experiment was run on a Windows computer equipped with an NVIDIA RTX 3060 GPU and a Python 3.9 environment. The evaluation metric ranges from 0 to 100%.

[0069] (3) Experimental results.

[0070] The rice seed instance segmentation network was trained on virtual training sets with three different densities: 36 seeds / sheet, 25 seeds / sheet, and 16 seeds / sheet, while maintaining a consistent experimental environment. The corresponding training weights were obtained for each set. mIoU was used as the core evaluation metric to assess the segmentation performance of these weights on virtual test sets with different densities. The results are shown in Table 1. The mIoU (mean squared error) of weights trained on 36-grain / image virtual datasets was 83.61%, 87.38%, and 89.83% on the 36-grain / image, 25-grain / image, and 16-grain / image virtual test sets, respectively. The mIoU of weights trained on 25-grain / image virtual datasets was 79.31%, 82.24%, and 85.43% on the same datasets. The mIoU of weights trained on 16-grain / image virtual datasets was 54.35%, 70.36%, and 82.80% on the same datasets. Experimental results show that weights trained on 36-grain / image datasets maintain the highest accuracy across all test scenarios, demonstrating that training the network model with a higher-density dataset can improve the model's generalization ability.

[0071] Table 1. Comparison of mIoU metrics for instance segmentation models under different training / test density combinations

[0072]

[0073] The instance segmentation model weights trained on a virtual training set with a density of 36 grains / sheet were used to validate the model on virtual test sets with different densities. The model was evaluated using three metrics: mIoU, AP [0.5:0.95], and F1 score, and the data for each metric were recorded. The model accuracy evaluation results are shown in Table 2, and the segmentation performance at different densities is as follows: Figure 2 As shown, Figure 2 In the image, (a), (b), and (c) are RGB images of virtually generated images containing 36, 25, and 16 seeds, respectively. Figure 2 (d), (e), and (f) in the text are respectively Figure 2 The model segmentation results corresponding to (a), (b), and (c) are shown in the figure.

[0074] Table 2 Evaluation results of the instance segmentation model on the virtual test set

[0075]

[0076] The results showed that under a virtual test set of 36 grains / sheet, the model's mIoU was 83.61%, AP was 74.95%, and F1 score was 95.81%; under a virtual test set of 25 grains / sheet, the model's mIoU was 87.38%, AP was 80.43%, and F1 score was 97.35%; and under a virtual test set of 16 grains / sheet, the model's mIoU was 89.83%, AP was 82.71%, and F1 score was 98.25%. The experimental results indicate that the mIoU index was above 80% at different densities. When the density decreased from 36 grains / sheet to 16 grains / sheet, the mIoU increased by 6.22%, AP increased by 7.76%, and F1 score increased by 2.44%, demonstrating higher test accuracy under low-density conditions.

[0077] The quantitative evaluation results of the segmentation accuracy of the rice seed instance segmentation model trained on the real test set (110 images) based on the 36-seed / sheet density virtual training set are shown in Table 3. The model's mIoU value is 88.41%, AP value is 88.92%, and F1 value is 97.45%.

[0078] Table 3 Evaluation results of the instance segmentation model on the real test set

[0079]

[0080] The semantic segmentation network was trained using the semantic segmentation training set, and the accuracy of the trained model was evaluated on the semantic segmentation test set containing 223 images. The results are shown in Table 4.

[0081] Table 4 Evaluation results of the semantic segmentation model on the test set

[0082]

[0083] Table 4 shows that the mIoU of the root is 81%, Pre is 89%, and Rec is 90%; the mIoU of the bud is 87%, and both Pre and Rec are 93%; the mIoU of the main body is 91%, Pre is 95%, and Rec is 96%; and at the overall level, the mIoU is 89%, and both Pre and Rec are 94%. The experimental results indicate that the semantic segmentation model achieves high-precision segmentation of all organ parts of rice seeds.

[0084] To verify the robustness of the tracking algorithm in dense scenes, germination time-series images of 15 rice varieties with a density of 25 grains / sheet were selected from 44 varieties that were not used in training for testing. Figure 3The demonstration shows the visualization of cross-frame tracking performance by mapping the subject tracking ID to the instance segmentation mask. In the figure, the same seed has the same ID number and mask color in different time frames. The ID continuity of 375 seeds across 221 time-series images was manually checked frame by frame. It was found that 14 seed IDs were incorrect during tracking, resulting in a TIDA (Time-Independent Tracking) accuracy rate of 96.3%. This result proves the effectiveness of this method for tracking individual seeds under dense conditions.

[0085] A cross-frame topological association is established based on the centroid of the seed body mask. By tracking the seed body, the associated root and shoot organs automatically inherit the same tracking ID, thus achieving synchronous tracking of roots and shoots. Figure 4 Showing Figure 3 Tracking performance of seed No. 22 in different frames: The main body mask obtained by semantic segmentation serves as the tracking reference, and the root and shoot regions associated with the main body automatically inherit the same tracking ID. Thus, while completing the tracking of the main body of the seed, the continuous tracking of the root and shoot organs of each seed is achieved simultaneously.

[0086] The experimental results above demonstrate that the three-stage processing architecture proposed in this invention can improve the accuracy of rice seed root and shoot tracking. It effectively solves the problem of not being able to directly track roots and shoots due to their thinness, irregular shape, multi-branched and curved roots, and the intertwining of roots and shoots during germination. In dense scenes, if roots and shoots are segmented before tracking, the intertwining of roots and shoots of different seeds makes it impossible to distinguish individual seeds, resulting in a broken correspondence between roots / shoots and individuals and confusion. Furthermore, the non-linear deformations of roots and shoots, such as bending, branching, and extension, and the dynamic addition and deletion of target structures due to root branching, make existing matching methods unable to adapt to these dynamic changes, easily leading to cross-frame matching errors and making accurate tracking difficult. Therefore, the three-stage processing architecture of this invention is of great significance for achieving efficient monitoring of rice seed root and shoot tracking, rice seed germination morphology, and screening of superior germplasm.

[0087] The rice root and shoot tracking system based on two-stage segmentation and topological feature fusion described in this invention includes:

[0088] The image acquisition unit is used to acquire several frames of images throughout the entire germination stage of rice seeds, with each frame containing several seeds.

[0089] The seed individual segmentation unit is used to perform instance segmentation on the image to obtain the region information of each seed individual;

[0090] The seed individual organ segmentation unit is used to crop the seed individual image based on the regional information of each seed individual, and to perform semantic segmentation on the cropped seed individual image to obtain the root, shoot and main body region information of the seed individual;

[0091] The seed body tracking unit is used to extract the centroid coordinates of the seed body region as spatial nodes, construct an adjacency graph, and establish seed body tracking in several frames of images based on the adjacency graph, thereby completing root tracking and bud tracking in several frames of images.

[0092] The electronic device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the rice root and shoot tracking method based on two-stage segmentation and topological feature fusion.

[0093] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the rice root and shoot tracking method based on two-stage segmentation and topological feature fusion.

[0094] The computer-readable storage medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory or any other medium that can be used to store program code in the form of instructions or data structures and is accessible by a computer.

[0095] The processor is used to execute a computer program stored in memory to implement the various steps in the methods described in the above embodiments.

Claims

1. A rice root and shoot tracking method based on two-stage segmentation and topological feature fusion, characterized in that, Includes the following steps: (1) Collect several frames of images of rice seeds throughout the entire germination stage, with each frame containing several seeds. (2) Perform instance segmentation on the image to obtain the region information of each seed individual; (3) Based on the regional information of each seed individual, crop the seed individual image, and perform semantic segmentation on the cropped seed individual image to obtain the root, shoot and main body region information of the seed individual; (4) Extract the centroid coordinates of the main seed region as spatial nodes, construct an adjacency graph, and establish seed tracking in several frames of images based on the adjacency graph to complete root tracking and bud tracking in several frames of images; The method for tracking the seed subject in several frames of images based on the adjacency graph in step (4) includes the following steps: In the adjacency graph of two consecutive frames, the correlation difference value between the seed main region in the previous frame and the seed main region in the next frame is calculated, and the pair of seed main regions with the smallest correlation difference value is selected by a greedy strategy to establish a correlation relationship. The correlation difference value is calculated by the spatial distance, topological difference, and difference in bud growth direction between the main seed region in the previous frame image and the main seed region in the next frame image. The correlation difference value S = α × D s +β×D t +λ×D d ; In the previous frame image, the node corresponding to the main seed region is P, and in the next frame image, the node corresponding to the main seed region is Q, D. s D is the normalized Euclidean distance between P and Q. t Let D be the absolute difference between the normalized P and Q and the sum of their K nearest neighbors in their respective adjacency graphs. d Let α, β, and λ be the difference in the bud growth direction corresponding to P and Q mapped to the interval [0,1], and let α, β, and λ be coefficients, where α + β + λ = 1.

2. The rice root and shoot tracking method based on two-stage segmentation and topological feature fusion according to claim 1, characterized in that, When the seed corresponding to node Q is in an ungerminated state, the directional difference weight λ is set to zero, and its weight value is proportionally distributed to α and β, that is, the weights are updated as follows: α′=α+λ2, β′=β+λ / 2. At this time, S=α′×D s +β′×D t The ungerminated state is defined as a seed bud length pixel value being less than 50% of the minimum bounding rectangle width pixel value of the seed body.

3. The rice root and shoot tracking method based on two-stage segmentation and topological feature fusion according to claim 1, characterized in that, In step (2), the iShapeInst network is used for instance segmentation, wherein an EMA attention module is embedded in each BasicBlock of the feature extraction backbone network Stage2 to Stage5 in the iShapeInst network.

4. The rice root and shoot tracking method based on two-stage segmentation and topological feature fusion according to claim 1, characterized in that, In step (3), the DeepLabv3+ network is used for semantic segmentation. The backbone network of the DeepLabv3+ network is MobileNetV2, and an adaptive spatial feature pooling module is inserted into the encoder of the DeepLabv3+ network.

5. A rice root and shoot tracking system based on two-stage segmentation and topological feature fusion, characterized in that, include: The image acquisition unit is used to acquire several frames of images throughout the entire germination stage of rice seeds, with each frame containing several seeds. The seed individual segmentation unit is used to perform instance segmentation on the image to obtain the region information of each seed individual; The seed individual organ segmentation unit is used to crop the seed individual image based on the regional information of each seed individual, and to perform semantic segmentation on the cropped seed individual image to obtain the root, shoot and main body region information of the seed individual; The seed body tracking unit is used to extract the centroid coordinates of the seed body region as spatial nodes, construct an adjacency graph, establish seed body tracking in several frames of images based on the adjacency graph, and complete root tracking and bud tracking in several frames of images. The method for tracking the seed subject in several frames of images based on the adjacency graph in the seed subject tracking unit includes the following steps: In the adjacency graph of two consecutive frames, the correlation difference value between the seed main region in the previous frame and the seed main region in the next frame is calculated, and the pair of seed main regions with the smallest correlation difference value is selected by a greedy strategy to establish a correlation relationship. The correlation difference value is calculated by the spatial distance, topological difference, and difference in bud growth direction between the main seed region in the previous frame image and the main seed region in the next frame image. The correlation difference value S = α × D s +β×D t +λ×D d ; In the previous frame image, the node corresponding to the main seed region is P, and in the next frame image, the node corresponding to the main seed region is Q, D. s D is the normalized Euclidean distance between P and Q. t Let D be the absolute difference between the normalized P and Q and the sum of their K nearest neighbors in their respective adjacency graphs. d Let α, β, and λ be the difference in the bud growth direction corresponding to P and Q mapped to the interval [0,1], and let α, β, and λ be coefficients, where α + β + λ = 1.

6. The rice root and shoot tracking system based on two-stage segmentation and topological feature fusion according to claim 5, characterized in that, When the seed corresponding to node Q is in an ungerminated state, the directional difference weight λ is set to zero, and its weight value is proportionally distributed to α and β, that is, the weights are updated α′=α+λ2, β′=β+λ2, and at this time S=α′×D s +β′×D t The ungerminated state is defined as a seed bud length pixel value being less than 50% of the minimum bounding rectangle width pixel value of the seed body.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the rice root and shoot tracking method based on two-stage segmentation and topological feature fusion as described in any one of claims 1-4.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the rice root and shoot tracking method based on two-stage segmentation and topological feature fusion as described in any one of claims 1-4.

Images