A method and system for monitoring and analyzing plant phenotypes with site-specific repeats

Abstract

The application discloses a plant phenotype fixed-point repeated monitoring and analysis method, which realizes early image associated detection frame and number or seed point and number by fixed-point repeated collection of plant images at multiple periods, and realizes plant automatic association under repeated monitoring by using the detection frame or the seed point for plant number matching in subsequent images, so that the automation degree and data consistency are improved; meanwhile, the space-time alignment of the plant and the number or the variety information is ensured in a marker correction mode, and finally, automatic and high-precision extraction and analysis of plant phenotypes are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to plant phenotype monitoring and analysis, in particular, a method and system for repeated monitoring and analysis of plant phenotypes. BACKGROUND

[0002] Dynamic monitoring of crop phenotypes is a key requirement for precision cultivation and intelligent breeding. Plant phenomics provides important technical support for dynamic phenotype monitoring through automated, multi-scale high-throughput phenotype data acquisition. Indoor high-throughput phenotype platforms are key technologies for dynamic phenotype monitoring, capable of accurately regulating various environmental factors and achieving comprehensive monitoring of plants in controlled environments through various high-resolution sensors and image acquisition equipment.

[0003] Existing phenotype analysis methods mainly include traditional methods based on manual measurement and automated methods based on computer vision. Traditional methods include manually measuring plant height using a scale, measuring leaf area using a weighing method or grid method, etc., and the accuracy and consistency of the data are difficult to guarantee, and they cannot be applied to large-scale data acquisition and analysis. Automated methods also have certain defects: (1) insufficient spatiotemporal consistency of phenotype data, existing phenotype platforms mostly use aerial strip acquisition methods, which can achieve wide coverage, but it is difficult to collect data at the same location with high frequency and high precision, resulting in poor spatiotemporal consistency of the data, making it difficult to meet the needs of growth dynamic analysis; (2) throughput and cost limitations, existing phenotype platforms associate plant and number information by scanning a two-dimensional code or recording the location with a hardware system and associating it with the global coordinates, resulting in high equipment costs and complex operations. SUMMARY

[0004] The purpose of the present application is to provide a method and system for repeated monitoring and analysis of plant phenotypes, which can achieve accurate spatiotemporal alignment in multiple image acquisitions by labeling seed points during initial image acquisition and using a marker correction method, solving the problems of spatial deviation and inconsistent data in the multiple acquisition process of existing technologies, and improving the accuracy, stability and consistency of the data.

[0005] Technical solution: The method for repeated monitoring and analysis of plant phenotypes according to the present application comprises the following steps:

[0006] Step 1: acquiring an initial monitoring image of the plants, and then repeatedly acquiring monitoring images of the plants at several time periods; the initial monitoring image and the monitoring images each contain a plurality of plants;

[0007] Step 2: segmenting the plant region in the initial monitoring image to obtain a first detection box; marking the position of the first detection box in the initial monitoring image and displaying its first detection box number, corresponding to inputting the true number of the plant, matching the true number of the plant with its first detection box coordinates;

[0008] In step 3, the monitoring image of each period is matched with the plant number, and the matching method comprises: performing instance segmentation on the monitoring image to obtain a second detection box, calculating the area ratio of the second detection box and the first detection box, and selecting the real number corresponding to the first detection box with the largest area ratio as the real number of the plant in the second detection box;

[0009] In step 4, the plant phenotype is analyzed for the initial monitoring image and the monitoring image.

[0010] Further, in step 1, a fixed-position marker is arranged near the plant, and the initial monitoring image and the monitoring image of the plant both contain the marker.

[0011] In step 2, the marker is detected in the initial monitoring image to obtain a third detection box of the marker.

[0012] In step 3, the monitoring image is corrected for offset, and the offset correction method comprises: detecting the marker in the monitoring image to obtain a fourth detection box of the marker, correcting the second detection box coordinates according to the offset amount between the fourth detection box and the third detection box, and performing plant number matching by using the corrected second detection box coordinates.

[0013] Further, the offset correction method specifically comprises: calculating a homography transformation matrix H t between the fourth detection box and the third detection box t , multiplying the second detection box coordinates by H t to obtain the corrected second detection box coordinates.

[0014] The plant phenotype point-repeated monitoring and analysis system provided by the application comprises:

[0015] A multi-period monitoring unit is configured to acquire an initial monitoring image of the plant and then repeatedly acquire monitoring images of the plant at a plurality of periods; the initial monitoring image and the monitoring images both contain a plurality of plants;

[0016] A detection box generation unit is configured to segment a plant region in the initial monitoring image to obtain a first detection box, mark the position of the first detection box in the initial monitoring image, display a first detection box number thereof, input a real number of the plant, and match the real number of the plant with the first detection box coordinates thereof.

[0017] A plant number matching unit is configured to perform plant number matching on the monitoring image of each period, and the plant number matching method comprises: performing instance segmentation on the monitoring image to obtain a second detection box, calculating the area ratio of the second detection box and the first detection box, and selecting the real number corresponding to the first detection box with the largest area ratio as the real number of the plant in the second detection box.

[0018] a phenotype analysis unit, performing plant phenotype analysis on the initial monitoring image and the monitoring image.

[0019] The method for monitoring and analyzing plant phenotypes according to the present application comprises the following steps:

[0020] Step 1: collecting an initial monitoring image of the plants, and then repeatedly collecting monitoring images of the plants at several time periods; the initial monitoring image and the monitoring images each contain a plurality of plants;

[0021] Step 2: generating seed points of the plants by using the initial monitoring image, segmenting a plant region in the initial monitoring image to obtain a first detection box, calculating a minimum convex polygon of the plant region by using a convex hull algorithm, and taking the vertices of the minimum convex polygon as seed points of the plant;

[0022] marking the positions of the seed points in the initial monitoring image and displaying the first detection box numbers thereof, inputting a true number of the plant, and matching the true number of the plant with the seed point coordinates thereof;

[0023] Step 3: performing plant number matching on each monitoring image, and the method for plant number matching comprises: performing instance segmentation on the monitoring image to obtain a second detection box, judging whether the seed point coordinates are within the second detection box, if the second detection box contains only seed point data of one plant, then the true number corresponding to the seed point data is the true number of the plant within the second detection box; if the second detection box contains seed point data of at least two plants, then calculating the Euclidean distances of each seed point to the center of the second detection box, and selecting the true number corresponding to the seed point with the smallest Euclidean distance as the true number of the plant within the second detection box;

[0024] Step 4: performing plant phenotype analysis on the initial monitoring image and the monitoring image.

[0025] Further, in Step 2, the seed points of the plants further comprise the center points of the first detection boxes.

[0026] Further, Step 1 further comprises setting a fixed marker near the plants, and the initial monitoring image and the monitoring images of the plants each contain the marker;

[0027] Step 2 further comprises detecting the marker in the initial monitoring image to obtain a third detection box of the marker;

[0028] Step 3 further comprises performing offset correction on the monitoring image, and the method for offset correction comprises: detecting the marker in the monitoring image to obtain a fourth detection box of the marker, correcting the seed point coordinates of the plants according to the offset amount between the fourth detection box and the third detection box, and performing plant number matching by using the corrected seed point coordinates.

[0029] Further, the offset correction method specifically comprises: calculating a homography transformation matrix H between the fourth detection frame and the third detection frame t , and multiplying the seed point coordinate vector and H t to obtain a corrected seed point coordinate vector.

[0030] The other plant phenotype monitoring and analysis system provided by the application comprises:

[0031] A multi-period monitoring unit is configured to acquire an initial monitoring image of the plants and then repeatedly acquire monitoring images of the plants at several periods, and the initial monitoring image and the monitoring images each contain a plurality of plants;

[0032] A seed point generation unit is configured to generate seed points of the plants by using the initial monitoring image, segment a plant region in the initial monitoring image to obtain a first detection frame, and calculate a minimum convex polygon of the plant region by using a convex hull algorithm, and the vertices of the minimum convex polygon are taken as the seed points of the plants;

[0033] The positions of the seed points are marked in the initial monitoring image, and the first detection frame numbers of the seed points are displayed, and the real numbers of the plants are input correspondingly, and the real numbers of the plants are matched with the seed point coordinates of the plants;

[0034] A plant number matching unit is configured to perform plant number matching on the monitoring images at each period, and the plant number matching method comprises: performing instance segmentation on the monitoring images to obtain a second detection frame, judging whether the seed point coordinates are in the second detection frame, if the second detection frame contains only seed point data of one plant, the real number corresponding to the seed point data is taken as the real number of the plant in the second detection frame, and if the second detection frame contains seed point data of at least two plants, the Euclidean distances of each seed point to the center of the second detection frame are calculated, and the real number corresponding to the seed point with the smallest Euclidean distance is taken as the real number of the plant in the second detection frame;

[0035] A phenotype analysis unit is configured to perform plant phenotype analysis on the initial monitoring image and the monitoring images.

[0036] The computer readable storage medium provided by the application stores a computer program, and the computer program is executed by a processor to implement the plant phenotype monitoring and analysis method.

[0037] Advantages: Compared with the prior art, the advantages of the present application are: (1) The present application significantly reduces the computational load through simple and efficient detection frame correction and matching algorithm design. At the same time, it can effectively cope with challenges such as light changes and plant shape changes, ensuring stability and reliability in different scenes. This efficient and robust design makes the present application widely applicable in plant phenotype monitoring, providing strong technical support for large-scale, multi-scene plant phenomics research; (2) The present application collects phenotype data at different growth stages of plants, extracts morphological features, and updates based on seed point labeling, achieving automatic and high-precision extraction and analysis of plant phenotypes. It provides data support for plant growth monitoring, environmental adaptability analysis, and intelligent breeding, and promotes the cultivation of high-yield, stress-resistant and high-quality crops, which is of great significance to improve crop production efficiency and stability and ensure food security. At the same time, the automatic method provided by the present application greatly reduces labor costs and improves the efficiency of plant phenomics research, providing important technical support for large-scale germplasm identification and screening; (3) The present application ensures the spatiotemporal alignment of plant and number or variety information through marker correction. Compared with manual labeling and simple position matching in the prior art, it realizes automatic multiple image acquisition offset correction. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The plant phenotype point repeat monitoring and analysis method flowchart of embodiment 1 of the present application.

[0039] Figure 2 The plant phenotype point repeat monitoring and analysis method flowchart of embodiment 2 of the present application. DETAILED DESCRIPTION

[0040] The technical solutions of the present application will be further described below with reference to the accompanying drawings.

[0041] Embodiment 1

[0042] As shown in Figure 1 , the plant phenotype point repeat monitoring and analysis method comprises the following steps.

[0043] Step 1, take an initial monitoring image of the plant, and then repeatedly take monitoring images at several periods during the growth of the plant.

[0044] Further, since the image position may be offset due to the offset of the image acquisition device when collecting images at multiple periods, a marker can be set at the initial position or the fixed place at the end of collection.

[0045] Step 2, initial detection frame generation.

[0046] In the initial monitoring image, single plant segmentation is performed using an instance segmentation algorithm (Mask R-CNN is used in this embodiment), the target plant region is separated from the background, and M plant regions are obtained. For the i-th segmented plant, first extract the four vertices of the corresponding initial detection box and the mask Mask i wherein i ∈ [1, M].

[0047] Further, for the case where the number of plants is large and cannot be covered by one image, since the present application adopts a fixed-point collection method, the collection position and sequence are fixed, therefore, according to the order of fixed-point collection, each collection position can be numbered in the order of 1, 2, 3, …; if one image can contain all plants, then the collection position numbering is not needed. These data are stored in a structured manner in a JSON file, with the following format:

[0048] {"plants":[{"temp_id":"i","seeds":[{"x":x_min,"y":y_min},{"x":x_max,"y":y_min},{"x":x_max,"y":y_max},{"x":x_min,"y":y_max}],"position":Position_1},...]};

[0049] wherein temp_id is the temporary number of the plant, its value is the sequential number of the corresponding instance detection box, the data type is string (string); x and y are the coordinates of the corner points of the initial detection box, the data type is integer (int), and position is the corresponding position number, the data type is integer (int).

[0050] Further, after obtaining the initial data of the detection box, in order to realize the association with the real variety / number of the plant in the experimental design and exclude plants that do not need to be analyzed (including protective rows and plants with severe occlusion, etc.), further information needs to be determined through human interaction. Specifically, first, based on the initial detection box information, the position of the detection box is marked by a rectangular box in the initial monitoring image, and the corresponding temporary number temp_id is displayed superimposed near the detection box. Then, traverse each detection box, manually input the corresponding real number real_id (variety number or experimental number) through command line prompt, and update the corresponding JSON data with the value of the field real_id as a string (string) format. For plants that do not need to participate in subsequent analysis, when inputting the real number, input exclude, then the corresponding data in the JSON file will be deleted and will not participate in subsequent phenotype feature extraction and analysis.

[0051] Furthermore, corresponding to the case where markers were set in step 1, the markers in the image are detected using the YOLOv8 object detection algorithm (other object detection algorithms can also be used), and their four corner points are obtained. The two-dimensional coordinates are obtained and stored in a JSON file for subsequent spatial offset correction. To facilitate correction, a marker is placed at the first shooting position. In this embodiment, a rectangular object is used, but it can be replaced with other materials or other markers, but its position must be kept fixed.

[0052] Step 3, corresponding to the case where markers were set in Step 1, performs spatial offset correction through marker detection. If no markers were set, proceed directly to Step 4.

[0053] For the monitoring image captured at period t, a target detection algorithm is used to detect the markers and obtain their vertex coordinates. By calculating the change in the marker's position relative to its initial position in the initial monitoring image, the spatial offset of the detection box corners is corrected. Specifically, the homography matrix estimation function findHomography in OpenCV is first used to calculate the homography matrix H. t :

[0054]

[0055] Applying the homography matrix calculated above to update the corner points of the initial detection box, and after offset correction, the new position of the corner points of the initial detection box for a single plant in an image taken at a certain location can be calculated using the following formula:

[0056]

[0057] The new positions of the initial detection frame corners collected in this period can be described as follows:

[0058]

[0059] in, and Let d represent the corner position of the detection box for the plant in the initial monitoring image, where d ∈ [1, 4]. Thus, by calculating the homography transformation matrix of the marker and applying it to update the corner position of the initial detection box, the matching accuracy between the plant and its identification number or variety information can be improved.

[0060] Step 4: Match plant IDs based on the intersection of detection boxes.

[0061] Step 4-1: Obtain the coordinates of the four vertices of the detection box through instance segmentation:

[0062] (xmin , y min , x max , y min , x max , y max , x min , y max

[0063] Then according to the collection position number position, retrieve the plant number or variety real_id under this position from the JSON file storing the seed point position information.

[0064] Step 4-2, calculate the coordinates of the four vertices of the intersection rectangle:

[0065]

[0066] Then calculate the intersection area S inter :

[0067] S inter = max(0, x inter_max -x initer_min )·max(0, y inter_max -y inter_min )

[0068] Then calculate the area of the detection frame and the initial detection frame respectively:

[0069] S i =(x max -x min )·(y max -y min )

[0070]

[0071] Finally, calculate the area ratio

[0072]

[0073] Step 4-3, calculate the area ratio of the detection frame of the instance to each initial detection frame, select the plant corresponding to the initial detection frame with the largest area ratio, take the plant number or variety real_id as the number of the current instance, and add the number or variety real_id and the coordinates of the four vertices of the detection frame to the set S match ; At the same time, add the four vertices of the instance segmentation detection frame in the initial monitoring image and the corresponding plant number or variety information to the set S match .

[0074] Step 4-5, then traverse all instances, if all ​Then it is considered that the instance does not match the appropriate initial detection box, and is not added to the set S match . For the instance that fails to match the initial detection box, it will not participate in the subsequent phenotype extraction.

[0075] Step 5, dynamic phenotype analysis.

[0076] The embodiment mainly extracts phenotypes in the crown area of the plant. The previous instance segmentation is to divide the whole plant, but the segmentation result of the whole plant may not meet the specific research needs. Therefore, in order to further improve the accuracy and pertinence of phenotype analysis, the plant area needs to be further divided into different organs such as stems and leaves, and the crown area needs to be extracted. The specific segmentation category can be refined according to the research content needs.

[0077] When labeling the seed points, in order to better cover the whole plant area, the labeling of the seed points is based on the integrity of the plant geometric structure. However, as the plant grows, its morphological structure will change, and the relative positions of each organ (such as stems and leaves) will also change. Therefore, in the case of not adding seed points and needing to know the corresponding number or variety information when extracting phenotypes in the leaf area, the scheme of first instance segmentation and number matching, then combining semantic segmentation refinement and screening the target area can significantly improve the accuracy and efficiency of subsequent phenotype feature extraction.

[0078] Specifically, the U-Net model is used for semantic segmentation of the plant, which is divided into different parts such as stems and leaves. The single instance in S match is extracted for phenotypes (the specific phenotype index can be calculated according to the specific scientific research needs, and the embodiment only takes the leaf area calculation as an example), and the specific steps are as follows:

[0079] 1) Leaf area mask creation: first, get the four corner coordinates of the instance / plant detection box stored in S match :

[0080] (x min ,y min ),(x max ,y min ),(x max ,y max ),(x min ,y max )

[0081] Then, based on the semantic segmentation result, create a leaf mask Mask leaf for subsequent analysis. Mask leafis a two-dimensional array with the same size as the original image, where each pixel point belonging to the leaf part is marked as 1, and the rest is 0. Then, based on the instance detection box coordinates, the Mask leaf is further filtered to ensure that only the leaf part within the instance is retained, and the filtering process is as follows:

[0082]

[0083] where Mask instance-leaf represents the leaf area mask corresponding to the instance / plant. The pixel points belonging to the leaf area of the instance are retained as 1, and the rest are set to 0. In this way, only the leaf area of the instance is extracted for phenotype feature extraction, avoiding interference from other plant regions, improving the accuracy and consistency of the extraction.

[0084] 2) Leaf area calculation: In Mask instance-leaf , count the number of pixel points with a value of 1, and this value is the leaf area, with units of pixels. Leaf area A leaf is calculated as follows:

[0085]

[0086] 3) Data storage: Store the plant / instance number real_id and phenotype data such as leaf area A leaf in a CSV file for subsequent data analysis and utilization.

[0087] 4) Traverse all instances in the set S match to complete the phenotype extraction operation of all plants in the current image.

[0088] In order to realize large-scale phenotype extraction, the above operation process is repeated for all images collected at different locations. Each image obtained at a collection location is subjected to instance segmentation, number matching, semantic segmentation, and phenotype extraction, and finally a comprehensive phenotype dataset covering all collection locations is formed. This efficient phenotype extraction process not only significantly improves the efficiency of data collection and analysis, but also ensures the consistency and reliability of the data, providing strong support for in-depth research on plant phenotypes.

[0089] Example 2

[0090] As shown in Figure 1 , the plant phenotype point repeated monitoring and analysis method comprises the following steps.

[0091] Step 1, take an initial monitoring image of the plant, and then repeatedly take monitoring images at several periods during the growth of the plant.

[0092] Further, since the image position may be offset due to the offset of the image acquisition device when acquiring images at multiple time periods, a marker can be arranged near the plant to ensure that the marker is included in each image.

[0093] Step 2, seed point generation.

[0094] In the initial monitoring image, single plant segmentation is performed using an instance segmentation algorithm (Mask R-CNN is used in this embodiment), the target plant region is separated from the background, and M plant regions are obtained. For the i-th segmented plant, the four vertices of the corresponding instance segmentation detection box are first extracted and the mask Mask i , and the pixel point set of the plant boundary is obtained, where i∈[1,M]. Then, based on the boundary points, the minimum convex polygon of the plant region is calculated using a convex hull algorithm (the cv2.convexHull function in OpenCV is used in this embodiment), and the vertices of the convex hull are obtained, where N is the number of convex hull vertices, which is part of the seed point corresponding to the plant. In addition, to ensure comprehensive coverage of the plant geometric information and enhance the robustness of the seed point in accurately describing the plant region, the geometric center point is calculated as a supplement to the seed point, and the coordinate calculation formula is as follows:

[0095]

[0096] where (x min ,y min ) and (x max ,y max ) are the coordinates of the top left corner and the bottom right corner of the instance segmentation detection box.

[0097] For the i-th plant, the seed point is composed of the convex hull vertices and the geometric center point, denoted as N is the number of convex hull vertices.

[0098] Further, for the case where the number of plants is large and all plants cannot be covered by one image, since the present application adopts a fixed-point acquisition method, the acquisition position and sequence are fixed, therefore, according to the sequence of fixed-point acquisition, each acquisition position can be numbered in the order of 1, 2, 3, …; if one image can contain all plants, the acquisition position numbering is not required. These data are stored in a JSON file in a structured manner, and the format is as follows:

[0099] {"plants":[{"temp_id":"i","seeds":[{"x":x_c,"y":y_c},{"x":x_1,"y":y_1},{"x":x_2,"y":y_2},…,{"x":x_N,"y":y_N}],"position":Position_1},...]};

[0100] Wherein, temp_id is the temporary number of the plant, which is the sequence number of the instance detection frame corresponding to it, and the data type is string; x and y are the coordinates of the seed point, and the data type is integer (int); position is the corresponding position number, and the data type is integer (int).

[0101] Further, after obtaining the initial data of the seed points, in order to realize the association with the real variety / number of the plant in the experimental design and exclude plants that do not need to be analyzed (including protective rows and plants with serious shading, etc.), further information needs to be determined through manual interaction. Specifically, first, based on the initial seed point information, the positions of the seed points are marked by circular markers in the initial monitoring image, and the corresponding temporary number temp_id is displayed on the seed point. Then, traverse each seed point, manually input the corresponding real number real_id (variety number or experiment number) through command line prompt, and update the corresponding JSON data as the value of the field real_id in the format of string (string). For plants that do not need to participate in subsequent analysis, when inputting the real number, input exclude, then the corresponding data in the JSON file will be deleted and will not participate in subsequent phenotype feature extraction and analysis.

[0102] Further, for the case where a marker is set in step 1, use the target detection algorithm YOLOv8 (other target detection algorithms can also be used) to detect the marker in the image, obtain the two-dimensional coordinates of its four corners , and store them in a JSON file for subsequent seed point updating. In order to facilitate correction, a marker is placed at the first shooting position, which is a rectangular object in this embodiment, and other materials or other markers can also be used, but the position needs to be fixed.

[0103] Step 3, for the case where a marker is set in step 1, the spatial offset correction is performed through marker detection. If no marker is set, directly execute step 4.

[0104] For the monitoring image taken at the tth period, use the target detection algorithm to detect the marker, and get its vertex coordinates The correction of the spatial shift of the seed point is achieved by calculating the change of the marker relative to the initial position in the initial monitoring image. Specifically, a homography matrix H is first calculated using the findHomography function in OpenCV t :

[0105]

[0106] The homography matrix calculated above is applied to the seed point update. After the correction by the shift, for a single plant in the image taken at a certain position, the new position of the seed point can be calculated by the following formula:

[0107]

[0108] Then the new position of the seed point collected at this time can be expressed as:

[0109]

[0110] wherein, and are the seed point positions of the plant in the initial monitoring image, j ∈ [1, N+1], and N is the number of convex hull vertices in the initial image of the plant. In this way, by calculating the homographic transformation matrix of the marker and applying it to the seed point position update, the matching accuracy of the plant and its number or variety information can be improved.

[0111] Step 4: Plant number matching based on seed points.

[0112] For the monitoring image, the coordinates of the four vertices of the detection box are obtained through instance segmentation:

[0113] (x min ,y min ),(x max ,y min ),(x max ,y max ),(x min ,y max )

[0114] Then, according to the collection position number position, all the seed point information at this position is retrieved from the JSON file storing the seed point position information, including the plant number or variety real_id and its corresponding seed point coordinates.

[0115] Each set of seed point coordinates is traversed, and the preliminary association of the seed point and the instance is realized by judging whether the seed point is within the detection box of the instance. First, define the set S krepresents all seed points that the kth instance can contain, wherein the position information and the number or variety information of each seed point are stored. The judgment criteria are as follows:

[0116]

[0117] wherein (x i,j ,y i,j ) is the position of the jth seed point of the ith plant. For the seed points meeting the condition, the coordinates (x i,j ,y i,j ) and the number or variety information id i of the seed points are added to the set S k , indicating that the seed points belong to the kth instance.

[0118] For the seed points in the set S k , it is necessary to further determine the correspondence between the instance and the plant. If the detection box contains only one set of seed points (i.e., only one id i ), the number or variety of the plant corresponding to the instance is the value of the id i ; if the detection box contains multiple sets of seed points (i.e., multiple id i ), it is necessary to further determine according to the distance between the seed points and the instance. In this embodiment, the Euclidean distance d i,j between each seed point and the center (x center ,y center ) of the instance is calculated, and the position of the instance center can be calculated by the geometric center of the detection box:

[0119]

[0120] Subsequently, the Euclidean distance between each seed point in the set S k and the center of the instance is calculated:

[0121]

[0122] The seed point with the smallest d i,j is selected, and the value of the id i of the seed point is taken as the number or variety information of the plant corresponding to the instance. All instances successfully matched to the seed points are added to the set S match , and the four vertex coordinates of the instance segmentation detection box in the initial monitoring image and the corresponding plant number or variety information are also added to the set S match . In this way, each instance can be uniquely associated with its number or variety, facilitating subsequent data management and analysis. Meanwhile, the instances that fail to match the seed points will not participate in the subsequent phenotype extraction.

[0123] Step 5, dynamic phenotyping analysis.

[0124] After completing the number matching and screening of instances, the next step is to extract the phenotypic characteristics of the instances in the set S match The dynamic phenotyping analysis of this embodiment is the same as Step 5 of Embodiment 1, which will not be repeated here.

[0125] Embodiment 3

[0126] This embodiment takes soybean salt-tolerant high-throughput phenotype extraction as an example to illustrate the implementation process of the method described in Embodiment 1.

[0127] (1) Experimental design

[0128] The experiment was conducted in a greenhouse to evaluate the salt tolerance of soybean varieties. A total of 261 soybean varieties were selected for the experiment, which lasted from November 2023 to February 2024. The experimental conditions were controlled at a greenhouse temperature of 26±2℃ and a light period of 14 hours. Soybeans were cultivated in plastic containers filled with quartz sand and supplied with nutrient solution. After the soybeans grew to the V2 stage (two-leaf stage), salt stress treatment was started, using NaCL solution to adjust the electrical conductivity to simulate a salt stress environment.

[0129] (2) Data collection

[0130] To accurately evaluate the phenotypic response of soybean varieties under salt stress conditions, this embodiment uses a depth camera (Intel RealSense D435) and an RGB camera (OAK-1-MAX) to collect morphological information of plants to comprehensively characterize the growth changes of plants. Data collection was performed once a day at 2 pm after the start of the experiment, with 28 images taken each time, at a fixed frequency for 13 days to ensure the time consistency and high quality of the data. The collected data is automatically stored in a computer and regularly transmitted to a dedicated storage server for backup to ensure reliable data storage.

[0131] (3) Initial bounding box generation

[0132] To ensure that subsequent analysis can accurately locate the position of the plant and obtain relevant phenotypic information, this embodiment generates an initial bounding box for the soybean plants in the initially collected images, i.e., the initial bounding box is generated in the images collected on the first day. The main purpose of the initial bounding box generation is to determine the position of each soybean in the image and establish the association between the number / variety and each plant in the image, thereby realizing the accurate correspondence between the data and the specific plant and laying the foundation for subsequent phenotypic feature extraction and analysis.

[0133] Specifically, initial detection box generation is performed in the initially collected images. As previously described, initial detection box generation and labeling are performed for each soybean plant. All initial detection box labeling information, including the coordinates of the initial detection box corner points, the number / variety, etc., is stored in a structured manner as a JSON file. Not only is it convenient to read and modify the initial detection box generation, but it also provides a convenient access method for subsequent data analysis and processing.

[0134] At the same time, initial marker detection box generation is performed in the initially collected images. As previously described, the markers in the images are detected, their corner point coordinates are obtained, and they are stored in a JSON file for subsequent spatial offset correction.

[0135] (4) Spatial offset correction

[0136] Since the images collected at different times may be offset due to the offset of the image collection device, etc. Therefore, before the plant number matching of the present example, the initial detection box is corrected for spatial offset to improve the efficiency and accuracy of subsequent data processing and phenotype extraction.

[0137] (5) Plant number matching based on the intersection of detection boxes

[0138] To ensure accurate tracking of each soybean plant in multiple collections and improve the spatiotemporal consistency of the data. The present application uses the intersection of detection box areas to match plant numbers based on images collected at different time points to track the growth changes of the same plant. Ensures that when detecting the growth status of soybean plants under salt stress conditions, the data at different time points can be accurately corresponded to each soybean, thereby providing a reliable data basis for subsequent salt tolerance analysis. The four vertex coordinates of the example successfully matched to the initial detection box and its corresponding plant number are added to the set S match .

[0139] (6) Dynamic phenotype analysis

[0140] To evaluate the performance of different soybean varieties under salt stress conditions, the present application analyzes the soybean canopy area. First, the U-Net model is used to perform semantic segmentation on the soybean plants, dividing them into leaf, cotyledon, and background. Then, various phenotype indicators such as morphology are extracted from the leaf area of each soybean plant, providing a scientific basis for further physiological and breeding analysis.

[0141] (7) Data storage

[0142] To ensure the continuous use and subsequent analysis of data, the present application will store the phenotypic characteristic data obtained from each collection and analysis. The data to be saved mainly include the number (real_id) of each soybean, leaf area (A leaf ), which will be stored in CSV format. Finally, all instances in the set S match are traversed, and the phenotypic extraction operation of all soybean plants in the current image is completed.

[0143] The above operation process is repeated for all 28 images in this collection, forming phenotypic data covering all collection locations. To further explore the dynamic phenotypic characteristics of plants at different growth stages, the above process is repeated for image data collected at multiple time periods to construct a time series dataset for salt tolerance analysis and comparison analysis between varieties.

[0144] Example 4

[0145] This example illustrates the implementation process of the method described in Example 2 using poplar seedling growth monitoring as an example.

[0146] (1) Experimental design

[0147] The experiment was conducted in a greenhouse to monitor the growth status of poplar seedlings. A total of 40 poplar seedlings were selected for the experiment, with the time period from November 2024 to December 2024. The experimental conditions were as follows: the greenhouse temperature was controlled at 26±2℃, and the light period was 13 hours. The poplars were cultured in plastic containers filled with nutrient soil.

[0148] (2) Data collection

[0149] To accurately monitor the growth status of poplar seedlings, this example uses a depth camera (Intel RealSense D435) and an RGB camera (OAK-1-MAX) to collect plant morphological information to fully characterize the growth changes of the plants. Data collection is performed once a day at 9 am after the experiment begins, with one image taken at each of the 40 collection locations each time, with a fixed collection frequency to ensure the time consistency and high quality of the data. The collected data is automatically stored in the computer and regularly transmitted to a dedicated storage server for backup to ensure reliable data storage.

[0150] (3) Seed point generation

[0151] To ensure that subsequent analysis can accurately locate the position of the plant and obtain the relevant phenotype information, the embodiment generates seed points for 40 poplar trees in the initially collected image, that is, generates seed points in the image collected on the first day. The main purpose of seed point generation is to determine the position of each poplar tree in the image and establish the association between the number / variety and each plant in the image, thereby realizing accurate correspondence of data and specific plants and laying a foundation for subsequent phenotype feature extraction and analysis.

[0152] Specifically, seed point generation is performed in the initially collected image. As described previously, seed point labeling is performed for each poplar tree. All seed point labeling information, including the coordinates of seed points, numbers / varieties, etc., is stored in a structured manner as a JSON file. Not only is it convenient to read and modify the seed points, but it also provides a convenient access method for subsequent data analysis and processing.

[0153] (4) Plant number matching

[0154] To ensure that each poplar tree can be accurately tracked in multiple collections and improve the spatiotemporal consistency of data, the present application matches the plant numbers based on images collected at different time points to track the growth changes of the same plant (the error of the collected images in the embodiment is small, so no spatial correction is performed). When detecting the growth state of poplar seedlings, ensure that the data at different time points can be accurately corresponded to each poplar tree, thereby providing a reliable data basis for subsequent growth dynamic analysis. The four vertex coordinates of the instance successfully matched to the seed point and the corresponding plant number are added to the set S match .

[0155] (5) Dynamic phenotype analysis

[0156] To monitor the growth performance of poplar, the embodiment analyzes the poplar canopy area. First, the U-Net model is used for semantic segmentation of poplar, which is divided into leaf, stem and background. Then, various phenotype indicators such as morphology are extracted from the leaf area of each poplar, thereby providing a scientific basis for further physiological and breeding analysis.

[0157] (6) Data storage

[0158] To ensure the continuous use of data and subsequent analysis, the present application stores the phenotype feature data obtained by each collection and analysis. The data to be saved mainly include the number (real_id) of each poplar tree, the leaf area (A leaf ), which will be stored in CSV format. Finally, all instances in the set S match are traversed to complete the phenotype extraction operation of all poplar plants in the current image.

[0159] The above procedure was repeated for all 40 images in this acquisition, forming phenotyping data covering all acquisition positions. To further mine the dynamic phenotyping characteristics of plants at different growth stages, the above procedure was repeated for the image data acquired at multiple stages, constructing a time series dataset for growth monitoring.

Claims

1. A method for monitoring and analyzing plant phenotypes at a specific location, characterized in that, The method comprises the following steps: Step 1, acquiring an initial monitoring image of plants, and then repeatedly acquiring monitoring images of plants at several periods; the initial monitoring image and the monitoring images each contain a plurality of plants; Step 2, segmenting a plant region in the initial monitoring image to obtain a first detection box; marking the position of the first detection box in the initial monitoring image and displaying a first detection box number thereof, corresponding to inputting a true number of the plant, and matching the true number of the plant with the first detection box coordinates thereof; Step 3, performing plant number matching for each monitoring image at a period, and the plant number matching method comprises: performing instance segmentation on the monitoring image to obtain a second detection box, calculating an area ratio of the second detection box to the first detection box, and selecting a true number corresponding to a first detection box with the largest area ratio as a true number of a plant in the second detection box; Step 4, performing plant phenotype analysis on the initial monitoring image and the monitoring images; In step 1, a fixed-position marker is further arranged near the plants, and the initial monitoring image and the monitoring images of the plants each contain the marker; In step 2, the marker is further detected in the initial monitoring image to obtain a third detection box of the marker; In step 3, the monitoring image is further corrected for offset, and the offset correction method comprises: detecting the marker in the monitoring image to obtain a fourth detection box of the marker, correcting the second detection box coordinates according to an offset amount between the fourth detection box and the third detection box, and performing plant number matching by using the corrected second detection box coordinates.

2. The method for fixed-site repeated monitoring and analysis of plant phenotypes according to claim 1, characterized in that, The offset correction method specifically comprises: A homography transformation matrix between the fourth detection frame and the third detection frame is calculated The second detection frame coordinates are multiplied by to obtain the corrected second detection frame coordinates.

3. A system for monitoring and analyzing site-specific repeats of plant phenotypes, characterized by, The method comprises: a multi-period monitoring unit configured to acquire an initial monitoring image of plants, and then repeatedly acquire monitoring images of plants at several periods; the initial monitoring image and the monitoring images each contain a plurality of plants; a detection box generation unit configured to segment a plant region in the initial monitoring image to obtain a first detection box; mark the position of the first detection box in the initial monitoring image and display a first detection box number thereof, corresponding to inputting a true number of the plant, and matching the true number of the plant with the first detection box coordinates thereof; a plant number matching unit configured to perform plant number matching for each monitoring image at a period, and the plant number matching method comprises: performing instance segmentation on the monitoring image to obtain a second detection box, calculating an area ratio of the second detection box to the first detection box, and selecting a true number corresponding to a first detection box with the largest area ratio as a true number of a plant in the second detection box; a phenotype analysis unit configured to perform plant phenotype analysis on the initial monitoring image and the monitoring images; The multi-period monitoring unit further comprises: a fixed-position marker arranged near the plants, and the initial monitoring image and the monitoring images of the plants each contain the marker; The detection box generation unit further comprises: detecting the marker in the initial monitoring image to obtain a third detection box of the marker; The plant number matching unit further comprises offset correction on the monitoring image, and the offset correction method comprises: detecting a marker in the monitoring image to obtain a fourth detection box of the marker, correcting the second detection box coordinates according to the offset amount between the fourth detection box and the third detection box; and performing plant number matching by using the corrected second detection box coordinates.

4. A method for monitoring and analyzing a plant phenotype by targeted repeats, characterized by, The method comprises the following steps: Step 1: collecting an initial monitoring image of plants, and then repeatedly collecting monitoring images of plants at several periods; the initial monitoring image and the monitoring images each contain a plurality of plants; Step 2: generating seed points of the plants by using the initial monitoring image, segmenting a plant region in the initial monitoring image to obtain a first detection box, and calculating a minimum convex polygon of the plant region by using a convex hull algorithm, wherein the vertices of the minimum convex polygon are taken as the seed points of the plants; marking the positions of the seed points in the initial monitoring image and displaying the first detection box numbers thereof, inputting the real numbers of the plants corresponding to the seed points, and matching the real numbers of the plants with the seed point coordinates thereof; Step 3: performing plant number matching on each monitoring image at a period, and the plant number matching method comprises: performing instance segmentation on the monitoring image to obtain a second detection box, judging whether the seed point coordinates are in the second detection box, if the second detection box contains only seed point data of one plant, the real number corresponding to the seed point data is taken as the real number of the plant in the second detection box, and if the second detection box contains seed point data of at least two plants, calculating the Euclidean distances of each seed point to the center of the second detection box, and selecting the real number corresponding to the seed point with the minimum Euclidean distance as the real number of the plant in the second detection box; Step 4: performing plant phenotype analysis on the initial monitoring image and the monitoring images; Step 1 further comprises arranging a marker fixed in position near the plants, and the initial monitoring image and the monitoring images of the plants each contain the marker; Step 2 further comprises detecting the marker in the initial monitoring image to obtain a third detection box of the marker; Step 3 further comprises offset correction on the monitoring image, and the offset correction method comprises: detecting a marker in the monitoring image to obtain a fourth detection box of the marker, correcting the plant seed point coordinates according to the offset amount between the fourth detection box and the third detection box; and performing plant number matching by using the corrected seed point coordinates.

5. The method for fixed-site repeated monitoring and analysis of plant phenotypes according to claim 4, characterized in that, In step 2, the seed points of the plants further comprise the center points of the first detection boxes.

6. The method for fixed-site repeated monitoring and analysis of plant phenotypes according to claim 4, characterized in that, The offset correction method specifically comprises: Compute homography transformation matrix between fourth detection frame and third detection frame , the seed point coordinate vector is multiplied by to obtain the modified seed point coordinate vector.

7. A system for monitoring and analyzing site-specific repeats of plant phenotypes, comprising: comprises: a multi-period monitoring unit for collecting an initial monitoring image of plants, and then repeatedly collecting monitoring images of plants at several periods; the initial monitoring image and the monitoring images each contain a plurality of plants; a seed point generation unit for generating seed points of the plants by using the initial monitoring image, segmenting a plant region in the initial monitoring image to obtain a first detection box, and calculating a minimum convex polygon of the plant region by using a convex hull algorithm, wherein the vertices of the minimum convex polygon are taken as the seed points of the plants; marking the positions of the seed points in the initial monitoring image and displaying the first detection box numbers thereof, inputting the real numbers of the plants corresponding to the seed points, and matching the real numbers of the plants with the seed point coordinates thereof; The plant number matching unit is configured to perform plant number matching on each period of monitoring images, and the plant number matching method comprises: performing instance segmentation on the monitoring images to obtain a second detection box; judging whether seed point coordinates are in the second detection box; if the second detection box contains only seed point data of one plant, the real number corresponding to the seed point data is the real number of the plant in the second detection box; if the second detection box contains seed point data of at least two plants, calculating the Euclidean distance of each seed point to the center of the second detection box, and selecting the real number corresponding to the seed point with the smallest Euclidean distance as the real number of the plant in the second detection box; The phenotype analysis unit is configured to perform plant phenotype analysis on the initial monitoring image and the monitoring image. The multi-period monitoring unit further comprises a fixed marker arranged near the plant, and the initial monitoring image and the monitoring image of the plant both contain the marker. The seed point generation unit further comprises detecting the marker in the initial monitoring image to obtain a third detection box of the marker. The plant number matching unit further comprises offset correction on the monitoring image, and the offset correction method comprises: detecting the marker in the monitoring image to obtain a fourth detection box of the marker, correcting the seed point coordinates of the plant according to the offset amount between the fourth detection box and the third detection box; and performing plant number matching by using the corrected seed point coordinates.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to realize the plant phenotype point-to-point repeated monitoring and analysis method according to any one of claims 1-2 and 4-6.

Images