A method, device, equipment and storage medium for measuring phenotype of wheat plant

Abstract

Embodiments of the present application disclose a wheat plant phenotype measurement method, device, equipment and storage medium, the method comprising: acquiring a first wheat image corresponding to a wheat area to be measured; inputting the first wheat image into a target mask generation model for semantic segmentation to obtain a first mask image corresponding to the wheat area; performing extraction operation on the first mask image to obtain a second mask image corresponding to each target wheat in the wheat area; determining a target phenotype image corresponding to each target wheat based on the first wheat image and the second mask image; the target phenotype image includes at least one phenotype of the target wheat; determining a target phenotype length corresponding to each phenotype of each target wheat based on the target phenotype image and a length marker in the first wheat image, thereby realizing automatic measurement of the length of each phenotype of the wheat, and multiple wheat plants can be measured at the same time, improving measurement efficiency and accuracy.

Description

Technical Field

[0001] The embodiments of the present invention relate to image processing technology, and more particularly to a method, apparatus, device and storage medium for measuring wheat plant phenotype. Background Technology

[0002] Wheat, a member of the Poaceae family, is an important food crop and is widely cultivated. Wheat yield and quality directly impact people's quality of life. Currently, wheat is typically studied and bred based on its phenotype. For example, over a period of time, it is necessary to measure and record the lengths of important phenotypes such as plant height, ear length, stem length, and stem width. This is usually done manually. However, this manual measurement method is inefficient, time-consuming, and labor-intensive, and cannot guarantee accuracy and efficiency when dealing with large-scale measurements. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and storage medium for measuring wheat plant phenotypes, enabling automatic measurement of the length of various wheat phenotypes and allowing simultaneous measurement of multiple wheat plants, thereby improving measurement efficiency and accuracy.

[0004] In a first aspect, embodiments of the present invention provide a method for measuring the phenotype of wheat plants, comprising:

[0005] Obtain the first wheat image corresponding to the wheat region to be measured;

[0006] The first wheat image is input into the target mask generation model for semantic segmentation to obtain the first mask image corresponding to the wheat region;

[0007] The first mask image is extracted to obtain a second mask image corresponding to each target wheat in the wheat region;

[0008] Based on the first wheat image and the second mask image, a target phenotypic image corresponding to each target wheat is determined; the target phenotypic image includes at least one phenotype of the target wheat;

[0009] Based on the length markers in the target phenotypic image and the first wheat image, the target phenotypic length corresponding to each phenotypic of each target wheat is determined.

[0010] Secondly, embodiments of the present invention also provide a device for measuring the phenotype of wheat plants, comprising:

[0011] The first wheat image acquisition module is used to acquire the first wheat image corresponding to the wheat region to be measured;

[0012] The first mask image acquisition module is used to input the first wheat image into the target mask generation model for semantic segmentation to obtain the first mask image corresponding to the wheat region;

[0013] The second mask image acquisition module is used to extract the first mask image to obtain the second mask image corresponding to each target wheat in the wheat region;

[0014] A target phenotypic image determination module is used to determine a target phenotypic image corresponding to each target wheat based on the first wheat image and the second mask image; the target phenotypic image includes at least one phenotype of the target wheat;

[0015] The target phenotypic length determination module is used to determine the target phenotypic length corresponding to each phenotypic of each target wheat based on the target phenotypic image and the length markers in the first wheat image.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0017] One or more processors;

[0018] Memory, used to store one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for measuring wheat plant phenotypes as provided in any embodiment of the present invention.

[0020] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for measuring wheat plant phenotypes as provided in any embodiment of the present invention.

[0021] The technical solution of this invention involves: acquiring a first wheat image corresponding to a wheat region to be measured; inputting the first wheat image into a target mask generation model for semantic segmentation to obtain a first mask image corresponding to the wheat region; performing an extraction operation on the first mask image to obtain a second mask image corresponding to each target wheat in the wheat region, thereby obtaining a mask image for each target wheat; determining a target phenotypic image corresponding to each target wheat based on the first wheat image and the second mask image, thereby obtaining an image containing each individual target wheat, and determining the target phenotypic image based on the obtained individual target wheat image; the target phenotypic image includes at least one phenotype of the target wheat; and determining the target phenotypic length corresponding to each phenotype of each target wheat based on the target phenotypic image and length markers in the first wheat image, thereby achieving automatic measurement of the length of each phenotypic of wheat without the need for manual measurement, and allowing simultaneous measurement of multiple wheat plants, improving measurement efficiency and accuracy.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a method for measuring the phenotype of wheat plants provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is an example diagram illustrating the determination of section thickness according to Embodiment 1 of the present invention;

[0026] Figure 3 This is a flowchart of a method for measuring wheat plant phenotypes provided in Embodiment 2 of the present invention;

[0027] Figure 4 This is an example diagram illustrating how a second wheat image is determined based on a mask image, as described in Embodiment 2 of the present invention.

[0028] Figure 5 This is an example diagram illustrating how a third wheat image is obtained by cropping a second wheat image, as described in Embodiment 2 of the present invention.

[0029] Figure 6This is an example diagram of key point recognition and image segmentation according to Embodiment 2 of the present invention;

[0030] Figure 7 This is a schematic diagram of the structure of a wheat plant phenotyping device provided in Embodiment 3 of the present invention;

[0031] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] Figure 1 This is a flowchart illustrating a method for measuring wheat plant phenotypes according to an embodiment of the present invention. This embodiment is applicable to situations requiring automatic measurement of the lengths of various wheat phenotypes, and is particularly suitable for simultaneous automatic measurement of multiple wheat plants. This method can be executed by a wheat plant phenotype measuring device, which can be implemented in software and / or hardware and integrated into an electronic device. Figure 1 As shown, the method specifically includes the following steps:

[0036] S110. Obtain the first wheat image corresponding to the wheat region to be measured.

[0037] The wheat region can refer to the area where the wheat to be measured is placed. The first wheat image can refer to an image containing the complete wheat region acquired by an image acquisition device.

[0038] Specifically, the necessary acquisition environment for image acquisition is set up, including the acquisition platform, light source, tablecloth, and the position of the image acquisition device. The wheat to be measured can be cropped to reduce measurement errors caused by occlusion; that is, unnecessary parts such as leaves and roots are cropped away. The cropped wheat is placed non-overlapping in the prepared acquisition environment, such as placing it parallel to the tablecloth. Length markers are also placed on the tablecloth, ensuring they do not overlap with the wheat. The image acquisition device, with its lens perpendicular to the acquisition platform, acquires the first image of the wheat area to be measured. The controller can retrieve the first image of the wheat area from the image acquisition device or its corresponding image memory.

[0039] S120. Input the first wheat image into the target mask generation model for semantic segmentation to obtain the first mask image corresponding to the wheat region.

[0040] In this context, an image mask refers to using a selected image, graphic, or object to occlude (fully or partially) the image to be processed, thereby controlling the area or process of image processing. The first mask image can be an image containing the mask corresponding to all target wheat areas. The target wheat can be the wheat within the wheat region to be measured. Specifically, the first wheat image is input into the target mask generation model for semantic segmentation to obtain the first mask image corresponding to the wheat region. This can be understood as the wheat in the first wheat image representing the desired semantics, while other parts of the first image represent the unwanted semantics. The desired semantics are retained, and the unwanted semantics are removed. Finally, the first mask image is generated based on these two parts.

[0041] For example, the target mask generation model is obtained by training a pre-defined mask generation model using a training sample image set. The pre-defined mask generation model can refer to a pre-set, untrained mask generation model. For example, the pre-defined mask generation model can be, but is not limited to, a UNet network model, a DeepLab network model, or an FCN network model. The training sample image set contains a large number of annotated historical first wheat images. Furthermore, to increase the diversity and complexity of the training data in the training sample image set, data augmentation techniques can be used to enrich the samples.

[0042] It should be noted that image annotation tools (such as Labelme) can be used to annotate historically acquired images and perform data augmentation on them. Data augmentation methods can include, but are not limited to, image flipping, image rotation, random brightness adjustment, and random scaling. The historical first wheat image is input into a preset mask generation model for training. For example, the input image size is 1920×1920, where the preset mask generation model can use ResNet50 as its backbone feature extraction network, that is, the encoder part of the preset mask generation model uses ResNet50 instead of the original simple convolutional structure. The decoder will upsample the encoder output four times, each upsampling operation is achieved through deconvolution, the purpose of which is to gradually restore the spatial resolution of the original input image. During the upsampling process, skip connections are used to connect the feature maps of the corresponding encoder stage to the decoder stage to supplement more low-level features and spatial information. During model training, cross-entropy is used as the loss function, L2 regularization is used, and the L2 norm of the model weights is added as a penalty term to constrain the model complexity, prevent the model from overfitting, and improve the model's generalization ability.

[0043] S130. Extract the first mask image to obtain the second mask image corresponding to each target wheat in the wheat region.

[0044] The second mask image can be an image containing a mask corresponding to a single target wheat. Specifically, the wheat region can contain at least one target wheat. If the wheat region contains only one target wheat, the first mask image can be directly used as the second mask image. If the wheat region contains at least two target wheat, a contour extraction operation is performed on the first mask image, and a mask containing only a single target wheat is created based on the extracted contours, thereby obtaining the second mask image corresponding to each target wheat in the wheat region. The more target wheat contained in the wheat region, the shorter the time required to measure a batch of target wheat, thus improving the measurement efficiency of the target wheat phenotypic length.

[0045] S140. Based on the first wheat image and the second mask image, determine the target phenotypic image corresponding to each target wheat; the target phenotypic image includes at least one phenotype of the target wheat.

[0046] Here, phenotype can refer to a specific phenotype. For example, a phenotype can be, but is not limited to, the ear and stem of wheat. A target phenotype image can refer to an image containing the phenotype to be measured in the target wheat. For example, a target phenotype image can be, but is not limited to, an image containing only one phenotype of the target wheat, or an image containing multiple phenotypes of the target wheat, with each target phenotype selected by a bounding box.

[0047] Specifically, the first wheat image can be cropped based on the second mask image to obtain an image containing a target wheat in the wheat region, and the wheat feature recognition operation can be performed on the image to obtain the target phenotypic image corresponding to each target wheat.

[0048] For example, when the wheat features to be measured are identified, the image can be cropped to obtain an image containing only the wheat features to be measured.

[0049] S150. Based on the length markers in the target phenotypic image and the first wheat image, determine the target phenotypic length corresponding to each phenotypic of each target wheat.

[0050] The length marker can be, but is not limited to, a ruler, a square piece of paper, or a circular piece of paper. This embodiment does not limit the shape or material of the length marker. Specifically, the phenotypic pixel length corresponding to the phenotypic to be measured in the target phenotypic image is identified. The correspondence between the actual distance and the pixel distance is determined based on the length marker in the first wheat image. Based on the correspondence between the phenotypic pixel length and the actual distance and the pixel distance, the target phenotypic length corresponding to the phenotypic to be measured is determined, thereby determining the target phenotypic length corresponding to each phenotypic of each target wheat. For example, if the phenotypic to be measured is the ear of wheat, the pixel length corresponding to the ear of wheat in the target phenotypic image is determined, and based on the correspondence between the pixel length corresponding to the ear and the actual distance and the pixel distance, the actual length corresponding to the ear is determined, i.e., the target phenotypic length corresponding to the ear of wheat. If the phenotypic to be measured is the stem of wheat, the pixel length corresponding to the stem of wheat in the target phenotypic image is determined, and based on the correspondence between the pixel length corresponding to the stem and the actual distance and the pixel distance, the actual length corresponding to the stem is determined, i.e., the target phenotypic length corresponding to the stem of wheat.

[0051] It should be noted that, depending on the different training sample datasets for the model, the phenotypic length of other crops can be automatically measured.

[0052] The technical solution of this invention involves: acquiring a first wheat image corresponding to a wheat region to be measured; inputting the first wheat image into a target mask generation model for semantic segmentation to obtain a first mask image corresponding to the wheat region; performing an extraction operation on the first mask image to obtain a second mask image corresponding to each target wheat in the wheat region, thereby obtaining a mask image for each target wheat; determining a target phenotypic image corresponding to each target wheat based on the first wheat image and the second mask image, thereby obtaining an image containing each individual target wheat, and determining the target phenotypic image based on the obtained individual target wheat image; the target phenotypic image includes at least one phenotype of the target wheat; and determining the target phenotypic length corresponding to each phenotype of each target wheat based on the target phenotypic image and the length markers in the first wheat image, thereby achieving automatic measurement of the length of each phenotypic of wheat without the need for manual measurement, and allowing simultaneous measurement of multiple wheat plants, improving measurement efficiency and accuracy.

[0053] Based on the above technical solution, S130 may include: extracting the contour of the first mask image to obtain a third mask image corresponding to each target wheat in the wheat region; extracting the image skeleton of the third mask image to determine the central axis corresponding to each target wheat, and obtaining a second mask image corresponding to each target wheat in the wheat region based on the central axis.

[0054] The third mask image can be an image containing a mask corresponding to a target wheat. The second mask image contains a mask corresponding to a target wheat and the central axis corresponding to that target wheat. Points on the central axis are equidistant from both sides of the target wheat.

[0055] Specifically, contour extraction is performed on the first mask image obtained from semantic segmentation. Each extracted contour represents a single target wheat stalk. A mask corresponding to each single target wheat stalk is then constructed using this contour, resulting in a third mask image for each target wheat stalk in the wheat region. Based on the third mask image corresponding to a single target wheat stalk, a skeletonization algorithm can be used to extract the central axis corresponding to that single target wheat stalk. A second mask image for each target wheat stalk in the wheat region can then be obtained based on this central axis. For example, the skeletonization algorithm can be, but is not limited to, the Zhang-Suen skeletonization algorithm.

[0056] Based on the above technical solution, S150 may include: determining a partial central axis corresponding to each phenotype based on the central axis in the target phenotype image; determining the target pixel length corresponding to each partial central axis based on the pixel distance between each adjacent pixel in the partial central axis; determining the length conversion ratio based on the length marker in the first wheat image; and determining the target phenotype length corresponding to each phenotype of each target wheat based on the target pixel length and the length conversion ratio.

[0057] Here, "partial centerline" can refer to the centerline measured manually during testing. "Target pixel length" can refer to the pixel length corresponding to the phenotype. In actual measurement, the target wheat may have bent stems. In this case, it is necessary to measure the pixel length after the partial centerline is straightened, which is the pixel length of the target wheat after it is straightened.

[0058] For example, partial midlines corresponding to phenotypes such as spike head, spikelet stem, other nodes, and spike tail are determined based on the midline in the target phenotypic image. The lengths of the stalk, spike, and each node are calculated based on the determined partial midlines. Each node is segmented based on its corresponding features. The process of measuring the actual node length is as follows: determine the partial midline corresponding to the wheat node; determine the target pixel length corresponding to this partial midline based on the pixel distance between each adjacent pixel in this partial midline; determine the target phenotypic length corresponding to the wheat node based on the target pixel length and the length conversion ratio. The process of measuring the actual node thickness is as follows: take a position 2 cm below the node, and let this point be P(x... i ,y i ), calculate the equation of the tangent at that position, and calculate the equation of the perpendicular passing through that point. When the slope of the tangent equation is not 0, the tangent equation (1) and the perpendicular equation (2) can be obtained as follows:

[0059] y = m * (xx i )+y i (1)

[0060] y = (-1 / m) * (xx) i )+y i (2)

[0061] When the slope of the tangent equation is 0, the vertical equation can be directly obtained. Based on the vertical equation and the mask image corresponding to each segment, the pixel segment thickness of each segment is calculated. Based on the pixel segment thickness and length conversion ratio, the segment thickness length corresponding to each segment of the wheat is determined. Figure 2 An example diagram for determining section thickness is provided. See also... Figure 2 This includes a partial central axis, point P, a tangent line at point P that is tangent to the partial central axis, and a perpendicular line at point P that is perpendicular to the tangent line. Point P is the intersection of the triangle and the partial central axis.

[0062] Based on the above technical solution, "determining the length conversion ratio based on the length marker in the first wheat image" may include: obtaining the actual radius of the length marker in the first wheat image; performing color filtering on the first wheat image and determining the pixel area corresponding to the length marker in the filtered first wheat image; determining the pixel radius corresponding to the length marker based on the pixel area; and determining the length conversion ratio between the pixel length and the actual length based on the actual radius and the pixel radius.

[0063] In this embodiment, a light blue circular marker is used as an example to represent the length marker. The light blue circular marker is obtained through color filtering. For example, the first wheat image is converted to an HSV channel, and a lower limit (100, 20, 100) and an upper limit (150, 255, 255) for color filtering are set. Color filtering is then performed to extract the marker. The pixel area corresponding to the marker is calculated, and the pixel radius is calculated according to the formula for the area of ​​a circle. Based on the actual radius of the marker and the pixel radius, the length conversion ratio is determined.

[0064] Example 2

[0065] Figure 3 This is a flowchart of a method for measuring wheat plant phenotypes according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment describes in detail the process of determining the target phenotype image corresponding to the target wheat. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0066] like Figure 3 As shown, the method includes:

[0067] S210. Obtain the first wheat image corresponding to the wheat region to be measured.

[0068] S220. Input the first wheat image into the target mask generation model for semantic segmentation to obtain the first mask image corresponding to the wheat region.

[0069] S230. Extract the first mask image to obtain the second mask image corresponding to each target wheat in the wheat region.

[0070] S240. Perform bitwise operations on the first wheat image and the second mask image to determine the second wheat image corresponding to each target wheat; the second wheat image includes one target wheat.

[0071] The second wheat image can be an image of the same size as the first wheat image, but containing only one target wheat. In this embodiment, placing multiple target wheat images in the wheat region and performing phenotypic measurements together can improve the efficiency of phenotypic measurements, but it also puts pressure on computational resources, i.e., it consumes a large amount of computational resources. By performing bitwise operations on the first wheat image and the second mask image to determine the second wheat image corresponding to each target wheat, the computational resources can be distributed, and the phenotypic corresponding to each target wheat can be measured accurately one by one, thereby improving the measurement efficiency while ensuring the measurement accuracy.

[0072] S250. Based on the second wheat image, perform phenotypic recognition to obtain the target phenotypic image corresponding to each target wheat.

[0073] For example, Figure 4 An example diagram is provided for determining a second wheat image based on a mask image. See also... Figure 4 Taking a target wheat as an example, an extraction operation is performed on the first mask image to obtain the second mask image corresponding to the target wheat in the wheat region; the first wheat image and the second mask image are then subjected to bitwise operations to determine the second wheat image corresponding to the target wheat.

[0074] S260. Based on the length markers in the target phenotype image and the first wheat image, determine the target phenotype length corresponding to each phenotype of each target wheat.

[0075] The technical solution of this invention improves the efficiency of phenotypic measurement by placing multiple target wheat plants in a wheat region and performing phenotypic measurements together. However, this also puts pressure on computing resources, i.e., it consumes a large amount of computing resources. By performing bitwise operations on the first wheat image and the second mask image to determine the second wheat image corresponding to each target wheat plant, and performing phenotypic recognition based on the second wheat image to obtain the target phenotypic image corresponding to each target wheat plant, the computational resources can be distributed. The phenotypic corresponding to each target wheat plant can be measured accurately one by one, thereby improving the measurement efficiency while ensuring the measurement accuracy.

[0076] Based on the above technical solution, S250 may include: performing wheat coordinate recognition on the second wheat image, and cropping the image based on the recognized wheat coordinates to obtain a third wheat image corresponding to each target wheat; inputting the third wheat image into the target phenotype recognition model, and performing key point recognition and image segmentation in the target phenotype recognition model to obtain a target phenotype image corresponding to each target wheat.

[0077] Identifying key points allows us to obtain their corresponding coordinates. The third wheat image can refer to an image obtained by cropping along the smallest rectangle containing the wheat. Figure 5 An example image is provided showing how to crop a second wheat image to obtain a third wheat image. The target phenotypic recognition model is obtained after training a pre-defined phenotypic recognition model. The target phenotypic recognition model can be, but is not limited to, a Faster R-CNN model, an SSD model, or a YOLO model.

[0078] For example, the training process of the target phenotype recognition model is as follows: The Labelme tool is used to annotate historical wheat images. The spikelet, lower stem, other nodes, and spike tail are labeled using dots, and 64×64 ground truth bounding boxes are generated centered on these labeled points as the correct labels for network model training. Data augmentation can also be performed on the labeled images, including random cropping, image flipping, image rotation, random brightness, and random scaling. For example, the input image size can be 1920×1920, and ResNet50 is used as the backbone feature extraction network to extract features from the input image. 64×64 anchor boxes are generated from the feature map generated by the backbone feature extraction network, with a stride of 8 during the sliding window. The RPN layer classifies each generated anchor box into foreground and background binary categories and performs bounding box regression to obtain more accurate localization.

[0079] In bounding box regression, the mapping t between the proposal box and the ground truth bounding box is defined as follows:

[0080]

[0081]

[0082]

[0083]

[0084] Among them, a x a y a h a w These are the center coordinates, width, and height of the suggestion box, respectively. x g y g h g w These are the center coordinates, width, and height of the true bounding box. An inverse mapping is used, which calculates the corrected bounding box using the predicted correction values ​​and the proposed boxes.

[0085] g x =t x ·a w +a x

[0086] g y=t y ·a h +a y

[0087]

[0088]

[0089] RoI pooling is performed on the proposal bounding box regions, and RoIAlign replaces RoIPooling. RoIAlign eliminates the quantization step in RoIPooling and instead uses bilinear interpolation to sample from the precise location of each sub-window. Even if the sub-window size is not an integer or does not precisely fall on pixel boundaries, RoIAlign maintains spatial alignment between features and the original image, thus improving detection accuracy. Classification is performed through fully connected layers, including background, spike head, lower spike stem, other nodes, and spike tail. Boundary regression is also performed, identical to the bounding box regression in the RPN layer. Taking the center point yields the coordinates of the spike head, lower spike stem, other nodes, and spike tail of a single wheat stalk.

[0090] The total loss function is:

[0091]

[0092] The classification loss function for the RPN layer is:

[0093]

[0094] The regression loss function for the RPN layer is:

[0095]

[0096] The classification loss function for the last layer is:

[0097]

[0098] The loss function for the last layer regression is:

[0099]

[0100] During model training, combined cross-entropy is used as the loss function. Cross-entropy is used as the loss function for classification in the RPN layer and the last layer. SmoothL1 Loss is used as the loss function for bounding box regression in the RPN layer and regression in the last layer. L2 regularization adds the L2 norm of the model weights as a penalty term to prevent model overfitting.

[0101] Based on the above technical solution, "to perform key point recognition and image segmentation in the target phenotypic recognition model to obtain the target phenotypic image corresponding to each target wheat" may include: in the target phenotypic recognition model, identifying image features in the third wheat image, determining key points in each third wheat image based on the image features and preset phenotypic features, and performing image segmentation based on the key points; and obtaining the target phenotypic image corresponding to each target wheat based on the output of the target phenotypic recognition model.

[0102] For example, Figure 6 An example diagram of keypoint recognition and image segmentation is provided. See also... Figure 6 In the image, the circular patterns represent identified key points. Image segmentation is performed based on these key points to obtain the spike head, the lower stalk (equivalent to the first node), other nodes, and the spike tail (equivalent to the fifth node). The other nodes, from the spike head to the spike tail, are designated as the second, third, and fourth nodes. For example, two different phenotypes can be identified first, and key points on the target wheat can be determined based on the phenotype boundary line and the target wheat. Alternatively, key points on the target wheat can be determined based on the characteristics of key points between different phenotypes, such as the angle of inflection and the degree of curvature.

[0103] It should be noted that the determination of the target phenotypic image can also be performed before semantic segmentation. That is, the first wheat image is labeled with phenotypic features such as the head of the ear, the stem below the ear, other nodes, and the tail of the ear, and the target phenotypic recognition model is trained to predict the features of the wheat. Then, the individual target wheat is extracted through semantic segmentation, contour extraction and other steps. The previously predicted wheat features are assigned to each individual target wheat according to the location information corresponding to the features. The subsequent measurement steps remain unchanged.

[0104] The following are embodiments of the wheat plant phenotyping device provided in this invention. This device and the wheat plant phenotyping method described in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the wheat plant phenotyping device, please refer to the embodiments of the wheat plant phenotyping method described above.

[0105] Example 3

[0106] Figure 7 This is a schematic diagram of a wheat plant phenotyping device provided in Embodiment 3 of the present invention. Figure 7 As shown, the device includes: a first wheat image acquisition module 310, a first mask image acquisition module 320, a second mask image acquisition module 330, a target phenotype image determination module 340, and a target phenotype length determination module 350.

[0107] The system includes a first wheat image acquisition module 310, used to acquire a first wheat image corresponding to the wheat region to be measured; a first mask image acquisition module 320, used to input the first wheat image into a target mask generation model for semantic segmentation to obtain a first mask image corresponding to the wheat region; a second mask image acquisition module 330, used to extract from the first mask image to obtain a second mask image corresponding to each target wheat in the wheat region; a target phenotype image determination module 340, used to determine a target phenotype image corresponding to each target wheat based on the first wheat image and the second mask image; the target phenotype image includes at least one phenotype of the target wheat; and a target phenotype length determination module 350, used to determine the target phenotype length corresponding to each phenotype of each target wheat based on length markers in the target phenotype image and the first wheat image.

[0108] The technical solution of this invention involves: acquiring a first wheat image corresponding to a wheat region to be measured; inputting the first wheat image into a target mask generation model for semantic segmentation to obtain a first mask image corresponding to the wheat region; performing an extraction operation on the first mask image to obtain a second mask image corresponding to each target wheat in the wheat region, thereby obtaining a mask image for each target wheat; determining a target phenotypic image corresponding to each target wheat based on the first wheat image and the second mask image, thereby obtaining an image containing each individual target wheat, and determining the target phenotypic image based on the obtained individual target wheat image; the target phenotypic image includes at least one phenotype of the target wheat; and determining the target phenotypic length corresponding to each phenotype of each target wheat based on the target phenotypic image and the length markers in the first wheat image, thereby achieving automatic measurement of the length of each phenotypic of wheat without the need for manual measurement, and allowing simultaneous measurement of multiple wheat plants, improving measurement efficiency and accuracy.

[0109] Optionally, the second mask image acquisition module 330 is specifically used to: extract the contour of the first mask image to obtain a third mask image corresponding to each target wheat in the wheat region; extract the image skeleton of the third mask image to determine the central axis corresponding to each target wheat, and obtain a second mask image corresponding to each target wheat in the wheat region based on the central axis.

[0110] Optionally, the target phenotypic image determination module 340 may include:

[0111] The second wheat image determination submodule is used to perform bitwise operations on the first wheat image and the second mask image to determine the second wheat image corresponding to each target wheat; the second wheat image includes one target wheat.

[0112] The target phenotypic image determination submodule is used to perform phenotypic recognition based on the second wheat image to obtain the target phenotypic image corresponding to each target wheat.

[0113] Optionally, the target phenotype image determination submodule may include:

[0114] The third wheat image determination unit is used to identify wheat coordinates in the second wheat image and crop the image based on the identified wheat coordinates to obtain a third wheat image corresponding to each target wheat.

[0115] The target phenotypic image determination unit is used to input the third wheat image into the target phenotypic recognition model, perform key point recognition and image segmentation in the target phenotypic recognition model, and obtain the target phenotypic image corresponding to each target wheat.

[0116] Optionally, the target phenotypic image determination unit is specifically used to: identify image features in the third wheat image in the target phenotypic recognition model, determine key points in each third wheat image based on the image features and preset phenotypic features, and perform image segmentation based on the key points; and obtain the target phenotypic image corresponding to each target wheat based on the output of the target phenotypic recognition model.

[0117] Optionally, the target phenotype length determination module 350 may include:

[0118] The partial midline determination submodule is used to determine the partial midline corresponding to each phenotype based on the midline in the target phenotype image;

[0119] The target pixel length determination submodule is used to determine the target pixel length corresponding to each partial central axis based on the pixel distance between each adjacent pixel in the partial central axis.

[0120] The length conversion ratio determination submodule is used to determine the length conversion ratio based on length markers in the first wheat image;

[0121] The target phenotypic length submodule is used to determine the target phenotypic length corresponding to each phenotypic of each target wheat based on the target pixel length and length conversion ratio.

[0122] Optionally, the length conversion ratio determination submodule is specifically used to: obtain the actual radius of the length marker in the first wheat image; perform color filtering on the first wheat image and determine the pixel area corresponding to the length marker in the filtered first wheat image; determine the pixel radius corresponding to the length marker based on the pixel area; and determine the length conversion ratio between the pixel length and the actual length based on the actual radius and the pixel radius.

[0123] The wheat plant phenotype measurement device provided in this embodiment of the invention can execute the wheat plant phenotype measurement method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the wheat plant phenotype measurement method.

[0124] It is worth noting that in the embodiments of the above-mentioned wheat plant phenotypic measurement device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0125] Example 4

[0126] Figure 8 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0127] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0129] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for measuring wheat plant phenotypes.

[0130] In some embodiments, the method for measuring wheat plant phenotypes may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for measuring wheat plant phenotypes described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the method for measuring wheat plant phenotypes by any other suitable means (e.g., by means of firmware).

[0131] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0136] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0137] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0138] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of measuring a phenotype of a wheat plant, characterized in that, The method comprises: acquiring a first wheat image corresponding to a wheat area to be measured; inputting the first wheat image into a target mask generation model for semantic segmentation to obtain a first mask image corresponding to the wheat area; performing extraction on the first mask image to obtain a second mask image corresponding to each target wheat in the wheat area; determining a target phenotype image corresponding to each target wheat based on the first wheat image and the second mask image; the target phenotype image comprises at least one phenotype of the target wheat; determining a target phenotype length corresponding to each phenotype of each target wheat based on the target phenotype image and a length marker in the first wheat image; wherein the determination of the target phenotype image corresponding to each target wheat based on the first wheat image and the second mask image comprises performing a bitwise operation on the first wheat image and the second mask image to determine a second wheat image corresponding to each target wheat; the second wheat image comprises one target wheat; and performing phenotype identification based on the second wheat image to obtain the target phenotype image corresponding to each target wheat; the phenotype identification based on the second wheat image to obtain the target phenotype image corresponding to each target wheat comprises: performing wheat coordinate identification on the second wheat image, and performing image cropping based on the identified wheat coordinates to obtain a third wheat image corresponding to each target wheat; inputting the third wheat image into a target phenotype identification model, performing key point identification and image segmentation in the target phenotype identification model, and obtaining the target phenotype image corresponding to each target wheat; the key point identification and image segmentation in the target phenotype identification model to obtain the target phenotype image corresponding to each target wheat comprises: identifying image features in the third wheat image in the target phenotype identification model, determining key points in each third wheat image based on the image features and preset phenotype features, and performing image segmentation based on the key points; and obtaining the target phenotype image corresponding to each target wheat based on the output of the target phenotype identification model.

2. The method of claim 1, wherein, the extraction of the first mask image to obtain the second mask image corresponding to each target wheat in the wheat area comprises: contour extraction on the first mask image to obtain a third mask image corresponding to each target wheat in the wheat area; image skeleton extraction on the third mask image to determine a central axis corresponding to each target wheat, and obtaining the second mask image corresponding to each target wheat in the wheat area based on the central axis.

3. The method of claim 1, wherein, the determination of the target phenotype length corresponding to each phenotype of each target wheat based on the target phenotype image and the length marker in the first wheat image comprises: determining a partial central axis corresponding to each phenotype based on the central axis in the target phenotype image; determining a target pixel length corresponding to each partial central axis based on a pixel distance between each adjacent pixel in the partial central axis; and determine a length conversion ratio based on the length marker in the first wheat image; determine a target phenotype length corresponding to each phenotype of each target wheat based on the target pixel length and the length conversion ratio.

4. The method of claim 1, wherein, The method further includes: obtain an actual radius of the length marker in the first wheat image; perform color filtering on the first wheat image and determine a pixel area corresponding to the length marker in the filtered first wheat image; determine a pixel radius corresponding to the length marker based on the pixel area; determine a length conversion ratio between a pixel length and an actual length based on the actual radius and the pixel radius.

5. A device for measuring a phenotype of a wheat plant, characterized in that, The method further includes: obtain a first wheat image corresponding to a wheat region to be measured; input the first wheat image into a target mask generation model to perform semantic segmentation and obtain a first mask image corresponding to the wheat region; perform extraction on the first mask image to obtain a second mask image corresponding to each target wheat in the wheat region; determine a target phenotype image corresponding to each target wheat based on the first wheat image and the second mask image; The target phenotype image includes at least one phenotype of the target wheat. determine a target phenotype length corresponding to each phenotype of each target wheat based on the target phenotype image and a length marker in the first wheat image. The target phenotype image determination module includes: a second wheat image determination submodule that determines a second wheat image corresponding to each target wheat by performing a bitwise operation on the first wheat image and the second mask image; and a target phenotype image determination submodule that determines a target phenotype image corresponding to each target wheat based on the second wheat image.

6. An electronic device, comprising: The target phenotype image determination submodule includes: a third wheat image determination unit that identifies wheat coordinates in the second wheat image and performs image cropping based on the identified wheat coordinates to obtain a third wheat image corresponding to each target wheat; and a target phenotype image determination unit that inputs the third wheat image into a target phenotype recognition model, performs key point recognition and image segmentation in the target phenotype recognition model, and obtains a target phenotype image corresponding to each target wheat. The target phenotype image determination unit is specifically configured to: identify image features in the third wheat image in the target phenotype recognition model, determine key points in each third wheat image based on the image features and preset phenotype features, and perform image segmentation based on the key points; and obtain a target phenotype image corresponding to each target wheat based on an output of the target phenotype recognition model. The electronic device includes: one or more processors; a memory for storing one or more programs; and When the one or more programs are executed by the one or more processors, the one or more processors implement a method of measuring a phenotype of a wheat plant as claimed in any one of claims 1-4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The program, when executed by a processor, implements a method of measuring a phenotype of a wheat plant as claimed in any one of claims 1-4.

Images