A machine learning-based method and system for detecting oat phenotypes and its application in oat breeding.

Abstract

This specification provides a machine learning-based oat phenotypic detection method, system, and its application in oat breeding. The machine learning-based oat phenotypic detection method includes: acquiring a whole-plant image of an oat plant to be detected; annotating multiple key morphological features of the oat plant in the whole-plant image using a phenotypic detection model; cropping the whole-plant image based on the generated annotation results to obtain a local image containing the key morphological features, and extracting the skeleton point set of the oat plant based on the local image; performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, spike length, length of the internode below the spike, leaf width, number of whorls, number of spikelets, and stem diameter.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of oat phenotyping technology, and in particular to a machine learning-based oat phenotyping method. Background Technology

[0002] Food security is a major global challenge, and crop phenomics is widely recognized as a key technology for accelerating breeding processes and bridging the genotype-phenotype gap. As an important crop used for both food and forage, oat's plant height, spike length, leaf morphology, and spikelet number directly determine its lodging resistance and yield potential. However, current oat phenotyping methods still primarily rely on manual field measurements. This method is not only labor-intensive and time-consuming, but also prone to causing irreversible damage to plants due to contact measurements, severely restricting the accurate screening and evaluation of large-scale germplasm resources. Therefore, developing high-throughput, non-destructive intelligent phenotyping technology using simplified equipment and artificial intelligence algorithms is of significant strategic importance for improving breeding timeliness.

[0003] In recent years, computer vision technologies, represented by deep learning, have made significant progress in fields such as crop disease identification, organ counting, and growth monitoring. In particular, the YOLO (You Only Look Once) series of algorithms, with their efficient end-to-end inference capabilities, have become the mainstream choice for real-time phenotypic detection.

[0004] However, existing general detection models still face significant challenges for high-density, morphologically complex gramineous crops: First, oat plants have long, slender, and interwoven leaves, and the severe shading in high-density planting environments makes it difficult for traditional convolutional neural networks to capture the complete features of small targets (such as spikelets and leaf tips), often leading to missed detections; second, the architecture of mainstream convolutional neural networks is limited by local receptive fields, lacking the ability to perceive the global topological structure of the entire plant, making it difficult to handle long-distance pixel dependencies; finally, high-precision models usually require a huge number of parameters and computational power, making it difficult to achieve smooth deployment on portable mobile terminals such as smartphones, thus limiting their application in field breeding. Therefore, an effective method is urgently needed to solve these problems. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a machine learning-based oat phenotypic detection method. One or more embodiments of this specification also relate to a machine learning-based oat phenotypic detection system, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a machine learning-based method for detecting oat phenotypes is provided, comprising: Obtain whole-plant images of oat plants to be tested; The phenotypic detection model is used to annotate multiple key morphological features of the oat plant in the whole plant image to be detected. The whole plant image to be detected is cropped based on the generated annotation results to obtain a local image containing key morphological features, and the skeleton point set of the oat plant is extracted based on the local image. Phenotypic calculations are performed based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, ear length, length of the internode below the ear, leaf width, number of whorls, number of spikelets, and stem diameter.

[0007] Optionally, the step of extracting the skeleton point set of the oat plant based on the local image includes: The local image is processed to generate a corresponding mask image, wherein the mask image contains a target mask for describing key morphological features; The target mask in the mask image is iteratively eroded using a thinning algorithm to generate the topological center line of the oat plant's leaves. Key points are extracted from the topological centerline, and these extracted key points are used as the skeleton point set of the oat plant's leaves.

[0008] Optionally, processing the local image to generate a corresponding mask image includes: Adaptive threshold segmentation is performed on the local image to generate an intermediate image containing the threshold segmentation results; Based on the threshold segmentation result, morphological closing operation is performed on the intermediate image to generate the corresponding mask image.

[0009] Optionally, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: In the case where the key morphological feature is an oat ear, the coordinates of the key points corresponding to the highest and lowest points of the oat ear in the local image are determined based on the annotation results. Calculate the Euclidean distance between the coordinates of the key points corresponding to the highest and lowest points of the oat ear, and determine the calculation result as the phenotypic detection result corresponding to the ear length of the oat ear.

[0010] Optionally, the annotation result includes a rectangular box; Accordingly, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: When the key morphological feature is an oat ear, the diagonal length of the rectangle corresponding to the oat ear is determined as the phenotypic detection result corresponding to the ear length of the oat ear.

[0011] Optionally, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: When the key morphological feature is a leaf, the least squares method is used to fit the skeleton point set of the leaf to generate the main direction vector corresponding to the leaf, wherein the main direction vector is used to characterize the growth direction of the leaf. Multiple sets of tangents are generated along the normal direction of the main direction vector, and the phenotypic detection result corresponding to the blade width is determined based on the distance between the intersection point of each set of tangents and the edge contour of the blade.

[0012] Optionally, the machine learning-based oat phenotypic detection method further includes: In cases where the key morphological features include blades, a target local image containing blades is determined; In the target local image, polygons are used to perform key point annotation processing to generate key point information corresponding to the blade; The target local image carrying the key point information is input into the leaf extraction model to optimize the leaf extraction model using the target local image carrying the key point information. The leaf extraction model is used to generate phenotypic detection results corresponding to the leaf width.

[0013] According to a second aspect of the embodiments of this specification, a machine learning-based oat phenotyping system is provided, comprising: The acquisition module is configured to acquire images of the entire oat plant to be detected; The annotation module is configured to annotate multiple key morphological features of the oat plant in the whole-plant image to be detected using a phenotypic detection model. The extraction module is configured to crop the whole plant image to be detected based on the generated annotation results to obtain a local image containing key morphological features, and extract the skeleton point set of the oat plant based on the local image; The generation module is configured to perform phenotypic calculations based on the skeleton point set and the annotation results, and generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, ear length, length of the lower internode, leaf width, number of whorls, number of spikelets, and stem diameter.

[0014] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement any of the steps of the machine learning-based oat phenotypic detection method.

[0015] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of any of the machine learning-based oat phenotypic detection methods described above.

[0016] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described oat phenotype detection method based on machine learning.

[0017] This embodiment of the specification acquires a whole-plant image of an oat plant to be detected; it then uses a phenotypic detection model to annotate multiple key morphological features of the oat plant in the whole-plant image; based on the generated annotation results, the whole-plant image is cropped to obtain a local image containing the key morphological features, and the skeleton point set of the oat plant is extracted from the local image; phenotypic calculation is performed based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results, whereby the phenotypic features include plant height, spike length, length of the internode below the spike, leaf width, number of whorls, number of spikelets, and stem diameter. This embodiment of the specification effectively enhances the global perception and long-distance feature capture capabilities of the phenotypic detection model in complex occlusion environments. Therefore, using the phenotypic detection model to annotate key morphological features in the whole-plant image to be detected can effectively capture the long-distance dependency between key morphological features and the background, thereby ensuring the positioning accuracy of each key morphological feature. Subsequent phenotypic calculation based on the annotation results of each key morphological feature can achieve accurate locking and visual confirmation of the counting target. This processing method effectively reduces the false negative rate, thereby improving the accuracy of the detection results. Attached Figure Description

[0018] Figure 1 This is a flowchart of a machine learning-based oat phenotypic detection method provided in one embodiment of this specification; Figure 2 This is a schematic diagram of the structure of a phenotypic detection model provided in one embodiment of this specification; Figure 3 This is a schematic diagram of a whole-plant image to be detected provided in one embodiment of this specification; Figure 4 This is a schematic diagram of a blade outline annotation result provided in one embodiment of this specification; Figure 5 This is a schematic diagram of the result of the number of whorls in the ear provided in one embodiment of this specification; Figure 6This is a schematic diagram of a spikelet number labeling result provided in one embodiment of this specification; Figure 7 This is a schematic diagram of an intelligent recognition result of the number of oat spikelets provided in one embodiment of this specification; Figure 8 This is a schematic diagram of an intermediate process for detecting the whole plant phenotype of oats, provided in one embodiment of this specification; Figure 9 This is a schematic diagram illustrating the changing trends of the loss function and evaluation metrics during the training process of a phenotypic detection model, as provided in one embodiment of this specification. Figure 10 This is a schematic diagram of the precision-recall (PR) curve of a spikelet counting model provided in one embodiment of this specification on the validation set; Figure 11 This is a schematic diagram of the structure of an oat phenotypic detection system based on machine learning, provided in one embodiment of this specification. Figure 12 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0019] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0020] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0022] This specification proposes a machine learning-based method for oat phenotypic detection. It primarily utilizes an improved YOLOv12 model that integrates a lightweight Vision Transformer (ViT) network for phenotypic detection. Specifically, based on the latest YOLOv12 architecture, this specification embeds a simplified, lightweight ViT network at the end of the backbone network. This design cleverly combines the local feature extraction advantages of convolutional neural networks with the self-attention mechanism of the Transformer, effectively enhancing the model's global perception and long-distance feature capture capabilities in complex occlusion environments. Furthermore, this specification constructs a cascaded analytical framework of "macroscopic localization—microscopic image slicing—geometric reconstruction," achieving precise quantification from image pixels to physical agronomic traits.

[0023] The embodiments in this specification aim to: (1) verify the robustness of the improved model in complex field backgrounds; (2) break through the limitations of traditional methods and realize automated high-precision measurement of whole oat plants and organs; (3) develop and verify a mobile-based intelligent analysis system (OatPheno App) to provide an efficient, low-cost and practical digital solution for smart breeding.

[0024] This specification provides a machine learning-based oat phenotypic detection method, and also relates to a machine learning-based oat phenotypic detection system, a computing device, a computer-readable storage medium, and a computer program, which are described in detail in the following embodiments.

[0025] Figure 1 A flowchart of a method for detecting the whole-plant phenotype of oats according to an embodiment of this specification is shown, which specifically includes the following steps.

[0026] Step 102: Obtain the whole-plant image of the oat plant to be tested.

[0027] Specifically, the obtained whole-plant image to be detected is an image of the oat plant for which phenotypic detection is required. The whole-plant image to be detected can be a front view of the oat plant taken from the front of the oat plant, which can ensure that the entire oat plant is within the field of view.

[0028] Step 104: Use a phenotypic detection model to annotate multiple key morphological features of the oat plant in the whole plant image to be detected.

[0029] Specifically, the phenotypic detection model is an improved YOLOv12 model, which involves embedding a lightweight visual Transformer network between the backbone and neck networks of the YOLOv12 model.

[0030] The embodiments in this specification are based on the latest YOLOv12 deep learning framework and construct a phenotypic analysis system consisting of a global localization master model (phenotypic detection model) and four task-specific local parsing sub-models (spikelet counting model, whorl identification model, leaf extraction model, and stem diameter localization model).

[0031] The phenotypic detection model is an improved YOLOv12 architecture that integrates a lightweight VisionTransformer network. Although the original YOLOv12 model enhances feature extraction capabilities by introducing AreaAttention and R-ELAN modules, it is still fundamentally limited by the local receptive field mechanism of a convolutional neural network (CNN). When dealing with complex objects like oat plants, which have elongated shapes and overlapping occlusions, traditional convolutional operations struggle to capture long-distance pixel dependencies, resulting in limited localization accuracy at the whole-plant scale. Furthermore, the large number of parameters in the original model leads to high computational redundancy and memory consumption. Therefore, this embodiment reconstructs the architecture at the connection points between the backbone and neck of the YOLOv12 model, generating an improved whole-plant oat detection model (phenotypic detection model) that balances accuracy and computational efficiency. The specific improvement strategies are as follows: (1) Introducing a global attention mechanism at the backbone end: such as Figure 2 As shown, in this embodiment, a custom Vision Transformer (ViT) module is embedded after the P5 deep feature map (32x downsampling) output by the backbone network. Given that the P5 layer gathers the highest-level semantic information of the image, a multi-head self-attention mechanism is introduced here to establish contextual relationships across the entire image before feature fusion (Neck), thereby effectively mitigating the problem of missed detections caused by local occlusion.

[0032] (2) Lightweight Structural Design: To reduce model computational complexity and improve inference efficiency, the embodiments in this specification retain the advantages of ViT global modeling while implementing targeted structural simplification. The high computational cost of traditional ViT is abandoned, and the Transformer module is restructured through a "parameter reduction" strategy: the embedding dimension is compressed from 256 dimensions to 128 dimensions, the number of attention heads is reduced from 8 to 4, and the number of stacked layers is limited to 2. This strategy reduces the number of internal parameters of the improved module by approximately 40%, effectively controlling the increase in floating-point operations (FLOPs) caused by the introduction of the global attention mechanism. This design successfully reduces the total number of model parameters at the cost of a slight increase in inference computational overhead, significantly improving parameter utilization efficiency.

[0033] The improved phenotypic detection model mainly undertakes the task of macroscopic skeleton extraction. Through end-to-end inference on full-resolution images, it quickly generates coarse-grained region suggestions containing components such as stems, ears, and leaves, providing high-quality, low-noise candidate regions for downstream fine analysis.

[0034] In addition, in the embodiments of this specification, the key morphological features may include six types: stem, ear, lower node of ear, flag leaf, second leaf from the top, and number of leaves (as a substitute indicator for the number of stem nodes).

[0035] After inputting the whole-plant image to be detected into the phenotypic detection model, the phenotypic detection model can predict the bounding boxes (excluding the number of leaves) of each key morphological feature by performing object detection tasks. The core of this strategy is to achieve "complete preservation of original resolution features", that is, based on the bounding box coordinates predicted by the phenotypic detection model, the corresponding local regions can be directly located and cropped on the high-resolution original image (the whole-plant image to be detected) for analysis, instead of using the low-resolution feature map scaled by the model.

[0036] In practical applications, before annotating multiple key morphological features of oat plants in the whole-plant image to be detected using the phenotypic detection model, the phenotypic detection model must first be trained.

[0037] Specifically, we can first select diverse oat germplasm resources from an agricultural research and experimental base as experimental subjects, construct a phenotypic dataset covering multi-scale traits, that is, collect images of each experimental subject to obtain sample data for model training.

[0038] In practical applications, to ensure the diversity and reliability of image data, three biological replicates were collected for each experimental material, resulting in the photographing of approximately 450 oat plants in total. All oat samples were photographed in an indoor environment to ensure consistent image quality. Uniform natural light was used during photography to prevent shadows and reflections from interfering with image quality. A single-color background cloth (white) was selected as the shooting background to reduce background interference and improve the accuracy of image segmentation. During photography, it was ensured that the oat samples were laid flat to avoid overlapping or obstruction.

[0039] During image capture, a transparent acrylic sheet was used to cover the oat leaves to ensure the flag and second-to-last leaves were flat and facilitate subsequent length and width measurements. A ruler was placed within the shooting area for later conversion of actual dimensions. When counting the oat spikelets, the spikelets were carefully removed from the oat plant, preserving their integrity and natural shape as much as possible. The removed oat spikelets were laid flat on black cardstock, ensuring no overlap between spikelets to minimize obstruction and facilitate subsequent target detection and counting. Black cardstock was chosen because its high-contrast background helps improve the accuracy of target detection. Finally, before starting the shooting, the stability of the measuring rod and the equipment used for image capture was carefully checked to ensure there was no looseness or shaking. A schematic diagram of the captured whole-plant image is shown below. Figure 3 As shown.

[0040] After image acquisition, this embodiment breaks through the traditional single-layer full-feature annotation strategy and proposes a hierarchical annotation strategy of "main feature localization + region of interest (ROI) cascaded parsing". For the original high-resolution image, the aim is to establish the macroscopic topological architecture of the plant. Using the SAM (Segment Anything Model) assisted annotation tool, six key morphological features are defined and labeled: stem, spike, lower node of spike, flag leaf, second leaf from the top, and leaf number (as a substitute indicator for stem node number). The annotated image is then input into the phenotypic detection model to be trained, thus obtaining the phenotypic detection model.

[0041] Given the significant scale imbalance in whole-plant images: on the one hand, minute features such as stem thickness, whorl number, and spikelets have extremely low pixel counts in panoramic images, directly performing full-scale detection easily leads to insufficient mapping of the Feature Pyramid Network (FPN) hierarchy, resulting in a high false detection rate; on the other hand, large-scale organs such as leaves are limited by the downsampling mechanism at the input of deep learning networks, resulting in severe loss of high-frequency edge information, which cannot meet the subsequent requirements for leaf width measurement based on pixel-level accuracy. Therefore, a "macro-localization-first" annotation strategy is adopted to remove minute feature labels during the training of the phenotypic detection model, focusing on achieving fast and robust localization of the Region of Interest (ROI) for each organ.

[0042] After training, for leaf blades such as the flag leaf and the second-to-last leaf, the phenotypic detection model predicts their bounding rectangles by performing object detection tasks. The core of this strategy lies in achieving "complete preservation of original resolution features." That is, based on the bounding box coordinates predicted by the phenotypic detection model, the corresponding local regions can be directly located and cropped from the high-resolution original image for analysis, rather than using the low-resolution feature map scaled down from the model. This cascaded processing mode of "coarse-grained localization - fine-grained analysis" effectively avoids the edge blurring effect caused by downsampling at the input of deep learning networks, maximizing the preservation of high-frequency detail information at the leaf margin, and laying a solid data foundation for subsequent accurate leaf width measurement based on geometric morphology.

[0043] In addition, to overcome the bottleneck of detecting subtle phenotypic indicators such as leaf edge blurring, whorl number, and spikelet number, the embodiments in this manual adopt a cascaded processing strategy of "localization-cropping-re-annotation" to construct dedicated subset datasets for different resolution requirements, and use each dedicated subset dataset to train the corresponding sub-model.

[0044] (1) Fine-grained segmentation and annotation of leaf morphology: To address the loss of high-frequency information of leaf edges due to downsampling when inputting panoramic images into the model, a dedicated segmentation subset for "leaf morphology" was constructed. High-resolution leaf sub-images were non-destructively cropped from the original image using the ROI coordinates located by the phenotypic detection model. At the sub-layer level, polygons were then used for pixel-level fine-grained annotation of the leaf contours (e.g., ...). Figure 4 (As shown). After annotation, the leaf extraction model can be trained using the leaf sub-image containing leaf contour annotation data. The construction of this dataset effectively recovers leaf margin details, ensuring that the subsequent leaf extraction model can output a high-confidence binary mask, laying the foundation for accurate quantization of leaf length and width. The trained leaf extraction model can provide high-quality input for subsequent morphological skeleton extraction algorithms. This model focuses on foreground and background separation within the leaf ROI, enhances edge response to high-frequency information at the leaf base and tip, and effectively suppresses texture interference introduced by background weeds, ensuring the integrity of leaf shape reconstruction.

[0045] (2) Fine-grained labeling of the number of whorls: Based on the ear region initially detected by the phenotypic detection model, a high-resolution close-up image of it is automatically extracted using an algorithm, and then a special dataset for fine-grained labeling of the number of whorls is constructed (e.g., Figure 5(As shown). After annotation, the dataset can be used to train a whorl number recognition model. This hierarchical processing method effectively suppresses background noise by focusing the analysis view on the target subject, significantly enhancing the signal-to-noise ratio (SNR) of the whorl number feature. The trained whorl number recognition model receives a high-resolution whorl ROI sub-map of the ear, cropped by the phenotypic detection model, targeting the complex hierarchical whorl topology of the ear. Through inference within the local microscopic field of view, the spatial distribution characteristics of the whorl layer nodes are accurately analyzed, effectively solving the feature aliasing problem at the full-map scale.

[0046] (3) Semi-automatic spikelet labeling process: Addressing the challenge of extremely small spikelet size and lack of texture features, a composite process of "traditional image processing + human-computer interactive labeling" was designed. First, the OpenCV algorithm was used to perform threshold segmentation and contour extraction on the original image, automatically separating the spikelet region against the black cardboard background; then, the data was imported into the TrexLabel platform for batch labeling (e.g., ...). Figure 6 As shown in the figure, its efficient interactive interface reduces the annotation cost of dense small targets; finally, the annotated data is standardized into YOLO format. After annotation, the spikelet counting model can be trained using the annotated data. The trained spikelet counting model is designed for the detection task of high-density, extremely small-scale targets. This model is trained on a dedicated subset dataset, and by optimizing the loss function, it significantly improves the recall and localization accuracy of overlapping targets in densely occluded scenes.

[0047] Furthermore, based on the stem region initially detected by the phenotypic detection model, a high-resolution close-up image of it can be automatically extracted using an algorithm. This image is then used to train a stem thickness localization model, which focuses on identifying effective stem segments with clear texture and complete morphology. This model possesses semantic filtering capabilities, automatically avoiding leaf sheath obstruction and damaged root areas, adaptively searching for and locking onto the most representative unobstructed areas, providing high-confidence optimal observation sites for subsequent geometric morphology-based stem thickness measurement.

[0048] In summary, by constructing a hierarchical multi-scale dataset containing both macroscopic main features and microscopic local features, we not only solved the sample imbalance problem of multi-objective training in a single network, but also laid a high-quality localization foundation for subsequent integration of deep learning and traditional image processing techniques (such as geometric measurement and skeletonization analysis), thereby significantly improving the overall robustness of the phenotypic analysis algorithm.

[0049] Step 106: Based on the generated annotation results, crop the whole plant image to be detected to obtain a local image containing key morphological features, and extract the skeleton point set of the oat plant based on the local image.

[0050] In one optional implementation, extracting the skeleton point set of the oat plant based on the local image includes: The local image is processed to generate a corresponding mask image, wherein the mask image contains a target mask for describing key morphological features; The target mask in the mask image is iteratively eroded using a thinning algorithm to generate the topological center line of the oat plant's leaves. Key points are extracted from the topological centerline, and these extracted key points are used as the skeleton point set of the oat plant's leaves.

[0051] Furthermore, the process of processing the local image to generate a corresponding mask image includes: Adaptive threshold segmentation is performed on the local image to generate an intermediate image containing the threshold segmentation results; Based on the threshold segmentation result, morphological closing operation is performed on the intermediate image to generate the corresponding mask image.

[0052] Specifically, the thinning algorithm is a classic algorithm in image morphology processing. It is used to peel away the foreground objects in a binary image layer by layer and finally extract their skeletons—that is, the center line with a width of one pixel—while keeping the original object's topological structure (connectivity, endpoints, holes) basically unchanged.

[0053] Adaptive thresholding is an image segmentation technique used to separate the foreground (such as an object) from the background in an image.

[0054] In addition, the closing operation is a combination of dilation and erosion. Its main effects are to fill small holes inside the target object in the image, close narrow breaks or gaps, eliminate small discrete noise, and make the object's outline smoother and more continuous.

[0055] In the embodiments of this specification, the whole-plant model, i.e., the phenotypic detection model, is first used to perform coarse-grained localization on the whole-plant image to be detected, and the ROI of each key morphological feature is extracted. Differentiated preprocessing strategies are designed for the morphological characteristics of different key morphological features.

[0056] (1) Adaptive correction of the spikelet ROI region: For dense spikelet samples placed on black cardstock, the algorithm first calculates the minimum bounding rectangle of the entire spikelet region (i.e., the black cardstock background). Then, affine transformation is applied to perform global rotation correction, standardizing the rectangular cardstock region that was originally tilted due to shooting or placement into a horizontal or vertical orthogonal viewpoint before inputting it into a dedicated spikelet counting model. This "field of view standardization" strategy effectively eliminates the spatial noise introduced by the background tilt, ensuring the consistency of the model input data in the geometric direction, thereby significantly improving the detection recall rate of dense small targets.

[0057] (2) Hierarchical Redundancy Removal Mechanism: To address the problem of hierarchical redundancy detection boxes that may occur in complex backgrounds (such as redundancy in large frames of the whole plant containing small frames of leaves), this specification's embodiments introduce an improved non-maximum suppression strategy. This algorithm filters detection boxes (rectangular boxes) based on their confidence level and spatial hierarchy. When multiple detection boxes highly overlap, it prioritizes retaining high-confidence results representing key local morphological features and automatically suppresses large-scale detection boxes containing redundant backgrounds, thereby ensuring that each object delivered from the phenotypic detection model to the downstream sub-model is unique.

[0058] Furthermore, although the embodiments in this specification utilize an acrylic plate for physical flattening during the image acquisition stage to eliminate the interference of three-dimensional leaf curling, directly measuring leaf width based on the detection box output by the phenotypic detection model still faces significant systematic errors. This is mainly due to two factors: first, the leaves exhibit random tilting postures, causing the width of the horizontal bounding box to fail to represent the true physical width; second, the downsampling operation during the full-scale image inference process induces edge aliasing effects, making it difficult to meet the requirements of millimeter-level precision measurement. Therefore, the embodiments in this specification adopt a "full-image localization and cropping—sub-image fine analysis" strategy, introducing a morphological skeleton extraction algorithm within the cropped high-resolution ROI subdomain to construct an accurate geometric measurement benchmark. The specific process is as follows: (1) ROI cropping and adaptive segmentation: First, based on the localization coordinates of the phenotypic detection model, ROI sub-images containing complete leaves and ROI sub-images containing complete stems are cropped non-destructively from the whole-plant image to be detected. For the sub-image domain, image channels are separated and contrast enhancement is performed on the green channel (for thick stems, the S channel of HSV space is processed) to maximize the feature difference between the foreground and background. Subsequently, adaptive threshold segmentation and morphological closing operation are jointly applied to generate a high-quality binarized mask with smooth edges and no internal holes to effectively recover the high-frequency edge information lost in the whole-image prediction.

[0059] (2) Iterative skeleton extraction: For the leaves, a morphological thinning algorithm is used to iteratively erode the binary mask to extract the topological centerline of the leaves. Key points are extracted from the topological centerline and used as the skeleton point set of the oat plant leaves. This step abstracts the leaf entity with a certain width into a skeleton structure with a single pixel width, providing a unique path reference for subsequent geometric calculations.

[0060] Once the skeleton point set is extracted, the subsequent phenotypic calculation process can begin.

[0061] Step 108: Perform phenotypic calculations based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, ear length, length of the internode below the ear, leaf width, number of whorls, number of spikelets, and stem diameter.

[0062] In one optional implementation, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: In the case where the key morphological feature is an oat ear, the coordinates of the key points corresponding to the highest and lowest points of the oat ear in the local image are determined based on the annotation results. Calculate the Euclidean distance between the coordinates of the key points corresponding to the highest and lowest points of the oat ear, and determine the calculation result as the phenotypic detection result corresponding to the ear length of the oat ear.

[0063] In one optional implementation, the annotation result includes a rectangular box; Accordingly, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: When the key morphological feature is an oat ear, the diagonal length of the rectangle corresponding to the oat ear is determined as the phenotypic detection result corresponding to the ear length of the oat ear.

[0064] In one optional implementation, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: When the key morphological feature is a leaf, the least squares method is used to fit the skeleton point set of the leaf to generate the main direction vector corresponding to the leaf, wherein the main direction vector is used to characterize the growth direction of the leaf. Multiple sets of tangents are generated along the normal direction of the main direction vector, and the phenotypic detection result corresponding to the blade width is determined based on the distance between the intersection point of each set of tangents and the edge contour of the blade.

[0065] Specifically, when the key morphological feature is the leaf, the `cv2.fitLine` function can be used to fit the extracted leaf skeleton point set using the least squares method to calculate the principal direction vector representing the leaf's growth direction. The algorithm generates multiple sets of scanning tangents along the normal vector direction of the fitted line and calculates the distance between the intersection points of the tangents and the leaf edge contour (i.e., the slice width). Finally, the maximum or average value of the multiple sets of slice data is taken as the maximum leaf width. This processing method completely avoids the problem of the rectangular frame width being artificially high due to the tilted placement of the leaf, achieving accurate measurement that conforms to biological definitions.

[0066] It should be noted that for ROI sub-images containing complete stems, adaptive threshold segmentation and morphological closing operations are applied to process them to generate a high-quality binary mask with smooth edges and no internal holes. The specific implementation process of determining stem thickness through iterative skeleton extraction and least squares normal vector fitting is similar to the corresponding implementation process of calculating leaf width mentioned above, and will not be repeated here.

[0067] In the leaf and stem diameter measurement embodiments described in this specification, the algorithm uses the fitting result of the skeleton point set as the central axis. By establishing a local orthogonal coordinate system, it performs dense scanning along its normal direction to locate edge points, and then calculates the statistical values ​​of the width (such as the maximum and average values). This method achieves adaptive and high-precision quantification of irregular shapes by ensuring that the measurement direction is always perpendicular to the local growth tangent.

[0068] It should be noted that the aforementioned leaf width calculation process and stem diameter calculation process can be implemented through the leaf extraction model and stem diameter positioning model, respectively.

[0069] In addition, the plant height, spike length, and length of the lower internode of oats can be determined by calculating the coordinates of its first and last key points. and Euclidean distance between Obtain the true length. Compared to only calculating the vertical projection difference ( This method effectively compensates for measurement losses caused by natural plant tilting or shooting angle deviations. For example, when calculating ear length, the coordinates of key points corresponding to the highest and lowest points of the oat ear in the local image are first determined based on the annotation results output by the phenotypic detection model. Then, the Euclidean distance between the coordinates of the key points corresponding to the highest and lowest points of the oat ear is calculated, and the calculation result is determined as the phenotypic detection result corresponding to the ear length of the oat ear.

[0070] In practical applications, since the annotation results output by the phenotypic detection model are rectangular boxes, the highest point and the lowest point of the oat ear in the local image can be the two points connected by any diagonal of the rectangular box. Based on this, the diagonal length of the rectangular box corresponding to the oat ear can be determined as the phenotypic detection result corresponding to the ear length of the oat ear.

[0071] For the number of whorls and the number of spikelets, the corresponding phenotypic detection results can be generated by the whorl recognition model and the spikelet counting model, respectively.

[0072] In one optional implementation, the machine learning-based oat phenotypic detection method further includes: In cases where the key morphological features include blades, a target local image containing blades is determined; In the target local image, polygons are used to perform key point annotation processing to generate key point information corresponding to the blade; The target local image carrying the key point information is input into the leaf extraction model to optimize the leaf extraction model using the target local image carrying the key point information. The leaf extraction model is used to generate phenotypic detection results corresponding to the leaf width.

[0073] Specifically, after phenotypic detection is completed, the whole-plant image to be detected can be used to optimize the phenotypic detection model and its sub-models. Taking the optimization of the leaf extraction model as an example, a target local image containing the leaves can be determined in the whole-plant image to be detected. Then, key point annotation processing is performed on the target local image using polygons to generate key point information corresponding to the leaves. The target local image carrying the key point information is then input into the leaf extraction model to optimize the leaf extraction model using the target local image carrying the key point information.

[0074] Furthermore, to comprehensively evaluate the combined performance of the improved model and the phenotypic analysis algorithm, this specification's embodiments construct an evaluation system from two dimensions: "target detection accuracy" and "phenotypic parameter measurement accuracy." The calculation formulas for each indicator are detailed in Table 1.

[0075] For the feature extraction and localization performance of the five independent deep learning models (phenotypic detection model, whorl number recognition model, spikelet counting model, leaf extraction model, and stem thickness localization model) in the embodiments of this specification, precision (P), recall (R), F1 score, and mean average precision (mAP@0.5) are mainly used as core evaluation metrics. These metrics are calculated based on independent test sets (including standard slice data) for each subtask and aim to evaluate the model's feature capture capability under ideal input conditions.

[0076] (1) Precision: measures the accuracy of the model, that is, the proportion of oat organs (key morphological features) among the targets predicted as positive samples by the model.

[0077] (2) Recall: This measures the model's recall rate, which is the proportion of all real oat organs that are correctly identified by the model. This metric is particularly important for tasks that are prone to missed detection, such as spikelet counting.

[0078] (3) F1 score: the harmonic mean of precision and recall, used to comprehensively evaluate the model’s performance in balancing the two, avoiding the one-sidedness of a single indicator.

[0079] (4) Mean Precision (mAP@0.5): This comprehensively considers the precision performance under different recall thresholds and is an authoritative indicator for measuring the overall robustness of object detection algorithms.

[0080] To verify the comprehensive application effect of the multi-model cascade algorithm in actual field yield measurement, the embodiments in this manual perform regression analysis on the phenotypic prediction values ​​output by the fully automated parsing process and the manually measured values ​​in the field. This evaluation step is based on the original high-resolution whole-plant image and covers the errors of the entire process from whole-image detection and ROI cropping to geometric measurement. The coefficient of determination (R²) is used. 2 The root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are used to quantify the end-to-end measurement accuracy of the system.

[0081] (1) Coefficient of determination (R) 2 ): Characterizes the degree of linear fit between the algorithm's measurement results and manually measured data. The closer the value is to 1, the higher the consistency of the algorithm.

[0082] (2) Root mean square error (RMSE): measures the average deviation between the predicted value and the true value. It is more sensitive to outliers. The smaller the value, the lower the absolute error of the measurement.

[0083] (3) Mean Absolute Error (MAE): Reflects the average level of the absolute error between the predicted value and the observed value, and can more intuitively reflect the magnitude of the actual measurement error.

[0084] (4) Mean Absolute Percentage Error (MAPE): Measures the relative percentage error of a measurement, eliminating the dimensional differences between different traits (such as plant height and stem diameter), and facilitating horizontal comparison between different indicators.

[0085] Table 1. Formulas for evaluating model detection performance and measurement accuracy

[0086] To elucidate the reasoning mechanism of the algorithm and intuitively verify the reliability of phenotypic measurement data, this specification presents an embodiment that constructs a comprehensive visualization verification framework. This framework associates and fuses the "deep features" extracted by deep learning with the "morphological structure parameters" of plant geometric morphology, achieving end-to-end transparent output from image perception to digital quantization.

[0087] To address the high-density distribution and minute-scale characteristics of spikelets in oat ears, this method employs a coarse-to-fine detection strategy. First, image correction is performed on the ROI of the ear, followed by deployment of the improved YOLOv12 model proposed in the embodiments of this specification. Thanks to the introduced Vision Transformer global attention mechanism, the model can effectively capture the long-range dependencies between spikelets and the background. For example... Figure 7 As shown, even in complex scenarios with severe overlapping and occlusion, the algorithm still exhibits robust recall and effectively suppresses the interference of background noise in non-early grains, achieving accurate locking and visual confirmation of the counting target.

[0088] In the whole-plant phenotypic analysis phase, the visualization module overcame the limitations of a single detection box output and established a multi-level visual semantic overlay mechanism to verify the biological accuracy of morphological measurements. For example... Figure 8 As shown, the visualization output contains the following levels: (1) Localization layer (primary semantics): semantic segmentation and localization of stem, leaf, spike and whorl regions are performed by different colored bounding boxes, which intuitively represents the feature capture capability and recall rate of the model in complex occlusion and morphological interlacing scenarios.

[0089] (2) Morphological layer (refined semantics): For key organs such as the flag leaf and the second leaf from the bottom, the algorithm reverse-maps the pixel-level mask and morphological skeleton generated by instance segmentation to the original image (see the red shaded area in the figure). This process confirms that the measurement data is not based on a rough estimate of the rectangular box, but is strictly based on the real biomorphic boundaries of the plant organs.

[0090] (3) Geometric correction layer (spatial quantization): The underlying calculation logic of the plant height and the length of the lower node of the ear marked in the figure has been upgraded to the Euclidean geometric distance algorithm based on key points. The visual projection error caused by the tilt of the plant posture is effectively avoided by visual connection.

[0091] Regarding the automated annotation and data traceability system for phenotypic parameters: (1) Parameter listing: The algorithm renders the 10 key phenotypic indicators (including number of whorls, number of spikelets, spike length, length of the lower node of the spike, length / width of the flag leaf, length / width of the second leaf from the top, stem diameter, etc.) in real time as highlighted text on the left side of the image. Among them, the length and width data are accurate to four decimal places (pixel unit), which provides a high-precision quantitative basis for subsequent genetic analysis.

[0092] (2) Visual verification: The analysis results of each sample are archived as high-resolution images with detailed morphological annotations. In the manual verification stage, researchers can directly check the fit of the detection box and the fit of the skeleton line to quickly locate and remove outliers. This visual data storage method greatly improves the credibility and scientific research reuse value of the phenotypic dataset.

[0093] In addition, to verify the effectiveness of the "multi-model cascaded architecture" proposed in the embodiments of this specification, the core models for whole-plant morphology, spikelet counting, and whorl identification, which have been trained, were visually evaluated. By analyzing the convergence trend of the loss function, the precision-recall curve, and the confusion matrix during the training process, the robustness of the algorithm was verified from multiple dimensions.

[0094] To monitor the model's learning dynamics and generalization performance, the most representative whole-plant morphology model was selected for a full-cycle (2000 epochs) training process analysis. For example... Figure 9 As shown in the chart, the left side displays the changes in the loss function, while the right side shows the improving trend of key accuracy metrics.

[0095] (1) Loss function convergence analysis: The bounding box loss (box_loss) and classification loss (cls_loss) show a steep downward trend in the early stage of training (first 100 epochs), indicating that the model is rapidly learning feature extraction and localization rules; subsequently, the curves gradually flatten and converge to extremely low steady-state values. It is worth noting that the validation set loss (Val Loss) curve closely follows the training set curve and does not rebound (i.e., the validation error increases) during long-term training, which strongly proves that the model has a very strong generalization ability and no overfitting phenomenon has occurred.

[0096] (2) Analysis of the evolution of accuracy indicators: The evaluation indicators on the right side of the chart show that the mean accuracy (mAP@0.5) and precision rapidly increased and stabilized at a high level close to 1.0 in the early stage of training. This indicates that the whole-plant model has very small localization deviations in plant height, stem and leaf and ear regions, and has the ability to perform high-confidence detection in complex backgrounds.

[0097] To systematically evaluate the gains in model feature representation capabilities made by introducing a lightweight Vision Transformer (ViT) module at the end of the backbone network, an ablation experiment was designed in the embodiments of this specification. Given that local tasks such as spikelet counting primarily rely on texture features and have limited receptive field requirements, the global modeling advantages of ViT are difficult to highlight. Therefore, this experiment specifically selects the most challenging whole-plant morphology detection as the validation scenario, aiming to focus on evaluating the core contributions of the improved module in handling long-distance semantic dependencies, complex occlusion, and large-scale geometric deformations.

[0098] (1) Overall performance evaluation: Under unified training hyperparameters and dataset partitioning, the performance of the model after introducing the Lite-ViT module was compared with that of the original YOLOv12 as the baseline (Table 2). Experimental results show that the improved YOLOv12+ViT model successfully broke the constraint between accuracy and efficiency. Specifically, the improved model achieved a significant leap in detection accuracy while reducing the number of parameters by 4.5 M (approximately 7.6%): the mean accuracy (mAP@0.5) increased by 2.0% to 98.2%. More importantly, the mAP@50-95 index, which reflects the high-precision bounding box regression capability, was significantly improved by 2.8% to 80.6%. This confirms that the introduction of the ViT module not only improved the recall capability of the target (Recall increased by 2.2%), but also significantly optimized the localization accuracy of the bounding box (Precision increased by 3.1%), achieving the best balance between lightweight model and high performance.

[0099] Table 2. Comparison of Model Performance Before and After Improvement: Experimental Results

[0100] (2) Analysis of detection accuracy of each phenotypic trait: In order to explore in depth the adaptability of the improved module to different agronomic traits, especially the ability to mine "difficult samples", the embodiments of this manual have conducted a comparative analysis of the detection accuracy of each sub-category (Table 3). The results show that the improved model exhibits differentiated advantages in tasks of different difficulty: As can be seen from the sub-results in Table 3, the improved model exhibits excellent adaptability and differentiated advantages in phenotypic tasks of different difficulty. First, when dealing with simple targets with significant and rigid features (such as plant height, ear length and the first / second leaf from the top), the improved model continues the high accuracy performance of the benchmark model, with mAP@0.5 consistently above 99%, which confirms that the lightweight ViT module, while introducing global context, does not introduce noise or weaken the ability to extract basic morphological features, and maintains the robustness of the model to simple samples. Crucially, for the leaf, which suffers the most severe shading and greatest flexible deformation in the field, the model, thanks to the global receptive field provided by the Lite-ViT module, effectively utilizes contextual information to infer the shading area. This resulted in significant increases in leaf mAP@0.5 and mAP@50-95 of 10.0% and 8.7%, respectively, successfully overcoming the bottleneck of missed detections in high-density planting environments. Furthermore, in the detection of the small target "internode length," although mAP@0.5 showed slight fluctuations, the more stringent mAP@50-95 increased by 4.0% (from 61.3% to 65.3%). This phenomenon indicates that the improved model's bounding box regression for slender targets is more compact, significantly reducing measurement errors caused by bounding box drift, thereby effectively improving the reliability of phenotypic parameter measurements.

[0101] In summary, the introduction of the lightweight ViT module not only maintains excellent stability in simple tasks, but also fundamentally solves the limitations of traditional CNNs in feature extraction for severe occlusion and slender target detection by enhancing global context modeling capabilities. This fully verifies the effectiveness and robustness of the strategy in complex field phenotypic analysis.

[0102] Table 3 Comparison of detection accuracy for different phenotypic traits

[0103] To verify the independent performance of each functional module in the hierarchical cascaded architecture, the embodiments in this specification quantitatively evaluate the whole-plant localization and its three downstream core sub-models (Table 4). The experiments focused on examining the adaptability of different types of Region of Interest (ROI) extraction strategies and model architectures to specific phenotypic parsing tasks, and the sources of detection errors were thoroughly investigated using confusion matrices and PR curves. The results show that each cascaded module exhibits excellent robustness in specific tasks; (1) Macroscopic localization layer (whole plant detection): As the entry point of the cascade system, the whole plant detection model (Plant-Det) exhibits extremely high localization accuracy after improvement, with mAP@0.5 reaching 98.2% and high threshold accuracy mAP@50-95 reaching as high as 80.6%. This high-precision bounding box regression ensures the accuracy of the downstream task's field of view and effectively avoids the accumulation of cascade errors caused by localization deviation. In order to analyze the semantic recognition ability of this model for various anatomical structures, this study generates a normalized confusion matrix based on the improved ViT model. Matrix analysis shows that the Lite-ViT module's global feature extraction capability significantly enhances the model's discrimination boundary for similar categories: the model not only achieved a perfect accuracy of 1.00 in recognizing "plant height" and "first / second leaf from the bottom," successfully overcoming the interference of biological similarity between "flag leaf / second leaf from the bottom" and "ordinary leaf," achieving zero misclassification; simultaneously, in recognizing "internode length below the ear," which is easily confused with leaf sheath texture, the model maintained a high confidence level of 0.91, and the recognition accuracy for "ordinary leaf" also reached 0.95 (with only a 0.04 probability of misclassification as background). Overall, the extremely low values ​​in the off-diagonal region demonstrate that the whole-plant model has clear and robust feature discrimination capabilities in complex field backgrounds.

[0104] (2) Microscopic Analysis Layer (Sub-organ Analysis): For microscopic sub-tasks, differentiated model strategies significantly improved the detection ceiling. Specifically, for irregularly shaped leaves, the introduction of the instance segmentation architecture not only increased the Mask mAP@0.5 to 94.3%, but also generated a pixel-level mask that closely fits the leaf edge. At the same time, facing the weak round number detection of stem node features, the model achieved 78.1% mAP@0.5, successfully capturing small stem node targets effectively. In the most challenging spikelet counting task, thanks to the clean input provided by the background threshold segmentation strategy, the model achieved a high-precision detection rate of 95.5%. Figure 10The precision-recall curve (PR curve) exhibits an extremely strong "right-angled" trend, closely following the ideal observation point (1, 1) in the upper right corner. The area under the curve (mAP@0.5) is as high as 0.955, indicating that the model can maintain a very high precision close to 100% even within a wide range of recalls as high as 0.9. Even for high-density, easily occluded spikelet targets, the algorithm effectively overcomes the contradiction between "false positives" and "false negatives," demonstrating extremely strong anti-interference capabilities.

[0105] Table 4 shows the performance evaluation results of each subtask model in the cascaded framework.

[0106] To apply the "multi-model cascaded detection algorithm" constructed in the embodiments of this specification to practice and solve the problems of cumbersome data recording and bulky equipment in traditional field surveys, the embodiments of this specification designed and developed a mobile-based intelligent analysis system for oat phenotypic characteristics (OatPheno App). This system aims to provide breeding experts and researchers with a portable phenotypic analysis tool that allows for "what you see is what you get".

[0107] The system adopts a modular design concept and integrates two core functional modules: "intelligent recognition" and "field survey".

[0108] (1) Multi-mode recognition engine: In order to better adapt to the complex environmental conditions in the field (such as shading and changes in light) and meet the diverse needs of users, the system breaks through the limitations of a single detection mode and designs a flexible hierarchical function matrix. For indoor scenes with relatively simple backgrounds, the system provides a "whole plant detection" mode, which can call the entire model to extract 10 core indicators such as plant height, ear length, and number of whorls at once; while for the local observation needs in the later stage of field growth, the system further splits into independent organ-level detection entry points such as "ear detection", "leaf detection", "stem detection" and "ear number recognition". This design allows users to focus on specific organs (such as taking pictures of complex ears or specific leaves), which can not only effectively reduce the interference of complex background noise, but also greatly improve the flexibility and usability of the system in actual field application scenarios.

[0109] (2) Digital Field Survey Book: The bottom navigation bar integrates the "Survey" module, which is designed to replace the traditional paper record book. This module allows users to create electronic files for different fields and automatically fills the corresponding files with the phenotypic data obtained by intelligent identification, thereby achieving seamless connection between data collection and storage.

[0110] The system integrates a lightweight model inference engine, enabling rapid online analysis of images captured or uploaded by users.

[0111] (1) Image interaction: Users can obtain images through "on-site shooting" or "local upload".

[0112] (2) Structured Display of Results: After recognition, the system not only draws a visual detection box on the original image, but also transforms the unstructured image information into a structured list of phenotypic data. The spike length (15 cm), number of whorls (8), number of spikelets (32), and leaf length and width data are all automatically calculated and clearly listed. The establishment of this system framework has initially verified the feasibility of deploying the algorithm on mobile devices, laying a solid technical foundation for the future realization of an automated closed loop from "pixel input" to "agronomic indicator output".

[0113] This specification addresses the challenges of the large scale span and complex geometric morphology of oat phenotypic features by constructing a hierarchical multi-scale dataset of "macroscopic localization-microscopic analysis." It also innovatively proposes a phenotypic analysis algorithm based on the deep integration of the YOLOv11 deep learning framework and traditional computer vision techniques. By designing a cascaded architecture consisting of a global localization model and a local fine-grained analysis model, combined with ROI adaptive pruning and hierarchical non-maximum suppression strategies, the algorithm effectively solves the feature attenuation and missed detection problems of small targets (such as spikelets and whorls) in a panoramic field of view, establishing a hierarchical processing strategy of "macroscopic localization guidance-microscopic fine-grained analysis."

[0114] Regarding the accuracy of phenotypic parameter quantification, the embodiments in this specification overcome the limitations of traditional object detection that relies solely on bounding box geometric size estimation, proposing a hybrid algorithm architecture of "deep learning cascaded detection + traditional computer vision fine measurement." By introducing morphological skeleton extraction and Euclidean geometric distance calculation methods, pixel-level accurate measurement of morphological indicators such as flag leaf, second leaf from the top, and stem thickness is achieved, effectively correcting visual projection errors caused by plant posture tilt or shooting angle deviations. Experimental results show that the algorithm exhibits extremely high robustness in spikelet counting tasks under densely occluded scenes, with a validation set average accuracy (mAP@0.5) as high as 0.955. Furthermore, the full-element visualization verification framework constructed in the embodiments of this specification achieves full-process transparency and verifiability from image perception to the final digital result.

[0115] To promote the practical application of research findings, this specification describes and develops a mobile-based intelligent oat phenotyping system (OatPheno App). This system integrates a lightweight multi-modal recognition model with a digital field survey module, achieving end-to-end automation from image acquisition and automatic analysis to data archiving. For complex field scenarios, the system is designed with tiered recognition modes of "whole plant detection" and "organ-specific detection," significantly improving its flexibility and adaptability in practical use. This tool provides breeders with convenient and efficient "one-click acquisition and instant analysis" digital support, initially solving the problems of cumbersome manual recording and inconvenient equipment carrying in traditional field surveys.

[0116] In summary, the embodiments of this specification acquire a whole-plant image of an oat plant to be detected; annotate multiple key morphological features of the oat plant in the whole-plant image using a phenotypic detection model; crop the whole-plant image based on the generated annotation results to obtain a local image containing key morphological features, and extract the skeleton point set of the oat plant based on the local image; perform phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, spike length, length of the internode below the spike, leaf width, number of whorls, number of spikelets, and stem diameter. The embodiments of this specification effectively enhance the global perception and long-distance feature capture capabilities of the phenotypic detection model in complex occlusion environments. Therefore, using the phenotypic detection model to annotate key morphological features in the whole-plant image to be detected can effectively capture the long-distance dependency between key morphological features and the background, thereby ensuring the positioning accuracy of each key morphological feature. Subsequent phenotypic calculation based on the annotation results of each key morphological feature can achieve accurate locking and visual confirmation of the counting target. This processing method effectively reduces the false negative rate, thereby improving the accuracy of the detection results.

[0117] Corresponding to the above method embodiments, this specification also provides embodiments of an oat phenotyping system based on machine learning. Figure 11 A schematic diagram of a machine learning-based oat phenotyping system according to an embodiment of this specification is shown. Figure 11 As shown, the system includes: The acquisition module 1102 is configured to acquire a whole-plant image of an oat plant to be detected; The annotation module 1104 is configured to annotate multiple key morphological features of the oat plant in the whole plant image to be detected using a phenotypic detection whole plant model. The extraction module 1106 is configured to crop the whole plant image to be detected based on the generated annotation results to obtain a local image containing key morphological features, and extract the skeleton point set of the oat plant based on the local image. The generation module 1108 is configured to perform phenotypic calculations based on the skeleton point set and the annotation results, and generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, ear length, length of the lower internode, leaf width, number of whorls, number of spikelets, and stem diameter.

[0118] Optionally, the extraction module 1106 is further configured to: The local image is processed to generate a corresponding mask image, wherein the mask image contains a target mask for describing key morphological features; The target mask in the mask image is iteratively eroded using a thinning algorithm to generate the topological center line of the oat plant's leaves. Key points are extracted from the topological centerline, and these extracted key points are used as the skeleton point set of the oat plant's leaves.

[0119] Optionally, the extraction module 1106 is further configured to: Adaptive threshold segmentation is performed on the local image to generate an intermediate image containing the threshold segmentation results; Based on the threshold segmentation result, morphological closing operation is performed on the intermediate image to generate the corresponding mask image.

[0120] Optionally, the generation module 1108 is further configured to: In the case where the key morphological feature is an oat ear, the coordinates of the key points corresponding to the highest and lowest points of the oat ear in the local image are determined based on the annotation results. Calculate the Euclidean distance between the coordinates of the key points corresponding to the highest and lowest points of the oat ear, and determine the calculation result as the phenotypic detection result corresponding to the ear length of the oat ear.

[0121] Optionally, the annotation result includes a rectangular box; Accordingly, the generation module 1108 is further configured to: When the key morphological feature is an oat ear, the diagonal length of the rectangle corresponding to the oat ear is determined as the phenotypic detection result corresponding to the ear length of the oat ear.

[0122] Optionally, the generation module 1108 is further configured to: When the key morphological feature is a leaf, the least squares method is used to fit the skeleton point set of the leaf to generate the main direction vector corresponding to the leaf, wherein the main direction vector is used to characterize the growth direction of the leaf. Multiple sets of tangents are generated along the normal direction of the main direction vector, and the phenotypic detection result corresponding to the blade width is determined based on the distance between the intersection point of each set of tangents and the edge contour of the blade.

[0123] Optionally, the machine learning-based oat phenotyping system further includes a processing module configured to: In cases where the key morphological features include blades, a target local image containing blades is determined; In the target local image, polygons are used to perform key point annotation processing to generate key point information corresponding to the blade; The target local image carrying the key point information is input into the leaf extraction model to optimize the leaf extraction model using the target local image carrying the key point information. The leaf extraction model is used to generate phenotypic detection results corresponding to the leaf width.

[0124] The above is an illustrative scheme of an oat phenotypic detection system based on machine learning according to this embodiment. It should be noted that the technical solution of this oat phenotypic detection system based on machine learning belongs to the same concept as the technical solution of the oat phenotypic detection method based on machine learning described above. Details not described in detail in the technical solution of the oat phenotypic detection system based on machine learning can be found in the description of the technical solution of the oat phenotypic detection method based on machine learning described above.

[0125] Figure 12 A structural block diagram of a computing device 1200 according to an embodiment of this specification is shown. The components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and a database 1250 is used to store data.

[0126] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0127] In one embodiment of this specification, the aforementioned components of the computing device 1200 and Figure 12 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 12The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0128] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 1200 can also be a mobile or stationary server.

[0129] The processor 1220 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described oat phenotype detection method based on machine learning.

[0130] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the oat phenotypic detection method based on machine learning described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the oat phenotypic detection method based on machine learning described above.

[0131] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described oat phenotypic detection method based on machine learning.

[0132] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the oat phenotypic detection method based on machine learning described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the oat phenotypic detection method based on machine learning described above.

[0133] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described oat phenotypic detection method based on machine learning.

[0134] The above is an illustrative example of a computer program in this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the aforementioned oat phenotypic detection method based on machine learning. Details not described in detail in the computer program's technical solution can be found in the description of the aforementioned oat phenotypic detection method based on machine learning.

[0135] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0136] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0137] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A machine learning-based method for detecting oat phenotypes, comprising: Obtain whole-plant images of oat plants to be tested; The phenotypic detection model is used to annotate multiple key morphological features of the oat plant in the whole plant image to be detected. The whole plant image to be detected is cropped based on the generated annotation results to obtain a local image containing key morphological features, and the skeleton point set of the oat plant is extracted based on the local image. Phenotypic calculation is performed based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results; The phenotypes include plant height, ear length, length of the internode below the ear, leaf width, number of whorls, number of spikelets, and stem diameter.

2. The oat phenotypic detection method based on machine learning according to claim 1, wherein extracting the skeleton point set of the oat plant based on the local image includes: The local image is processed to generate a corresponding mask image, wherein the mask image contains a target mask for describing key morphological features; The target mask in the mask image is iteratively eroded using a thinning algorithm to generate the topological center line of the oat plant's leaves. Key points are extracted from the topological centerline, and these extracted key points are used as the skeleton point set of the oat plant's leaves.

3. The oat phenotypic detection method based on machine learning according to claim 2, wherein processing the local image to generate a corresponding mask image includes: Adaptive threshold segmentation is performed on the local image to generate an intermediate image containing the threshold segmentation results; Based on the threshold segmentation result, morphological closing operation is performed on the intermediate image to generate the corresponding mask image.

4. The oat phenotypic detection method based on machine learning according to claim 1, wherein the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: In the case where the key morphological feature is an oat ear, the coordinates of the key points corresponding to the highest and lowest points of the oat ear in the local image are determined based on the annotation results. Calculate the Euclidean distance between the coordinates of the key points corresponding to the highest and lowest points of the oat ear, and determine the calculation result as the phenotypic detection result corresponding to the ear length of the oat ear.

5. The oat phenotypic detection method based on machine learning according to claim 2, wherein the annotation result includes a rectangular bounding box; Accordingly, the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: When the key morphological feature is an oat ear, the diagonal length of the rectangle corresponding to the oat ear is determined as the phenotypic detection result corresponding to the ear length of the oat ear.

6. The oat phenotypic detection method based on machine learning according to claim 2 or 5, wherein the step of performing phenotypic calculation based on the skeleton point set and the annotation results to generate corresponding phenotypic detection results includes: When the key morphological feature is a leaf, the least squares method is used to fit the skeleton point set of the leaf to generate the main direction vector corresponding to the leaf, wherein the main direction vector is used to characterize the growth direction of the leaf. Multiple sets of tangents are generated along the normal direction of the main direction vector, and the phenotypic detection result corresponding to the blade width is determined based on the distance between the intersection point of each set of tangents and the edge contour of the blade.

7. The oat phenotypic detection method based on machine learning according to claim 1, further comprising: In cases where the key morphological features include blades, a target local image containing blades is determined; In the target local image, polygons are used to perform key point annotation processing to generate key point information corresponding to the blade; The target local image carrying the key point information is input into the leaf extraction model to optimize the leaf extraction model using the target local image carrying the key point information. The leaf extraction model is used to generate phenotypic detection results corresponding to the leaf width.

8. A machine learning-based oat phenotyping system, comprising: The acquisition module is configured to acquire images of the entire oat plant to be detected; The annotation module is configured to annotate multiple key morphological features of the oat plant in the whole-plant image to be detected using a phenotypic detection model. The extraction module is configured to crop the whole plant image to be detected based on the generated annotation results to obtain a local image containing key morphological features, and extract the skeleton point set of the oat plant based on the local image; The generation module is configured to perform phenotypic calculations based on the skeleton point set and the annotation results, and generate corresponding phenotypic detection results, wherein the phenotypic includes plant height, ear length, length of the lower internode, leaf width, number of whorls, number of spikelets, and stem diameter.

9. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the oat phenotypic detection method based on machine learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the oat phenotypic detection method based on machine learning as described in any one of claims 1 to 7.

Images