Method and device for measuring wheat ear phenotype traits in field environment

Abstract

The application discloses a kind of field environment under ear phenotype character measurement method and device, wherein the method includes: the ear image in field environment is obtained by mobile terminal;Ear image includes specified size scale, the actual size of each pixel in ear image is determined using specified size scale;The actual length and actual width of ear are determined by the actual size of each pixel and the pixel feature of ear in image, and the number of ear grains is determined by deep learning, simultaneously, according to the first distance interval between the points of each other after the center point of ear grain is projected to the vertical center line of image, and the second distance interval between the intersection points of each other of preset diagonal line through the center point of ear grain and vertical center line of image, the number of spikelets is determined.The application can obtain ear image without damaging the growth of wheat, determine more ear phenotype characters according to ear image, and improve the measurement accuracy of ear phenotype characters.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a method and device for measuring phenotypic traits of wheat ears in a field environment. BACKGROUND

[0002] This section is intended to provide background information to facilitate an understanding of embodiments of the application as set forth in the claims. The description herein does not constitute admission of prior art.

[0003] Wheat yield is a complex trait, significantly affected by genetic and environmental factors, with a low heritability. Therefore, it is usually genetically decomposed into three yield components, including spike number per unit area, grain number per spike, and grain weight. Among them, grain number per spike is considered to have a high heritability, and it is generally believed that improving the grain number per spike of wheat is very important for increasing wheat yield. The grain number per spike is largely affected by the inflorescence structure. A typical wheat ear is usually composed of a main axis called rachis and several sessile spikelets directly attached to the rachis, resulting in an unbranched ear structure. Most wheat varieties develop one spikelet on each rachis node and have multiple florets. Generally, only 1-3 florets can become fertile florets and develop into grains. Improvement in the number of spikelets and florets (floret primordia and fertile florets) makes a significant contribution to the increase in grain number per spike. The components of the spike inflorescence (e.g., spikelets, which are the basic units of the inflorescence, contain florets, floret primordia, and lemma) also affect each other. The number and arrangement of these spike components affect ear length, ear weight, chaff (i.e., non-grain biomass in the spike), spike grain number, spike grain weight, and spike number, all of which contribute to the final grain yield per ear. In addition, through the number of spikelets, fertile florets, and grains, the fertility of spikelets, the proportion of fertile florets, and the proportion of grains / fertile florets can be calculated to further evaluate the characteristics of the ear. In order to produce varieties with the highest efficient grain production in different environments, it is necessary to know how to predict and manipulate the spike morphology of wheat. Therefore, it is of great significance to count the number of spike shape, spikelets, and florets (grains) in the breeding process for screening high-yield wheat varieties.

[0004] The existing measurement method of ear phenotypic traits still relies on manual destructive sampling observation, which is time-consuming and labor-intensive. In recent years, there have been many studies on using image processing methods to obtain ear traits. These methods also require destructive sampling, and then images are obtained by shooting in the laboratory for measurement. This is extremely disadvantageous for small sample size breeding plots, and usually only a certain phenotypic trait of the wheat ear is measured, which is low in efficiency. For the measurement of the number of spikelets, previous studies have used wheat ear side images for detection, which is difficult to obtain images and has low measurement efficiency and poor detection accuracy. SUMMARY

[0005] The embodiment of the present application provides a wheat ear phenotype trait measurement method in a field environment, which is used for obtaining a wheat ear image without damaging the growth of wheat, determining more wheat ear phenotype traits according to the wheat ear image, and improving the measurement accuracy of the wheat ear phenotype traits.

[0006] The wheat ear image in the field environment is obtained through the mobile terminal; the wheat ear image includes a specified size scale, and the specified size scale is used for determining the size and the position of the wheat ear in the wheat ear image.

[0007] The image segmentation and extraction of the wheat ear and the specified size scale are performed on the wheat ear image, so that the image of the wheat ear and the specified size scale is obtained.

[0008] The actual size represented by each pixel in the wheat ear image is determined according to the image of the specified size scale.

[0009] The number of grains in the image of the wheat ear is identified through the trained deep learning model; and each grain is marked by a rectangular frame.

[0010] The minimum circumscribed rectangle with the wheat ear in the middle is determined through the minimum circumscribed rectangle method, and the actual length and the actual width of the wheat ear are determined according to the size of the minimum circumscribed rectangle and the actual size represented by each pixel.

[0011] The number of spikelets is determined according to the first distance interval between the points obtained by projecting the center points of the grains to the vertical center line of the image and the second distance interval between the intersection points of the preset oblique lines passing through the center points of the grains and the vertical center line of the image, wherein the center points of the grains are the centers of the rectangular frames, and each oblique line passes through a center point of a grain.

[0012] In an embodiment, the wheat ear image in the field environment is obtained through the mobile terminal, including: obtaining, through the mobile terminal, an imaging image of the wheat ear in the field environment placed in front of a preset background plate; and the preset background plate is marked with a specified size scale and a region for placing the photographed wheat ear.

[0013] In an embodiment, the actual size represented by each pixel in the wheat ear image is determined according to the image of the specified size scale, including: performing local adaptive thresholding, binarization, closing operation and filling operation on the image of the specified size scale to obtain a binarized image of the scale; determining the area of the scale in the image according to the binarized image of the scale; and determining the actual size represented by each pixel according to the area of the scale in the image and the actual area of the scale.

[0014] In an embodiment, the actual size represented by each pixel is determined according to the area of the scale in the image and the actual area of the scale, including: determining the actual size represented by each pixel according to the area of the scale in the image and the actual area of the scale according to the following formula:

[0015]

[0016] where SSOP is the actual size represented by each pixel, is the area of the ruler in the image, is the actual area of the ruler.

[0017] In an embodiment, the image segmentation and extraction of the wheat ear and the specified size ruler of the wheat ear image are performed to obtain the image of the wheat ear and the specified size ruler, including: performing image segmentation and extraction of the wheat ear and the specified size ruler of a plurality of wheat ear images by using a trained YOLOv8n model to obtain a plurality of images of the wheat ear and the specified size ruler; and the YOLOv8n model is trained by using historical wheat ear images, wheat ears extracted from the historical wheat ear images, and specified size rulers extracted from the historical wheat ear images.

[0018] In an embodiment, the deep learning model is a YOLOv8n model.

[0019] The image of the wheat ear is obtained by using a trained deep learning model to identify the number of grains in the image of the wheat ear, including: performing grain identification on a plurality of wheat ear images by using a trained YOLOv8n model to obtain the number of grains in the image of the wheat ear; and the YOLOv8n model is trained by using historical wheat ear images and grains marked in the historical wheat ear images.

[0020] In an embodiment, the smallest circumscribed rectangle with the wheat ear in the middle is determined by a smallest circumscribed rectangle method, and the actual length and the actual width of the wheat ear are determined according to the size of the smallest circumscribed rectangle and the actual size represented by each pixel, including: determining the smallest circumscribed rectangle with the wheat ear in the middle by the smallest circumscribed rectangle method; rotating the wheat ear and the smallest circumscribed rectangle to be vertical based on the inclination angle of the smallest circumscribed rectangle to obtain a vertical smallest circumscribed rectangle; determining the length and the width of the wheat ear in the image based on the pixel positions of the four vertices of the vertical smallest circumscribed rectangle; and determining the actual length and the actual width of the wheat ear according to the actual size represented by each pixel in the wheat ear image, the length and the width of the wheat ear in the image.

[0021] In an embodiment, the wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center points of the grains on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center points of the grains on the right side of the vertical center line of the image is negative.

[0022] The number of spikelets is determined according to the distance interval between the points obtained by projecting the center points of the spike grains to the vertical center line of the image and the distance interval between the intersection points of the preset inclined lines passing through the center points of the spike grains and the vertical center line of the image, and comprises: determining the length of each spike grain by using the rectangular frame mark of each spike grain through the minimum circumscribed rectangle method; determining the average length of the spike grains according to the length of each spike grain; determining a threshold value according to the average length of the spike grains; the threshold value is used to classify the spike grains into spikelets; two spike grains corresponding to a first distance interval smaller than the threshold value are classified into the same spikelet to obtain a first classification result; two spike grains corresponding to a second distance interval smaller than the threshold value are classified into the same spikelet to obtain a second classification result; the first classification result and the second classification result both represent the classification of the spike grains belonging to the same spikelet; and the first classification result and the second classification result are fused to obtain a final classification result; the final classification result comprises a plurality of classifications, and each classification comprises the spike grains belonging to the same spikelet.

[0023] The embodiment of the present application also provides a wheat ear phenotype trait measurement device in a field environment, which is used to obtain a wheat ear image without damaging the growth of wheat, determine more wheat ear phenotype traits according to the wheat ear image, and improve the measurement accuracy of the wheat ear phenotype traits.

[0024] A data acquisition module is configured to acquire the wheat ear image in the field environment through a mobile terminal; the wheat ear image comprises a specified size scale, and the specified size scale is used to determine the size and position of the wheat ear in the wheat ear image.

[0025] An image extraction module is configured to perform image segmentation and extraction of the wheat ear and the specified size scale in the wheat ear image, and obtain the images of the wheat ear and the specified size scale.

[0026] A pixel actual size determination module is configured to determine the actual size of each pixel in the wheat ear image according to the image of the specified size scale.

[0027] A spike grain number determination module is configured to identify the number of spike grains in the image of the wheat ear through a trained deep learning model; each spike grain is marked by a rectangular frame.

[0028] A wheat ear length and width determination module is configured to determine the minimum circumscribed rectangle with the wheat ear in the middle through the minimum circumscribed rectangle method, and determine the actual length and actual width of the wheat ear according to the size of the minimum circumscribed rectangle.

[0029] A spikelet number determination module is configured to determine the number of spikelets according to the first distance interval between the points obtained by projecting the center points of the spike grains to the vertical center line of the image and the second distance interval between the intersection points of the preset inclined lines passing through the center points of the spike grains and the vertical center line of the image; the center points of the spike grains are the centers of the rectangular frames; and each inclined line passes through a center point of a spike grain.

[0030] The embodiment of the present application also provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the ear phenotype trait measurement method in a field environment when executing the computer program.

[0031] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the ear phenotype trait measurement method in a field environment when executed by a processor.

[0032] The embodiment of the present application also provides a computer program product, which comprises a computer program, and the computer program implements the ear phenotype trait measurement method in a field environment when executed by a processor.

[0033] In the embodiment of the present application, the ear image in the field environment is acquired by the mobile terminal, the ear image of the wheat is acquired without damaging the growth of the wheat, the specified size scale is included in the ear image, the actual size represented by each pixel in the ear image is determined by using the specified size scale, the actual length and the actual width of the ear are determined by the actual size represented by each pixel and the pixel characteristics of the ear in the image, the number of spikelets is determined by deep learning, and compared with the prior art, the method in the embodiment of the present application can combine deep learning and image processing, more ear phenotype traits in the ear image are acquired, and the accuracy of the ear phenotype trait measurement is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0035] Figure 1 It is a flowchart of the ear phenotype trait measurement method in the field environment in the embodiment of the present application;

[0036] Figure 2 It is a schematic diagram of the preset background plate in the embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of the deep learning model identifying the ear and extraction in the embodiment of the present application;

[0038] Figure 4 Figure 1 is a schematic diagram of a rectangular scale image processing in an embodiment of the present application;

[0039] Figure 5 Figure 2 is a schematic diagram of a center point of a rectangular frame mark of a spikelet in an embodiment of the present application;

[0040] Figure 6 Figure 3 is a schematic diagram of spikelet number detection in an embodiment of the present application;

[0041] Figure 7 Figure 4 is a schematic diagram of a wheat spike phenotype measurement device in a field environment in an embodiment of the present application. DETAILED DESCRIPTION

[0042] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, further detailed description will be made to the embodiments of the present application in combination with the drawings. Herein, the schematic embodiments of the present application and the description thereof are used to explain the present application, but not as a limitation to the present application.

[0043] Figure 1 Figure 5 is a flowchart of a wheat spike phenotype measurement method in a field environment in an embodiment of the present application, as shown in the figure, the method comprises: Figure 1

[0044] Step 101, obtaining a wheat spike image in a field environment by a mobile terminal; the wheat spike image comprises a specified size scale, the specified size scale is used to determine the size and the position of the wheat spike in the wheat spike image;

[0045] Step 102, performing image segmentation and extraction of the wheat spike and the specified size scale of the wheat spike image, to obtain the image of the wheat spike and the specified size scale;

[0046] Step 103, determining the actual size represented by each pixel in the wheat spike image according to the image of the specified size scale;

[0047] Step 104, identifying the number of spikelets in the image of the wheat spike by a trained deep learning model; wherein each spikelet is marked by a rectangular frame;

[0048] Step 105, determining the minimum circumscribed rectangle with the wheat spike in the middle by a minimum circumscribed rectangle method, and determining the actual length and the actual width of the wheat spike according to the size of the minimum circumscribed rectangle and the actual size represented by each pixel;

[0049] Step 106, determining the number of spikelets according to the first distance interval between the points after the center points of the spikelets are projected to the vertical center line of the image, and the second distance interval between the intersection points of the slant lines passing through the center points of the spikelets and the vertical center line of the image; the center points of the spikelets are the centers of the rectangular frame marks; each slant line passes through a center point of a spikelet.

[0050] From​Figure 1 As can be seen from the flow chart, in the embodiment of the present application, the ear image of wheat in the field environment is acquired by the mobile terminal, so that the ear image of wheat is acquired without damaging the growth of wheat, the specified size scale is included in the ear image of wheat, then the actual size represented by each pixel in the ear image of wheat is determined by using the specified size scale, so that the actual length and the actual width of the ear of wheat are determined by the actual size represented by each pixel and the pixel feature of the ear of wheat in the image, and the number of spikelets of the ear of wheat is determined by deep learning, and compared with the prior art, the method proposed in the embodiment of the present application can combine deep learning and image processing, and more phenotypic traits of the ear of wheat in the ear image are acquired, so that the accuracy of the measurement of the phenotypic traits of the ear of wheat is effectively improved.

[0051] The method for measuring the phenotypic traits of the ear of wheat in the field environment will be explained in detail below. Figure 1 The method for measuring the phenotypic traits of the ear of wheat in the field environment will be explained in detail below.

[0052] Firstly, the ear of wheat is imaged without damaging the growth of wheat in the field. The growing ear of wheat is directly photographed by using a mobile terminal such as a mobile phone or a tablet computer.

[0053] The ear image of wheat photographed in the embodiment of the present application includes a specified size scale, which is used to determine the size and the position of the ear of wheat in the ear image.

[0054] In an embodiment, the ear image of wheat in the field environment is acquired by the mobile terminal, which can include: acquiring the imaging image of the ear of wheat in the field environment by the mobile terminal before the ear of wheat is placed in front of a preset background plate; the preset background plate is marked with a specified size scale and a region for placing the photographed ear of wheat.

[0055] Figure 2 The schematic diagram of the preset background plate in the embodiment of the present application is shown in the following figure, Figure 2 The size of the preset background plate in the embodiment of the present application is , which includes an image acquisition region , a rectangular scale , and a thumb pressing region. When photographing, the whole ear of wheat in the field is placed in the image acquisition region of the preset background plate, the lower stem of the ear of wheat is gently pressed by the thumb to fix the ear of wheat, and the ear of wheat in the image acquisition region is acquired by using the smart phone.

[0056] When photographing, a large number of ear image samples of wheat in the field can be acquired by the mobile terminal, which are then used for model training and measurement of the phenotypic traits of the ear of wheat.

[0057] In step 102, image segmentation of the wheat ear image and the specified size scale is performed to obtain the wheat ear image and the specified size scale image.

[0058] The wheat ear image and the specified size scale image are extracted, for example, by using conventional image processing techniques.

[0059] In order to improve the accuracy of image extraction, in an embodiment, the image segmentation of the wheat ear image and the specified size scale is performed to obtain the wheat ear image and the specified size scale image, including: using a trained YOLOv8n model to perform image extraction of the wheat ear and the specified size scale on a plurality of wheat ear images to obtain a plurality of wheat ear and specified size scale images; the YOLOv8n model is pre-trained using historical wheat ear images, extracted wheat ears in the historical wheat ear images, and extracted specified size scales in the historical wheat ear images to obtain the YOLOv8n model.

[0060] In implementation, a large number of wheat ear image samples are collected by different models of smart phones to construct a training set, a test set and a validation set, and a YOLOv8n model architecture is built, and the YOLOv8n model is trained using the training set. When training the model, Labelme is used to mark the rectangular frame of the image, and the marking targets include awnless ear, awned ear and rectangular scale. The YOLOv8n deep learning model is selected to perform transfer learning training and verification work, and a model capable of automatically recognizing awnless ear, awned ear and rectangular scale is obtained. According to the recognized position information, the awnless wheat ear and the rectangular scale are extracted, and the imcrop function is used to crop and segment the position of the awnless wheat ear and the rectangular scale in the image.

[0061] Figure 3 For the deep learning model in the embodiment of the application to identify the wheat ear and extract the schematic diagram, Figure 3 In the embodiment of the application, Figure 3 In the upper half of the model in the embodiment of the application, Figure 3 The lower half of the model in the embodiment of the application shows the process of extracting the wheat ear. Among them,

[0062] Backbone network: the main part of the model, used to extract basic features from the original input.

[0063] Multi-scale fusion network: used to fuse feature maps from different stages of the backbone network to enhance the feature representation capability.

[0064] Head network: a network used to process feature maps to produce model output results.

[0065] Loss calculation: composed of detection loss and classification loss, used for final target detection and classification tasks.

[0066] Decoupled head: decompose one detection head into two parts.

[0067] Intersection over union: the ratio of the intersection to the union of the detection box and the real box.

[0068] Distributed focal loss: solve the flexibility and accuracy problem in bounding box regression by allowing the coordinates of the bounding box to have a wider distribution.

[0069] Binary cross-entropy: handle binary classification problems, measure the difference between the probability distribution predicted by the model and the probability distribution of the true label.

[0070] Then, in order to obtain the phenotypic trait data of the actual ear, it is necessary to determine the conversion formula between the image size and the actual size. In step 103, according to the image of the specified size ruler, the actual size represented by each pixel in the ear image is determined.

[0071] In implementation, the size of the ruler is known, the pixels occupied by the ruler on the image can be obtained, and according to the imaging principle, the actual size represented by each pixel in the ear image can be obtained.

[0072] In an embodiment, according to the image of the specified size ruler, the actual size represented by each pixel in the ear image is determined, including the following steps:

[0073] Step 1, the image of the specified size ruler is subjected to local adaptive thresholding, binarization, closing operation and filling operation to obtain the binarized image of the ruler;

[0074] Step 2, according to the binarized image of the ruler, the area of the ruler in the image is determined;

[0075] Step 3, according to the area of the ruler in the image and the actual area of the ruler, the actual size represented by each pixel is determined.

[0076] Figure 4 For the processing schematic diagram of the rectangular ruler image in the embodiment of the application, refer to Figure 4 Taking the rectangular ruler in the preset background plate as the basis, the extracted rectangular ruler image is first processed, for example, using the adaptthresh function, imbinarize function, imclose function and imfill function to sequentially perform local adaptive thresholding, binarization, closing operation and filling operation on the original image, to obtain a complete rectangular ruler binarized image, and then calculate the number of all values of 1 in the rectangular ruler binarized image, which is the image area of the rectangular ruler, denoted as .

[0077] Further, according to the area of the ruler in the image and the actual area of the ruler, the actual size represented by each pixel can be determined according to the following formula:

[0078]

[0079] wherein SSOP is the actual size represented by each pixel, is the area of the ruler in the image, is the actual area of the ruler.

[0080] Then, based on the actual size represented by each pixel and the image of the ear extracted in step 102, the ear phenotype traits are measured. In the embodiment of the present application, the ear phenotype traits include ear shape (length, width), grain number per ear, spikelet number.

[0081] In step 104, the grain number per ear is identified in the image of the ear by the trained deep learning model; wherein each grain is marked by a rectangular frame.

[0082] In an embodiment, the YOLOv8n model used for image segmentation can be used for grain number identification.

[0083] In an embodiment, the deep learning model is a YOLOv8n model; the grain number per ear is identified in the image of the ear by the trained deep learning model, including: using the trained YOLOv8n model to identify the grains in a plurality of ear images, to obtain the grain number per ear in the image of the ear; wherein the YOLOv8n model is trained in advance using historical ear images and the grains marked in the historical ear images.

[0084] During model training, 1348 extracted ear image samples were selected, and the grains in the images were marked by a rectangular frame using Labelme. The previous YOLOv8n deep learning model was used for transfer learning training and verification work, to obtain a model capable of automatically identifying grains, and the grain number per ear was counted according to the grain identification result.

[0085] In step 105, the minimum circumscribed rectangle with the ear in the middle is determined by the minimum circumscribed rectangle method, and the actual length and actual width of the ear are determined according to the size of the minimum circumscribed rectangle and the actual size represented by each pixel.

[0086] The minimum circumscribed rectangle refers to the minimum area rectangle that can completely enclose a group of points or polygons. The minimum circumscribed rectangle method includes the equal interval rotation search method, the convex hull based method, and the approximation algorithm. The minimum circumscribed rectangle with the ear in the middle is determined by any minimum circumscribed rectangle method, and then the actual length and actual width of the ear are determined.

[0087] In an embodiment, the minimum circumscribed rectangle with the ear in the middle is determined by the minimum circumscribed rectangle method, and the actual length and actual width of the ear are determined according to the size of the minimum circumscribed rectangle and the actual size represented by each pixel, which can include the following steps:

[0088] Step 1, determine the minimum circumscribed rectangle with the ear in the middle by the minimum circumscribed rectangle method;

[0089] Step 2, based on the inclination angle of the minimum circumscribed rectangle, rotate the ear and the minimum circumscribed rectangle to be vertical to obtain the vertical minimum circumscribed rectangle;

[0090] Step 3, based on the pixel positions of the four vertices of the vertical minimum circumscribed rectangle, determine the length and width of the ear in the image;

[0091] Step 4, according to the actual size represented by each pixel in the ear image, the length and width of the ear in the image, determine the actual length and actual width of the ear.

[0092] In implementation, in order to measure the number of spikelets later, it is necessary to ensure that the ear is in a vertical state, which can be rotated based on the inclination angle of the minimum circumscribed rectangle to achieve the vertical state of the ear. Specifically, the coordinates of the four vertices of the minimum circumscribed rectangle are marked as , the inclination angle of the ear is solved based on the coordinates of the four vertices of the minimum circumscribed rectangle , and the imrotate() function is used for rotation to achieve the vertical state of the ear. The inclination angle is calculated as follows:

[0093]

[0094]

[0095] In the formula, Slope is an intermediate variable.

[0096] Continuing to refer to Figure 3 , Figure 3 The lower part shows the ear preprocessing, deep learning model identification of grains and extraction process.

[0097] The measurement method of ear length and ear width can use the minboundrect() function to solve the minimum circumscribed rectangle of the ear binary image, then the ear length EL and the width EW are calculated as follows:

[0098]

[0099]

[0100]

[0101]

[0102] In the formula, is an intermediate quantity.

[0103] In step 106, the number of spikelets is determined according to a first distance interval between the points obtained by projecting the center points of the spikelets to the vertical center line of the image and a second distance interval between the intersection points of the preset oblique lines passing through the center points of the spikelets and the vertical center line of the image; the center points of the spikelets are the center points of the rectangular frame markers; and each oblique line passes through a center point of a spikelet.

[0104] Figure 5 A schematic diagram of the center points of the rectangular frame markers of the spikelets in the embodiment of the present application is shown in FIG. 2. Figure 5 As shown in FIG. 2, based on the spikelet detection result of the YOLOv8n model, the rectangular frame of the spikelet in the wheat ear is obtained, and the center point position of each spikelet rectangular frame is calculated to replace the rectangular frame. The distance interval of the horizontal mapping of the center point of the spikelet and the distance interval of the horizontal mapping of the intercept point of the oblique line passing through the center point are fused to realize accurate classification of different spikelets on the same spikelet, thereby counting the number of spikelets.

[0105] In an embodiment, the wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center point of the spikelet on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center point of the spikelet on the right side of the vertical center line of the image is negative.

[0106] According to the distance interval between the points obtained by projecting the center points of the spikelets to the vertical center line of the image and the distance interval between the intersection points of the preset oblique lines passing through the center points of the spikelets and the vertical center line of the image, the number of spikelets can include the following steps:

[0107] Step 1, the length of each spikelet is determined by using the rectangular frame marker of each spikelet through the minimum circumscribed rectangle method.

[0108] Step 2, the average length of the spikelets is determined according to the length of each spikelet.

[0109] Step 3, a threshold value is determined according to the average length of the spikelets; the threshold value is used to classify the spikelets into spikelets.

[0110] Step 4, two spikelets corresponding to a first distance interval smaller than the threshold value are classified into the same spikelet to obtain a first classification result.

[0111] Step 5, two spikelets corresponding to a second distance interval smaller than the threshold value are classified into the same spikelet to obtain a second classification result; the first classification result and the second classification result both represent the classification of the spikelets belonging to the same spikelet.

[0112] Step 6, the first classification result and the second classification result are fused to obtain a final classification result; the final classification result includes multiple classifications, and each classification includes the spikelets belonging to the same spikelet.

[0113] Figure 6For the small spike number detection schematic diagram in the embodiments of the present application, refer to Figure 6 Take the vertical center line of the image (the center line of the ear of grain) as the Y axis. Considering that the grains on the same spike are distributed in a triangular shape, horizontal mapping of the grain center points to the Y axis cannot include the vertices of the triangular shape, therefore, a slant line is preset to solve the triangular distribution problem. The slant line passes through the grain center points, the size of the slope is fixed, the direction is variable, the slope is positive when the grain center points are on the left side of the Y axis, and the slope is negative when the grain center points are on the right side of the Y axis.

[0114] The first distance interval of horizontal mapping of the grain center points: the grain center points are horizontally mapped to the Y axis to form new marker points, the new marker points are arranged in order, the distance between adjacent marker points is calculated, a reasonable threshold is set to classify the new marker points, and the first classification result is obtained.

[0115] In implementation, the rectangular frame marker position of the grain is represented as (left, top; right, bottom), the coordinates of the grain center point GC are (left+width / 2, top+height / 2), the coordinates of the new marker point NP1 formed by horizontal mapping of the grain center point to the Y axis are (0, b), the sort() function is used to sort the new marker points NP1, the diff() function is used to calculate the distance d1 between adjacent points, and the classification threshold is set as th. If d1<th, the adjacent two points are classified into one class, otherwise, they are classified into different classes.

[0116] The second distance interval of horizontal mapping of the intercept point of the slant line passing through the center point: the intercept of the slant line passing through the grain center point on the Y axis is a new marker point, the new marker points are arranged in order, the distance between adjacent marker points is calculated, a reasonable threshold is set to classify the new marker points, and the second classification result is obtained.

[0117] Horizontal mapping of the grain center point to the Y axis cannot include the vertices of the triangular shape, and a slant line y=ax+b is further set. The acute angle of the triangular shape is left and right, therefore a=0.5 is set; the vertical center line of the image (the center line of the ear of grain) is taken as the Y axis, the slope of the grain center point on the left side of the Y axis is positive, that is, a=0.5, and the slope of the center point on the right side of the Y axis is negative, that is, a=-0.5. Then the slant line passes through the grain center point GC , , the value of b is obtained, the intercept of the slant line passing through the center point on the Y axis is a new marker point NP2, and the coordinates are (0, b). The sort() function is used to sort the new marker points NP2, the diff() function is used to calculate the distance d2 between adjacent points, and if d2<th, the adjacent two points are classified into one class, otherwise, they are classified into different classes. ​​​

[0118] Setting of threshold value th: the interval between the upper and lower spikelets is about half of the length of the spike grain, in order to ensure the accuracy of classification, one third of the mean of the length of all spike grains on the wheat ear is taken as the classification threshold value of the spikelet of each wheat ear, that is, th = mean (bottom-top) / 3.

[0119] Finally, combined with the comparison results of d1, d2 and th, all the spike grains are finally accurately classified into different spikelets, so that the number of spikelets is counted. Based on the distribution information of different spikelets, phenotypic traits related to the ear part, such as spikelet interval and loose ratio, can be further extracted. Reference Figure 6 The fusion process of the d1 classification result and the d2 classification result is finally obtained.

[0120] In summary, in the embodiment of the present application, a pre-set background plate is designed, and a mobile terminal such as a smart phone is used to obtain the wheat ear image in the field, solving the problem of destructive sampling. In the embodiment of the present application, deep learning algorithm and image processing algorithm are combined to realize rapid and accurate measurement of ear shape (length, width), number of spike grains and number of spikelets. The deep learning algorithm is used to extract wheat ears and grains, solving the problem of weak light resistance of traditional methods. The core algorithm for detecting the number of spikelets is to fuse the distance interval between the horizontal mapping of the center point of the grain and the intercept point of the inclined line passing through the center point, to realize accurate classification of different grains in the same spikelet, so as to count the number of spikelets and improve the accuracy of spikelet recognition.

[0121] In the embodiment of the present application, a wheat ear phenotype measurement device in a field environment is also provided, as described in the following embodiment. Since the principle of solving the problem of the device is similar to that of the wheat ear phenotype measurement method in the field environment, the implementation of the device can be referred to the implementation of the wheat ear phenotype measurement method in the field environment, and the repeated parts will not be described again.

[0122] Figure 7 A schematic diagram of the wheat ear phenotype measurement device in the field environment in the embodiment of the present application is shown in FIG. 7, which includes: Figure 7

[0123] The data acquisition module 701 is used to acquire the wheat ear image in the field environment through the mobile terminal; the wheat ear image includes a specified size scale, and the specified size scale is used to determine the size and position of the wheat ear in the wheat ear image;

[0124] The image extraction module 702 is used to perform image segmentation and extraction of the wheat ear and the specified size scale on the wheat ear image, to obtain the images of the wheat ear and the specified size scale;

[0125] ​The pixel actual size determination module 703 is configured to determine the actual size represented by each pixel in the wheat ear image according to the image of the specified size scale.

[0126] The grain number determination module 704 is configured to recognize the grain number in the image of the wheat ear by using the trained deep learning model, wherein each grain is marked by a rectangular frame.

[0127] The wheat ear length and width determination module 705 is configured to determine the minimum circumscribed rectangle with the wheat ear in the middle by using the minimum circumscribed rectangle method, and determine the actual length and the actual width of the wheat ear according to the size of the minimum circumscribed rectangle.

[0128] The spikelet number determination module 706 is configured to determine the spikelet number according to the first distance interval between the points obtained by projecting the grain center points to the vertical center line of the image and the second distance interval between the intersection points of the preset slant lines passing through the grain center points and the vertical center line of the image, wherein the grain center points are the centers of the rectangular frames, and each slant line passes through one grain center point.

[0129] In an embodiment, the data acquisition module 701 is specifically configured to acquire, by using a mobile terminal, an imaging image of the wheat ear in a field environment before the wheat ear is placed in front of a preset background plate, and the preset background plate is marked with a region for placing the photographed wheat ear and a specified size scale.

[0130] In an embodiment, the pixel actual size determination module 703 is specifically configured to perform local adaptive thresholding, binarization, closing operation and filling operation on the image of the specified size scale to obtain a binarized image of the scale, determine the area of the scale in the image according to the binarized image of the scale, and determine the actual size represented by each pixel according to the area of the scale in the image and the actual area of the scale.

[0131] In an embodiment, the pixel actual size determination module 703 is specifically configured to determine the actual size represented by each pixel according to the area of the scale in the image and the actual area of the scale according to the following formula:

[0132]

[0133] In the formula, SSOP represents the actual size represented by each pixel, S represents the area of the scale in the image, and S0 represents the actual area of the scale.

[0134] In an embodiment, the image extraction module 702 is specifically configured to: perform image extraction of the multiple ear images and the specified size scale by using a trained YOLOv8n model, to obtain multiple ear images and the specified size scale; and the YOLOv8n model is trained in advance by using historical ear images, ears extracted from the historical ear images, and the specified size scale extracted from the historical ear images.

[0135] In an embodiment, the deep learning model is a YOLOv8n model.

[0136] The ear grain number determination module 704 is specifically configured to: perform ear grain identification on the multiple ear images by using a trained YOLOv8n model, to obtain the ear grain number in the ear image; and the YOLOv8n model is trained in advance by using historical ear images and ear grains marked in the historical ear images.

[0137] In an embodiment, the ear length and width determination module 705 is specifically configured to: determine a minimum bounding rectangle with the ear in the middle by a minimum bounding rectangle method; rotate the ear and the minimum bounding rectangle to be vertical based on the tilt angle of the minimum bounding rectangle, to obtain a vertical minimum bounding rectangle; determine the length and width of the ear in the image based on the pixel positions of the four vertices of the vertical minimum bounding rectangle; and determine the actual length and the actual width of the ear according to the actual size represented by each pixel in the ear image, the length and the width of the ear in the image.

[0138] In an embodiment, the ear is located in the middle of the image, wherein the slope of the oblique line passing through the center points of the ear grains on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center points of the ear grains on the right side of the vertical center line of the image is negative.

[0139] The spikelet number determination module 706 is specifically configured to: determine the length of each ear grain by a minimum bounding rectangle method by using the rectangular frame mark of each ear grain; determine the average length of the ear grains according to the length of each ear grain; determine a threshold value according to the average length of the ear grains; the threshold value is used to classify the ear grains into spikelets; two ear grains corresponding to a first distance interval less than the threshold value are classified into the same spikelet to obtain a first classification result; two ear grains corresponding to a second distance interval less than the threshold value are classified into the same spikelet to obtain a second classification result; the first classification result and the second classification result both represent the classification of the ear grains belonging to the same spikelet; and the first classification result and the second classification result are fused to obtain a final classification result; the final classification result includes multiple classifications, and each classification includes ear grains belonging to the same spikelet.

[0140] The embodiment of the present application further provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method for measuring the ear phenotype of wheat in the field environment when executing the computer program.

[0141] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for measuring the ear phenotype of wheat in the field environment.

[0142] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the method for measuring the ear phenotype of wheat in the field environment.

[0143] In the embodiment of the present application, the ear image of wheat in the field environment is acquired by the mobile terminal, so that the ear image of wheat is acquired without damaging the growth of wheat, the specified size scale is included in the ear image, then the actual size represented by each pixel in the ear image is determined by using the specified size scale, so that the actual length and the actual width of the ear are determined by the actual size represented by each pixel and the pixel characteristics of the ear in the image, and the number of grains in the ear is determined by deep learning, meanwhile, the first distance interval between the points projected to the vertical center line of the image according to the center points of the grains and the second distance interval between the intersection points of the preset slant line passing through the center points of the grains and the vertical center line of the image are determined to determine the number of spikelets, compared with the prior art, the method provided in the embodiment of the present application can combine deep learning and image processing, and more ear phenotypes in the ear image are acquired, so that the accuracy of the measurement of the ear phenotype is effectively improved.

[0144] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0145] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0146] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0147] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 Figure 1 one or more flowcharts and / or blocks

[0148] The above-described specific embodiments, the purpose, technical solutions and advantages of the present application are further described in detail, it should be understood that the above-described only for the specific embodiments of the present application has been described, and not used to limit the scope of protection of the present application, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application, should be included within the scope of protection of the present application.

Claims

1. A method for measuring the phenotypic traits of wheat ears in a field environment, characterized in that, include: Images of wheat ears in a field environment are acquired via a mobile terminal; the wheat ear images include a specified size scale, which is used to determine the size and position of the wheat ears in the wheat ear images; The wheat ear image is segmented and extracted to obtain images of the wheat ear and the specified size scale. Determine the actual size represented by each pixel in the wheat ear image based on the image with a specified size scale. The number of grains per ear in a wheat ear image is identified using a trained deep learning model; each grain is marked with a rectangular box. The minimum bounding rectangle method is used to determine the minimum bounding rectangle with the wheat ear in the middle. Based on the size of the minimum bounding rectangle and the actual size represented by each pixel, the actual length and actual width of the wheat ear are determined. The number of spikelets is determined based on the first distance interval between the points after the spikelet center point is projected onto the vertical center line of the image, and the second distance interval between the intersection points of the diagonal line passing through the spikelet center point and the vertical center line of the image; the spikelet center point is the center of the rectangular frame mark; each diagonal line passes through one spikelet center point; Among them, the wheat ears are in an upright position; First distance interval: The center point of the ear of grains is horizontally mapped to the Y-axis to form a new marker point. The new marker points are arranged in order, the distance between adjacent marker points is calculated, and a threshold is set to classify the new marker points to obtain the first classification result; the Y-axis is the center line of the ear of wheat. Second distance interval: The intercept of the oblique line on the Y-axis through the center point of the ear of grain is the new marker point. The new marker points are arranged in order, the distance between adjacent marker points is calculated, and a threshold is set to classify the new marker points to obtain the second classification result. The first and second classification results are combined to obtain the final classification result.

2. The method as described in claim 1, characterized in that, Images of wheat ears in a field environment are acquired via mobile devices, including: The image of wheat ears placed in front of a preset background board is acquired through a mobile terminal; the preset background board is marked with the area where the wheat ears are placed and a specified size scale.

3. The method as described in claim 1, characterized in that, Based on the image with a specified size scale, determine the actual size represented by each pixel in the wheat ear image, including: Perform local adaptive thresholding, binarization, closing, and filling operations on the image of the scale bar of a specified size to obtain the binarized image of the scale bar; Determine the area of ​​the scale in the image based on the binarized image of the scale; The actual size represented by each pixel is determined based on the area of ​​the ruler in the image and the actual area of ​​the ruler.

4. The method as described in claim 3, characterized in that, Based on the area of ​​the ruler in the image and the actual area of ​​the ruler, determine the actual size represented by each pixel, including: The actual size represented by each pixel is determined using the following formula, based on the area of ​​the ruler in the image and the actual area of ​​the ruler: ； In the formula, SSOP represents the actual size of each pixel. The area of ​​the ruler in the image. This represents the actual area on the scale.

5. The method as described in claim 1, characterized in that, The wheat ear image is segmented and extracted to obtain images of the wheat ear and the specified size scale, including: The trained YOLOv8n model is used to extract images of wheat ears and a specified size scale from multiple wheat ear images to obtain multiple images of wheat ears and a specified size scale. The YOLOv8n model is trained in advance using historical wheat ear images, wheat ears extracted from historical wheat ear images, and specified size scales extracted from historical wheat ear images.

6. The method as described in claim 5, characterized in that, The deep learning model is the YOLOv8n model; The trained deep learning model identifies the number of grains per ear in an image of wheat, including: The trained YOLOv8n model is used to identify grains in multiple wheat ear images to obtain the number of grains in each image. The YOLOv8n model is trained in advance using historical wheat ear images and the marked grains in those images.

7. The method as described in claim 1, characterized in that, The minimum bounding rectangle method is used to determine the minimum bounding rectangle with the wheat ear in the center. Based on the size of the minimum bounding rectangle and the actual size represented by each pixel, the actual length and width of the wheat ear are determined, including: The minimum bounding rectangle with the wheat ear in the middle is determined by the minimum bounding rectangle method. Based on the tilt angle of the minimum bounding rectangle, rotate the wheat ears and the minimum bounding rectangle to be upright to obtain the upright minimum bounding rectangle; The length and width of the wheat ears in the image are determined based on the pixel positions of the four vertices of the vertical minimum bounding rectangle. The actual length and width of the wheat ear are determined based on the actual size represented by each pixel in the wheat ear image, and the length and width of the wheat ear in the image.

8. The method as described in claim 1, characterized in that, The wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center point of the grain on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center point of the grain on the right side of the vertical center line of the image is negative. The number of spikelets is determined based on the distance between points projected from the center point of the spike onto the vertical center line of the image, and the distance between the intersections of a pre-defined oblique line passing through the center point of the spike and the vertical center line of the image. This includes: The length of each ear of grain is determined by using the minimum bounding rectangle method and marking the rectangle of each ear of grain. The average length of grains per ear is determined based on the length of each grain. A threshold is determined based on the average length of grains per spike; this threshold is used to classify grains per spike into spikelets. Two grains corresponding to the first distance interval less than the threshold are classified as the same spikelet, and the first classification result is obtained. Two grains corresponding to a second distance interval less than the threshold are classified as the same spikelet, resulting in a second classification result; both the first and second classification results indicate that grains belonging to the same spikelet are classified. The first and second classification results are combined to obtain the final classification result; the final classification result includes multiple categories, and each category includes grains belonging to the same spikelet.

9. A device for measuring the phenotypic traits of wheat ears in a field environment, characterized in that, include: The data acquisition module is used to acquire images of wheat ears in a field environment through a mobile terminal; the wheat ear images include a specified size scale, which is used to determine the size and position of the wheat ears in the wheat ear images; The image extraction module is used to segment and extract images of wheat ears and a specified size scale from wheat ear images, resulting in images of wheat ears and a specified size scale. The pixel actual size determination module is used to determine the actual size represented by each pixel in the wheat ear image based on the image with a specified size scale. The grain count determination module is used to identify the number of grains per ear in a wheat ear image using a trained deep learning model; each grain is marked with a rectangular box. The wheat ear length and width determination module is used to determine the minimum bounding rectangle with the wheat ear in the middle using the minimum bounding rectangle method, and to determine the actual length and width of the wheat ear based on the size of the minimum bounding rectangle. The spikelet number determination module is used to determine the number of spikelets based on the first distance interval between the points after the spikelet center point is projected onto the vertical center line of the image, and the second distance interval between the intersection points of the diagonal line passing through the spikelet center point and the vertical center line of the image; the spikelet center point is the center of the rectangular frame mark; each diagonal line passes through one spikelet center point; wherein the wheat ear is in an upright state; First distance interval: The center point of the ear of grains is horizontally mapped to the Y-axis to form a new marker point. The new marker points are arranged in order, the distance between adjacent marker points is calculated, and a threshold is set to classify the new marker points to obtain the first classification result; the Y-axis is the center line of the ear of wheat. Second distance interval: The intercept of the oblique line on the Y-axis through the center point of the ear of grain is the new marker point. The new marker points are arranged in order, the distance between adjacent marker points is calculated, and a threshold is set to classify the new marker points to obtain the second classification result. The first and second classification results are combined to obtain the final classification result.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images