Method and device for measuring phenotypic characters of wheat ears in field environment

Abstract

The invention discloses a method and a device for measuring phenotypic characters of wheat ears in a field environment. The method comprises the following steps: acquiring a wheat ear image in the field environment through a mobile terminal; the wheat ear image comprises a specified size scale, and the actual size represented by each pixel in the wheat ear image is determined by using the specified size scale; the actual length and the actual width of the wheat ears are determined through the actual size represented by each pixel and the pixel features of the wheat ears in the image, and the number of the wheat ears is determined through deep learning; and determining the spikelet number according to a first distance interval between points after the center point of the spike grain is projected to the vertical center line of the image and a second distance interval between preset intersection points of oblique lines passing through the center point of the spike grain and the vertical center line of the image. According to the method, the wheat ear image can be obtained on the premise that wheat growth is not damaged, more wheat ear phenotypic characters are determined according to the wheat ear image, and the measurement accuracy of the wheat ear phenotypic characters is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for measuring wheat ear phenotypic traits in a field environment. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.

[0003] Wheat yield is a complex trait that is significantly affected by genetic and environmental factors and has low heritability. Therefore, it is usually genetically decomposed into three yield factors, including the number of spikes per unit area, the number of grains per spike, and the grain weight. Among them, the number of grains per spike is considered to have a high heritability, and it is generally believed that improving the number of grains per spike of wheat is very important for improving wheat yield. The number of grains per spike is largely affected by the inflorescence structure. A typical wheat spike usually consists of a main axis called the rachis and several sessile spikelets directly attached to the rachis, producing an unbranched spike structure. Most wheat varieties develop a spikelet on each rachis node and have multiple florets. Generally speaking, only 1-3 florets can become fertile florets and develop into grains. Improvements in the number of spikelets and florets (floret primordia and fertile florets) have significantly contributed to the increase in the number of grains per spike. The components of the spike inflorescence (such as spikelets, which are the basic units of the inflorescence and contain florets, florets, and lemmas) also affect each other. The number and arrangement of these spike components affect spike length, spike weight, ear bran (i.e., non-grain biomass in the spike), number of grains per spike, grain weight per spike, and number of spikelets per spike, all of which contribute to the final grain yield per spike. In addition, the number of spikelets, fertile florets, and grains can be calculated to further evaluate the characteristics of the spike by calculating the fertility of the spikelets, the proportion of fertile florets, and the ratio of grains / fertile florets. In order to produce varieties with the most efficient grain production in different environments, it is necessary to know how to predict and manipulate wheat spike morphology. Therefore, counting the number of spikelets, spikelets, and florets (grains) during the breeding process is important for screening high-yield wheat varieties.

[0004] The existing methods for measuring ear phenotypic traits still rely on manual destructive sampling observation, which is time-consuming and laborious. In recent years, there have been many studies on using image processing methods to obtain ear traits. These methods also require destructive sampling, and then take images in the laboratory for measurement. This is extremely unfavorable for breeding plots with small sample sizes. In addition, a certain phenotypic trait of the wheat ear is usually measured alone, which has low measurement efficiency. For the measurement of the number of spikelets, predecessors mostly used side images of the wheat ear for detection. Image acquisition is difficult, measurement efficiency is low, and detection accuracy is poor. Summary of the invention

[0005] The embodiment of the present invention provides a method for measuring wheat ear phenotypic traits in a field environment, which is used to obtain wheat ear images without destroying wheat growth, determine more wheat ear phenotypic traits based on the wheat ear images, and improve the measurement accuracy of wheat ear phenotypic traits. The method includes:

[0006] Acquiring an image of wheat ears in a field environment through a mobile terminal; the wheat ear image includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat ears in the wheat ear image;

[0007] Perform image segmentation and extraction of wheat ears and a scale of specified size on the wheat ear image to obtain images of the wheat ears and the scale of specified size;

[0008] According to the image of the specified size ruler, determine the actual size represented by each pixel in the wheat ear image;

[0009] Identify the number of grains in the wheat ear image using a trained deep learning model, where each grain is marked by a rectangular box;

[0010] The minimum enclosing rectangle method is used to determine the minimum enclosing rectangle with the wheat ear in the middle, and the actual length and width of the wheat ear are determined according to the size of the minimum enclosing rectangle and the actual size represented by each pixel;

[0011] The number of spikelets is determined based on the first distance interval between the points after the center point of the ear is projected onto the vertical center line of the image, and the second distance interval between the intersection points of a preset oblique line passing through the center point of the ear and the vertical center line of the image; the center point of the ear is the center of the rectangular frame mark; each oblique line passes through the center point of the ear.

[0012] In one embodiment, obtaining an image of wheat ears in a field environment through a mobile terminal includes: obtaining an imaging image of wheat ears in a field environment placed in front of a preset background plate through a mobile terminal; marking the placement area of ​​the photographed wheat ears and a specified size scale on the preset background plate.

[0013] In one embodiment, the actual size represented by each pixel in the wheat ear image is determined based on an image of a ruler of a specified size, including: performing local adaptive thresholding, binarization, closing operations, and filling operations on the image of the ruler of the specified size to obtain a binarized image of the ruler; determining the area of ​​the ruler in the image based on the binarized image of the ruler; and determining the actual size represented by each pixel based on the area of ​​the ruler in the image and the actual area of ​​the ruler.

[0014] In one embodiment, determining the actual size represented by each pixel according to the area of ​​the ruler in the image and the actual area of ​​the ruler includes: determining the actual size represented by each pixel according to the area of ​​the ruler in the image and the actual area of ​​the ruler according to the following formula:

[0015]

[0016] Where SSOP is the actual size represented by each pixel, Simage is the area of ​​the ruler in the image, and Sactual is the actual area of ​​the ruler.

[0017] In one embodiment, image segmentation and extraction of wheat ears and a scale of specified size are performed on a wheat ear image to obtain images of the wheat ears and the scale of specified size, including: using a trained YOLOv8n model to extract images of wheat ears and a scale of specified size from multiple wheat ear images to obtain multiple images of wheat ears and the scale of specified size; the YOLOv8n model is pre-trained using historical wheat ear images, wheat ears extracted from historical wheat ear images, and scales of specified size extracted from historical wheat ear images.

[0018] In one embodiment, the deep learning model is a YOLOv8n model;

[0019] The number of grains per ear in an image of a wheat ear is identified by using a trained deep learning model, including: using a trained YOLOv8n model to perform grain identification on a plurality of wheat ear images to obtain the number of grains per ear in the image of the wheat ear; wherein the YOLOv8n model is trained in advance using historical wheat ear images and grains marked in the historical wheat ear images.

[0020] In one embodiment, a minimum enclosing rectangle with an ear of wheat in the middle is determined by a minimum enclosing rectangle method, and the actual length and actual width of the ear of wheat are determined according to the size of the minimum enclosing rectangle and the actual size represented by each pixel, including: determining the minimum enclosing rectangle with the ear of wheat in the middle by a minimum enclosing rectangle method; rotating the ear of wheat and the minimum enclosing rectangle to stand upright based on the inclination angle of the minimum enclosing rectangle to obtain an upright minimum enclosing rectangle; determining the length and width of the ear of wheat in the image based on the pixel positions of the four vertices of the upright minimum enclosing rectangle; determining the actual length and actual width of the ear of wheat according to the actual size represented by each pixel in the ear of wheat image and the length and width of the ear of wheat in the image.

[0021] In one embodiment, the wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center point of the ear grain on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center point of the ear grain on the right side of the vertical center line of the image is negative;

[0022] The number of spikelets is determined according to the distance intervals between the points after the center point of the spike is projected onto the vertical center line of the image, and the distance intervals between the intersection points of the preset oblique line passing through the center point of the spike and the vertical center line of the image, including: determining the length of each spike using the rectangular frame mark of each spike by the minimum circumscribed rectangle method; determining the average length of the spike according to the length of each spike; determining a threshold according to the average length of the spike; the threshold is used to classify the spikelets; classifying two spikelets corresponding to a first distance interval less than the threshold as the same spikelet to obtain a first classification result; classifying two spikelets corresponding to a second distance interval less than the threshold as the same spikelet to obtain a second classification result; the first classification result and the second classification result both indicate that the spikelets belonging to the same spikelet are classified; the first classification result and the second classification result are merged to obtain a final classification result; the final classification result includes multiple categories, each category including spikelets belonging to the same spikelet.

[0023] The embodiment of the present invention further provides a wheat ear phenotypic trait measurement device in a field environment, which is used to obtain wheat ear images without destroying wheat growth, determine more wheat ear phenotypic traits based on the wheat ear images, and improve the measurement accuracy of wheat ear phenotypic traits. The device includes:

[0024] A data acquisition module, used for acquiring an image of wheat ears in a field environment through a mobile terminal; the image of wheat ears includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat ears in the image of wheat ears;

[0025] An image extraction module is used to perform image segmentation and extraction of wheat ears and a scale of a specified size on the wheat ear image to obtain images of the wheat ears and the scale of the specified size;

[0026] A pixel actual size determination module is used to determine the actual size represented by each pixel in the wheat ear image according to an image of a specified size ruler;

[0027] The grain number determination module is used to identify the number of grains in the wheat ear image through the trained deep learning model; each grain is marked by a rectangular frame;

[0028] The wheat ear length and width determination module is used to determine the minimum circumscribed rectangle with the wheat ear in the middle by the minimum circumscribed rectangle method, and determine the actual length and actual width of the wheat ear according to the size of the minimum circumscribed rectangle;

[0029] The spikelet number determination module is used to determine the spikelet number based on the first distance interval between the points after the center point of the spikelet is projected onto the vertical center line of the image, and the second distance interval between the intersection points of the preset oblique line passing through the center point of the spikelet and the vertical center line of the image; the center point of the spikelet is the center of the rectangular frame mark; each oblique line passes through the center point of the spikelet.

[0030] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for measuring wheat ear phenotypic traits in a field environment when executing the computer program.

[0031] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for measuring wheat ear phenotypic traits in a field environment is implemented.

[0032] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for measuring wheat ear phenotypic traits in a field environment is implemented.

[0033] In the embodiment of the present invention, a wheat ear image in a field environment is obtained by a mobile terminal, so that the wheat ear image is obtained without destroying the growth of wheat. The wheat ear image includes a specified size ruler, and then the specified size ruler is used to determine the actual size represented by each pixel in the wheat ear image, so that the actual length and actual width of the wheat ear are determined by the actual size represented by each pixel and the pixel features of the wheat ear in the image, and the number of grains per ear of the wheat ear is determined by deep learning. At the same time, the number of spikelets is determined according to the first distance interval between the points after the center point of the grain is projected to the vertical center line of the image, and the second distance interval between the intersection points of the preset oblique line passing through the center point of the grain and the vertical center line of the image. Compared with the prior art, the method proposed in the embodiment of the present invention can combine deep learning and image processing, and simultaneously obtain more wheat ear phenotypic traits in the ear image, thereby effectively improving the accuracy of measuring the phenotypic traits of the wheat ear. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0035] Figure 1 Schematic diagram of the process of measuring wheat ear phenotypic traits in a field environment in an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of a pre-set background plate in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of wheat ear recognition and extraction using a deep learning model in an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of rectangular ruler image processing in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the center point of the rectangular frame marking of the spikelets in an embodiment of the present invention;

[0040] Figure 6 Schematic diagram of spikelet number detection in an embodiment of the present invention;

[0041] Figure 7 Schematic diagram of a wheat ear phenotypic trait measuring device in a field environment in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0043] Figure 1 FIG. 1 is a flow chart of a method for measuring wheat ear phenotypic traits in a field environment according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0044] Step 101: acquiring an image of wheat ears in a field environment through a mobile terminal; the wheat ear image includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat ears in the wheat ear image;

[0045] Step 102, performing image segmentation and extraction of wheat ears and a scale of a specified size on the wheat ear image to obtain images of the wheat ears and the scale of the specified size;

[0046] Step 103, determining the actual size represented by each pixel in the wheat ear image according to the image of the designated size ruler;

[0047] Step 104: Identify the number of grains in the wheat ear image using the trained deep learning model; wherein each grain is marked by a rectangular frame;

[0048] Step 105: Determine the minimum bounding rectangle with the wheat ear in the middle by the minimum bounding rectangle method, and determine the actual length and width of the wheat ear according to the size of the minimum bounding rectangle and the actual size represented by each pixel;

[0049] Step 106, determine the number of spikelets based on the first distance interval between the points after the center point of the ear is projected onto the vertical center line of the image, and the second distance interval between the intersection points of a preset oblique line passing through the center point of the ear and the vertical center line of the image; the center point of the ear is the center of the rectangular frame mark; each oblique line passes through the center point of an ear.

[0050] from Figure 1 It can be seen from the shown process that in the embodiment of the present invention, the wheat ear image in the field environment is obtained by a mobile terminal, so that the wheat ear image is obtained without destroying the growth of wheat. The wheat ear image includes a specified size ruler, and then the specified size ruler is used to determine the actual size represented by each pixel in the wheat ear image, so that the actual length and actual width of the wheat ear are determined by the actual size represented by each pixel and the pixel feature of the wheat ear in the image, and the number of grains per ear of the wheat ear is determined by deep learning. At the same time, the number of spikelets is determined according to the first distance interval between the points after the center point of the grain is projected to the vertical center line of the image, and the second distance interval between the intersection points of the preset oblique line passing through the center point of the grain and the vertical center line of the image. Compared with the prior art, the method proposed in the embodiment of the present invention can combine deep learning and image processing, and simultaneously obtain more phenotypic traits of wheat ears in the ear image, thereby effectively improving the accuracy of measuring the phenotypic traits of wheat ears.

[0051] Below Figure 1 The method for measuring wheat ear phenotypic traits under field environment is explained in detail.

[0052] First, the wheat ears are imaged without destroying the wheat growth in the field. The growing wheat ears are directly photographed using mobile terminals and mobile devices such as mobile phones and tablets.

[0053] The wheat ear image captured in the embodiment of the present invention includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat awn and wheat ear in the wheat ear image.

[0054] In one embodiment, obtaining an image of wheat ears in a field environment through a mobile terminal may include: obtaining an imaging image of wheat ears in a field environment placed in front of a preset background plate through a mobile terminal; marking the placement area of ​​the photographed wheat ears and a specified size scale on the preset background plate.

[0055] Figure 2 is a schematic diagram of presetting a background plate in an embodiment of the present invention, Figure 2 The size of the preset background plate is 210mm×297mm, including an image acquisition area of ​​140mm×210mm, a rectangular ruler of 10mm×100mm, and a thumb pressing area. When shooting, place the wheat ears in the field as a whole in the image acquisition area of ​​the preset background plate, use your thumb to gently press the lower stem of the wheat ears to fix the wheat ears, and use your other hand to use a smartphone to capture the wheat ears in the image acquisition area.

[0056] During shooting, a large number of wheat field ear image samples can be collected through mobile terminals, which can then be used for model training and measurement of wheat ear phenotypic traits.

[0057] In step 102, the wheat ear image is segmented and extracted to obtain images of the wheat ear and the designated size scale.

[0058] For example, conventional image processing techniques are used to extract images of wheat awns, wheat ears, and designated size scales.

[0059] In order to improve the accuracy of image extraction, in one embodiment, image segmentation and extraction of wheat ears and designated size scales are performed on wheat ear images to obtain images of wheat ears and designated size scales, including: using a trained YOLOv8n model to extract images of wheat ears and designated size scales from multiple wheat ear images to obtain multiple images of wheat ears and designated size scales; the YOLOv8n model is pre-trained using historical wheat ear images, wheat ears extracted from historical wheat ear images, and designated size scales extracted from historical wheat ear images.

[0060] During implementation, a large number of wheat field ear image samples were collected in advance using different models of smartphones to construct training sets, test sets, and validation sets, and the YOLOv8n model architecture was built, and the YOLOv8n model was trained using the training set. During model training, Labelme was used to mark the image with a rectangular frame, and the marked targets included ears without awns, ears with awns, and rectangular rulers. The YOLOv8n deep learning model was selected to perform transfer learning training and verification on the data to obtain a model that can automatically identify ears without awns, ears with awns, and rectangular rulers. The ears without awns and rectangular rulers were extracted based on the identified position information, specifically using the imcrop function to crop and segment the positions of the ears without awns and rectangular rulers in the image.

[0061] Figure 3 This is a schematic diagram of wheat ear recognition and extraction using a deep learning model in an embodiment of the present invention. Figure 3 In, increase Figure 3 The upper part of the model is a text description. Figure 3 The lower middle part shows the process of wheat ear extraction.

[0062] Backbone network: The main part of the model, used to extract basic features from the raw input.

[0063] Multi-scale fusion network: used to fuse feature maps from different stages of the backbone network to enhance feature representation capabilities.

[0064] Head network: A network used to process feature maps to produce model output results.

[0065] Loss calculation: It consists of detection loss and classification loss, which is used for the final target detection and classification tasks.

[0066] Decoupling head: decompose the original detection head into two parts.

[0067] Intersection-to-union ratio: The ratio of the intersection and union of the detection box and the true box.

[0068] Distributional Focal Loss: Addresses the flexibility and accuracy issues in bounding box regression by allowing the coordinates of the bounding box to have a wider distribution.

[0069] Binary Cross Entropy: It handles binary classification problems and measures the difference between the probability distribution of the model prediction and the probability distribution of the true label.

[0070] Next, in order to obtain the phenotypic trait data of the actual wheat ear, it is necessary to determine the conversion formula between the image size and the actual size. In step 103, the actual size represented by each pixel in the wheat ear image is determined based on the image of the specified size scale.

[0071] During implementation, the size of the ruler is known, the pixels occupied by the ruler on the image can be obtained, and the actual size represented by each pixel in the wheat ear image can be obtained based on the imaging principle.

[0072] In one embodiment, determining the actual size represented by each pixel in the wheat ear image according to the image of the designated size ruler includes the following steps:

[0073] Step 1, perform local adaptive threshold, binarization, closing operation, and filling operation on the image of the specified size scale to obtain a binarized image of the scale;

[0074] Step 2, according to the binary image of the ruler, determine the area of ​​the ruler in the image;

[0075] Step 3: Determine the actual size represented by each pixel based on the area of ​​the ruler in the image and the actual area of ​​the ruler.

[0076] Figure 4 is a schematic diagram of rectangular ruler image processing in an embodiment of the present invention, referring to Figure 4 , based on the rectangular ruler in the preset background plate, the extracted rectangular ruler image is first processed, for example, the adaptthresh function, imbinarize function, imclose function and imfill function are used to perform local adaptive threshold, binarization, closing operation, filling and other operations on the original image in turn to obtain a complete rectangular ruler binarized image, and then the number of all values ​​1 in the rectangular ruler binarized image is calculated as the image area of ​​the rectangular ruler, which is recorded as S 图像 .

[0077] Furthermore, the actual size represented by each pixel can be determined according to the area of ​​the ruler in the image and the actual area of ​​the ruler according to the following formula:

[0078]

[0079] Where SSOP is the actual size represented by each pixel, S 图像 is the area of ​​the ruler in the image, S 实际 Actual area of ​​ruler.

[0080] Afterwards, wheat ear phenotypic traits are measured based on the actual size represented by each pixel and the image of the wheat ear extracted in step 102. In the embodiment of the present invention, the wheat ear phenotypic traits include ear shape (length, width), number of grains per ear, and number of spikelets.

[0081] In step 104, the number of grains in the image of the wheat ear is identified by using the trained deep learning model; each grain is marked by a rectangular frame.

[0082] Among them, the YOLOv8n model used in image segmentation and extraction can be used to identify the number of grains per ear.

[0083] In one embodiment, the deep learning model is a YOLOv8n model; identifying the number of grains in an image of wheat ears by using a trained deep learning model includes: using the trained YOLOv8n model to perform grain recognition on multiple wheat ear images to obtain the number of grains in the wheat ear images; wherein the YOLOv8n model is trained in advance using historical wheat ear images and the grains marked in the historical wheat ear images.

[0084] During model training, 1,348 samples of wheat ear images were selected, and the kernels in the images were marked with rectangular frames using Labelme. The previous YOLOv8n deep learning model was used to perform transfer learning training and verification on the data to obtain a model that can automatically identify kernels, and the number of kernels per ear was counted based on the kernel recognition results.

[0085] In step 105, the minimum bounding rectangle with the wheat ear in the middle is determined by the minimum bounding rectangle method, and the actual length and width of the wheat ear are determined according to the size of the minimum bounding rectangle and the actual size represented by each pixel.

[0086] The minimum enclosing rectangle refers to the rectangle with the smallest area that can completely enclose a set of points or polygons. Methods for calculating the minimum enclosing rectangle include the equal-interval rotation search method, the convex hull-based method, and the approximate algorithm. The minimum enclosing rectangle with the wheat ear in the middle is determined by any minimum enclosing rectangle method, and then the actual length and actual width of the wheat ear are determined.

[0087] In one embodiment, the minimum enclosing rectangle method is used to determine the minimum enclosing rectangle with the wheat ear in the middle, and the actual length and width of the wheat ear are determined according to the size of the minimum enclosing rectangle and the actual size represented by each pixel, which may include the following steps:

[0088] Step 1: Determine the minimum enclosing rectangle with the wheat ear in the middle by using the minimum enclosing rectangle method;

[0089] Step 2: Based on the tilt angle of the minimum circumscribed rectangle, rotate the wheat ear and the minimum circumscribed rectangle to be upright, so as to obtain an upright minimum circumscribed rectangle;

[0090] Step 3: Determine the length and width of the wheat ear in the image based on the pixel positions of the four vertices of the erected minimum circumscribed rectangle;

[0091] Step 4: Determine the actual length and width of the wheat ear according to the actual size represented by each pixel in the wheat ear image and the length and width of the wheat ear in the image.

[0092] During implementation, in order to measure the number of spikelets later, it is necessary to ensure that the wheat ears are in an upright state. Here, the wheat ears can be rotated based on the inclination angle of the minimum circumscribed rectangle to achieve the upright state of the wheat ears. Specifically, the four vertex coordinates of the minimum circumscribed rectangle are marked as ([x1, y1], [x2, y2], [x3, y3], [x4, y4]), and the inclination angle θ of the wheat ears is solved based on the four vertex coordinates of the minimum circumscribed rectangle. Then, the imrotate() function is used to perform a rotation operation to achieve the upright state of the wheat ears. Among them, the calculation formula of the inclination angle θ is as follows:

[0093] Slope=(x1-x2) / (x1-x4)

[0094]

[0095] Where Slope is an intermediate calculation variable.

[0096] Continue to refer Figure 3 , Figure 3 The lower part shows the wheat ear preprocessing, deep learning model recognition of grains and extraction process.

[0097] The measurement method of wheat ear length and wheat ear width can use the minboundrect() function to solve the minimum bounding rectangle of the wheat ear binary image, and the calculation formula of wheat ear length EL and width EW is:

[0098]

[0099] EL(mm)=max(recta,rectb)×SSOP

[0100] EW(mm)=min(recta,rectb)×SSOP

[0101] In the formula, reata and reatb are intermediate quantities.

[0102] In step 106, the number of spikelets is determined based on the first distance interval between the points after the center point of the ear is projected onto the vertical center line of the image, and the second distance interval between the intersection points of a preset oblique line passing through the center point of the ear and the vertical center line of the image; the center point of the ear is the center of the rectangular frame mark; each oblique line passes through the center point of an ear.

[0103] Figure 5 Schematic diagram of the center point of the rectangular frame marking of the spikelets in the embodiment of the present invention. Figure 5 As shown in the figure, based on the grain detection results of the YOLOv8n model, the rectangular frame of the grain in the wheat ear is obtained, and the center point position of each grain rectangular frame is obtained to replace the rectangular frame. The horizontal mapping distance interval of the grain center point is fused with the horizontal mapping distance interval of the intercept point of the oblique line passing through the center point to achieve accurate classification of different grains on the same spikelet, so as to count the number of spikelets.

[0104] In one embodiment, the wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center point of the ear grain on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center point of the ear grain on the right side of the vertical center line of the image is negative.

[0105] Determining the number of spikelets according to the distance intervals between points after the center point of the spikelet is projected onto the vertical center line of the image, and the distance intervals between the intersection points of a preset oblique line passing through the center point of the spikelet and the vertical center line of the image, may include the following steps:

[0106] Step 1: Determine the length of each ear grain by using the minimum circumscribed rectangle method and the rectangular frame mark of each ear grain;

[0107] Step 2, according to the length of each ear, determine the average length of the ear;

[0108] Step 3, determining a threshold value according to the average length of the spikelets; the threshold value is used to classify the spikelets into spikelets;

[0109] Step 4: classify two spikelets corresponding to a first distance interval smaller than a threshold value as the same spikelet, thereby obtaining a first classification result;

[0110] Step 5: classify the two spikelets corresponding to the second distance interval less than the threshold value as the same spikelet, and obtain a second classification result; the first classification result and the second classification result both indicate that the spikelets belonging to the same spikelet are classified;

[0111] Step 6: merge the first classification result and the second classification result to obtain a final classification result; the final classification result includes multiple categories, and each category includes spikelets belonging to the same spikelet.

[0112] Figure 6 Schematic diagram of spikelet number detection in the embodiment of the present invention, refer to Figure 6 Take the vertical center line of the image (the center line of the wheat ear) as the Y-axis. Considering that the grains on the same spikelet are triangularly distributed, horizontally mapping the center point of the grain to the Y-axis cannot incorporate the vertices of the triangle. Therefore, a slant line is preset to solve the triangular distribution problem. The slant line passes through the center point of the grain, the magnitude of the slope is fixed, and the direction is variable. The slope is positive when the center point of the grain is on the left side of the Y-axis, and the slope is negative when the center point of the grain is on the right side of the Y-axis.

[0113] The first distance interval of the horizontal mapping of the center point of the grain: Horizontally map the center point of the grain to the Y-axis to form new marked points, arrange the new marked points in order, calculate the distances between adjacent marked points, set a reasonable threshold to classify the new marked points, and obtain the first classification result.

[0114] During implementation, the marked position of the rectangular frame of the grain is expressed as (left, top; right, bottom), and the coordinates of the center point GC of the grain are The coordinates of the new marked point NP1 formed by horizontally mapping the center point of the grain to the Y-axis are Use the sort() function to sort the new marked points NP1, use the diff() function to calculate the distance d1 between adjacent points, set the classification threshold as th. If d1 < th, the adjacent two points are classified into the same category, otherwise they are classified into different categories.

[0115] The second distance interval of the horizontal mapping of the intercept point of the slant line passing through the center point: The intercept of the slant line passing through the center point of the grain on the Y-axis is the new marked point. Arrange the new marked points in order, calculate the distances between adjacent marked points, set a reasonable threshold to classify the new marked points, and obtain the first classification result.

[0116] Horizontally mapping the center point of the grain to the Y-axis cannot incorporate the vertices of the triangle. Further, a slant line y = ax + b is set. The acute angle of the triangle is about 45°. Therefore, a = 0.5 is set. Take the vertical center line of the image (the center line of the wheat ear) as the Y-axis. The slope is positive when the center point of the grain is on the left side of the Y-axis, that is, a = 0.5, and the slope is negative when the center point of the grain is on the right side of the Y-axis, that is, a = -0.5. Then the slant line y = ±0.5x + b passes through the center point GC of the grain Obtain the value of b. The intercept of the slant line passing through the center point on the Y-axis is the new marked point NP2, and the coordinates are (0, b). Use the sort() function to sort the new marked points NP2, use the diff() function to calculate the distance d2 between adjacent points. If d2 < th, the adjacent two points are classified into the same category, otherwise they are classified into different categories.

[0117] Setting of threshold th: The interval between the upper and lower spikelets is approximately half of the length of the grain. In order to ensure the accuracy of classification, one-third of the mean length of all grains on the wheat ear is used as the spikelet classification threshold of each wheat ear, that is, th = mean (bottom-top) / 3.

[0118] Finally, by combining the comparison results of d1, d2 and th, all the grains are accurately classified into different spikelets, and the number of spikelets is calculated. Based on the distribution information of different spikelets, the phenotypic traits related to the spikelet can be further extracted, such as spikelet spacing, looseness ratio, etc. Figure 6 In the process of merging the classification results of d1 and d2, the classification results of the grains according to their spikelets are finally obtained.

[0119] In summary, a preset background plate is designed in the embodiment of the present invention, and a mobile terminal such as a smart phone is combined to obtain wheat ear images in the field to solve the problem of destructive sampling. The embodiment of the present invention combines a deep learning algorithm with an image processing algorithm to achieve fast and accurate measurement of the ear shape (length and width), number of grains per ear, and number of spikelets. Wheat ears and grains are extracted through a deep learning algorithm to solve the problem of weak light resistance of traditional methods. The core algorithm for spikelet number detection is to propose a distance interval fusion between the horizontal mapping of the center point of the grain and the horizontal mapping of the intercept point of the oblique line passing through the center point, so as to achieve accurate classification of different grains on the same spikelet, thereby counting the number of spikelets and improving the accuracy of spikelet identification.

[0120] The present invention also provides a device for measuring wheat ear phenotypic traits in a field environment, as described in the following embodiments. Since the principle of the device to solve the problem is similar to the method for measuring wheat ear phenotypic traits in a field environment, the implementation of the device can refer to the implementation of the method for measuring wheat ear phenotypic traits in a field environment, and the repeated parts will not be repeated.

[0121] Figure 7 FIG. 1 is a schematic diagram of a device for measuring wheat ear phenotypic traits in a field environment according to an embodiment of the present invention. Figure 7 As shown, the wheat ear phenotypic trait measuring device 700 in the field environment includes:

[0122] The data acquisition module 701 is used to acquire a wheat ear image in a field environment through a mobile terminal; the wheat ear image includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat ear in the wheat ear image;

[0123] An image extraction module 702 is used to perform image segmentation and extraction of wheat ears and a scale of a specified size on the wheat ear image to obtain images of the wheat ears and the scale of the specified size;

[0124] The pixel actual size determination module 703 is used to determine the actual size represented by each pixel in the wheat ear image according to the image of the designated size ruler;

[0125] The grain number determination module 704 is used to identify the number of grains in the wheat ear image by using the trained deep learning model; wherein each grain is marked by a rectangular frame;

[0126] The wheat ear length and width determination module 705 is used to determine the minimum circumscribed rectangle with the wheat ear in the middle by the minimum circumscribed rectangle method, and determine the actual length and actual width of the wheat ear according to the size of the minimum circumscribed rectangle;

[0127] The spikelet number determination module 706 is used to determine the spikelet number based on the first distance interval between the points after the center point of the spikelet is projected onto the vertical center line of the image, and the second distance interval between the intersection points of a preset oblique line passing through the center point of the spikelet and the vertical center line of the image; the center point of the spikelet is the center of the rectangular frame mark; each oblique line passes through the center point of the spikelet.

[0128] In one embodiment, the data acquisition module 701 is specifically used to: acquire an image of wheat ears in a field environment placed in front of a preset background plate through a mobile terminal; the preset background plate marks the placement area of ​​the photographed wheat ears and a specified size scale.

[0129] In one embodiment, the pixel actual size determination module 703 is specifically used to: perform local adaptive thresholding, binarization, closing operations, and filling operations on an image of a ruler of a specified size to obtain a binary image of the ruler; determine the area of ​​the ruler in the image based on the binary image of the ruler; and determine the actual size represented by each pixel based on the area of ​​the ruler in the image and the actual area of ​​the ruler.

[0130] In one embodiment, the pixel actual size determination module 703 is specifically used to determine the actual size represented by each pixel according to the area of ​​the ruler in the image and the actual area of ​​the ruler according to the following formula:

[0131]

[0132] Where SSOP is the actual size represented by each pixel, Simage is the area of ​​the ruler in the image, and Sactual is the actual area of ​​the ruler.

[0133] In one embodiment, the image extraction module 702 is specifically used to: use a trained YOLOv8n model to extract images of wheat ears and a specified size scale from multiple wheat ear images to obtain multiple images of wheat ears and a specified size scale; the YOLOv8n model is pre-trained using historical wheat ear images, wheat ears extracted from historical wheat ear images, and specified size scales extracted from historical wheat ear images.

[0134] In one embodiment, the deep learning model is a YOLOv8n model;

[0135] The grain number determination module 704 is specifically used to: use the trained YOLOv8n model to perform grain recognition on multiple wheat ear images to obtain the number of grains in the wheat ear images; wherein the YOLOv8n model is pre-trained using historical wheat ear images and the grains marked in the historical wheat ear images.

[0136] In one embodiment, the wheat ear length and width determination module 705 is specifically used to: determine the minimum enclosing rectangle with the wheat ear in the middle through the minimum enclosing rectangle method; based on the inclination angle of the minimum enclosing rectangle, rotate the wheat ear and the minimum enclosing rectangle to be upright to obtain an upright minimum enclosing rectangle; based on the pixel positions of the four vertices of the upright minimum enclosing rectangle, determine the length and width of the wheat ear in the image; determine the actual length and actual width of the wheat ear based on the actual size represented by each pixel in the wheat ear image and the length and width of the wheat ear in the image.

[0137] In one embodiment, the wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center point of the ear grain on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center point of the ear grain on the right side of the vertical center line of the image is negative;

[0138] The spikelet number determination module 706 is specifically used for: determining the length of each spikelet by using the rectangular frame mark of each spikelet through the minimum circumscribed rectangle method; determining the average length of the spikelets based on the length of each spikelet; determining a threshold based on the average length of the spikelets; the threshold is used to classify the spikelets into spikelets; classifying two spikelets corresponding to a first distance interval less than the threshold as the same spikelet to obtain a first classification result; classifying two spikelets corresponding to a second distance interval less than the threshold as the same spikelet to obtain a second classification result; both the first classification result and the second classification result indicate that the spikelets belonging to the same spikelet are classified; merging the first classification result and the second classification result to obtain a final classification result; the final classification result includes multiple categories, each category including spikelets belonging to the same spikelet.

[0139] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for measuring wheat ear phenotypic traits in a field environment when executing the computer program.

[0140] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for measuring wheat ear phenotypic traits in a field environment is implemented.

[0141] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for measuring wheat ear phenotypic traits in a field environment is implemented.

[0142] In the embodiment of the present invention, a wheat ear image in a field environment is obtained by a mobile terminal, so that the wheat ear image is obtained without destroying the growth of wheat. The wheat ear image includes a specified size ruler, and then the specified size ruler is used to determine the actual size represented by each pixel in the wheat ear image, so that the actual length and actual width of the wheat ear are determined by the actual size represented by each pixel and the pixel features of the wheat ear in the image, and the number of grains per ear of the wheat ear is determined by deep learning. At the same time, the number of spikelets is determined according to the first distance interval between the points after the center point of the grain is projected to the vertical center line of the image, and the second distance interval between the intersection points of the preset oblique line passing through the center point of the grain and the vertical center line of the image. Compared with the prior art, the method proposed in the embodiment of the present invention can combine deep learning and image processing, and simultaneously obtain more wheat ear phenotypic traits in the ear image, thereby effectively improving the accuracy of measuring the phenotypic traits of the wheat ear.

[0143] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0147] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for measuring wheat ear phenotypic traits in a field environment, characterized in that: include: Acquiring an image of wheat ears in a field environment through a mobile terminal; the wheat ear image includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat ears in the wheat ear image; Perform image segmentation and extraction of wheat ears and a scale of specified size on the wheat ear image to obtain images of the wheat ears and the scale of specified size; According to the image of the specified size ruler, determine the actual size represented by each pixel in the wheat ear image; Identify the number of grains in the wheat ear image using a trained deep learning model, where each grain is marked by a rectangular box; The minimum enclosing rectangle method is used to determine the minimum enclosing rectangle with the wheat ear in the middle, and the actual length and width of the wheat ear are determined according to the size of the minimum enclosing rectangle and the actual size represented by each pixel; The number of spikelets is determined based on the first distance interval between the points after the center point of the ear is projected onto the vertical center line of the image, and the second distance interval between the intersection points of a preset oblique line passing through the center point of the ear and the vertical center line of the image; the center point of the ear is the center of the rectangular frame mark; each oblique line passes through the center point of the ear.

2. The method according to claim 1, characterized in that The wheat ear images in the field environment are obtained through the mobile terminal, including: An imaging image of wheat ears placed in front of a preset background plate in a field environment is obtained through a mobile terminal; the preset background plate marks the placement area of ​​the photographed wheat ears and a designated size ruler.

3. The method according to claim 1, characterized in that According to the image of the specified size ruler, determine the actual size represented by each pixel in the wheat ear image, including: Perform local adaptive threshold, binarization, closing operation, and filling operation on the image of the specified size scale to obtain a binarized image of the scale; According to the binary image of the ruler, determine the area of ​​the ruler in the image; The actual size represented by each pixel is determined based on the area of ​​the ruler in the image and the actual area of ​​the ruler.

4. The method according to claim 3, characterized in that According to the area of ​​the ruler in the image and the actual area of ​​the ruler, determine the actual size represented by each pixel, including: According to the following formula, the actual size represented by each pixel is determined based on the area of ​​the ruler in the image and the actual area of ​​the ruler: Where SSOP is the actual size represented by each pixel, Simage is the area of ​​the ruler in the image, and Sactual is the actual area of ​​the ruler.

5. The method according to claim 1, characterized in that The wheat ear image and the image of the specified size ruler are segmented and extracted to obtain the wheat ear and the image of the specified size ruler, including: The trained YOLOv8n model is used to extract images of wheat ears and a specified size scale from multiple wheat ear images to obtain multiple images of wheat ears and a specified size scale; the YOLOv8n model is pre-trained using historical wheat ear images, wheat ears extracted from historical wheat ear images, and specified size scales extracted from historical wheat ear images.

6. The method according to claim 5, characterized in that The deep learning model is a YOLOv8n model; The trained deep learning model is used to identify the number of grains in the wheat ear image, including: The trained YOLOv8n model is used to perform ear-grain recognition on multiple wheat ear images to obtain the number of grains in the wheat ear images; wherein the YOLOv8n model is trained in advance using historical wheat ear images and marked grains in the historical wheat ear images.

7. The method according to claim 1, characterized in that The minimum enclosing rectangle method is used to determine the minimum enclosing rectangle with the wheat ear in the middle. According to the size of the minimum enclosing rectangle and the actual size represented by each pixel, the actual length and width of the wheat ear are determined, including: By using the minimum enclosing rectangle method, determine the minimum enclosing rectangle with the wheat ear in the middle; Based on the tilt angle of the minimum circumscribed rectangle, the wheat ear and the minimum circumscribed rectangle are rotated to stand upright, thereby obtaining an upright minimum circumscribed rectangle; Based on the pixel positions of the four vertices of the erected minimum circumscribed rectangle, the length and width of the wheat ear in the image are determined; The actual length and width of the wheat ear are determined according to the actual size represented by each pixel in the wheat ear image and the length and width of the wheat ear in the image.

8. The method according to claim 1, characterized in that The wheat ear is located in the middle of the image, wherein the slope of the oblique line passing through the center point of the ear grain on the left side of the vertical center line of the image is positive, and the slope of the oblique line passing through the center point of the ear grain on the right side of the vertical center line of the image is negative; Determine the number of spikelets according to the distance interval between the points after the center point of the spike is projected onto the vertical center line of the image, and the distance interval between the intersection points of the preset oblique line passing through the center point of the spike and the vertical center line of the image, including: The length of each ear is determined by the minimum circumscribed rectangle method using the rectangular frame mark of each ear. Based on the length of each ear, determine the average ear length; A threshold is determined based on the average length of the spikelets; the threshold is used to classify the spikelets; Classify two spikelets corresponding to a first distance interval less than a threshold value as the same spikelet, and obtain a first classification result; Classify the two spikelets corresponding to the second distance interval less than the threshold value as the same spikelet, and obtain a second classification result; the first classification result and the second classification result both indicate that the spikelets belonging to the same spikelet are classified; The first classification result and the second classification result are merged to obtain a final classification result; the final classification result includes multiple categories, and each category includes spikelets belonging to the same spikelet.

9. A device for measuring wheat ear phenotypic traits in a field environment, characterized in that: include: A data acquisition module, used for acquiring an image of wheat ears in a field environment through a mobile terminal; the image of wheat ears includes a specified size ruler, and the specified size ruler is used to determine the size and position of the wheat ears in the image of wheat ears; An image extraction module is used to perform image segmentation and extraction of wheat ears and a scale of a specified size on the wheat ear image to obtain images of the wheat ears and the scale of the specified size; A pixel actual size determination module is used to determine the actual size represented by each pixel in the wheat ear image according to an image of a specified size ruler; The grain number determination module is used to identify the number of grains in the wheat ear image through the trained deep learning model; each grain is marked by a rectangular frame; The wheat ear length and width determination module is used to determine the minimum circumscribed rectangle with the wheat ear in the middle by the minimum circumscribed rectangle method, and determine the actual length and actual width of the wheat ear according to the size of the minimum circumscribed rectangle; The spikelet number determination module is used to determine the spikelet number based on the first distance interval between the points after the center point of the spikelet is projected onto the vertical center line of the image, and the second distance interval between the intersection points of the preset oblique line passing through the center point of the spikelet and the vertical center line of the image; the center point of the spikelet is the center of the rectangular frame mark; each oblique line passes through the center point of the spikelet.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images