High-throughput crop seed phenotype collection method and device for artificial intelligence breeding
The high-throughput crop seed phenotype collection method and device addresses the limitations of traditional manual methods by using binocular stereo vision and deep learning to efficiently extract various crop seed traits, improving data collection efficiency and accuracy for crop breeding.
Patent Information
- Application Number
- CN202311812900.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-27
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-12-27
AI Technical Summary
Traditional crop seed phenotype collection methods are labor-intensive, low in accuracy, and limited in parameter extraction, failing to meet the needs of high-throughput and efficient data collection required for large-scale crop breeding.
A high-throughput crop seed phenotype collection method and device utilizing binocular stereo vision, computer vision, and deep learning to extract multiple phenotypes including 100-grain weight, two-dimensional and three-dimensional morphological traits, seed coat color, and hilum characteristics.
Enables automated, high-throughput collection of multiple crop seed phenotypes, providing comprehensive data support for crop breeding and germplasm research, enhancing data collection efficiency and accuracy.
Smart Images

Figure CN117788705B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of plant phenotypes, and in particular to a high-throughput crop seed phenotype collection method and device for artificial intelligence breeding. Background Art
[0002] Plant phenotype is a standardized, digital representation of plant traits, which is affected by genes, environment and other factors. For the breeding field and agricultural management, conducting phenotypic analysis of crops is a key link in understanding the interaction mechanism of "gene-phenotype-environment". High-throughput phenotypic monitoring can significantly accelerate the entire breeding process and has important guiding value for designing breeding research.
[0003] "Seeds are the chips of agriculture." For crop breeding, how to quickly, accurately, and high-throughput collect the various phenotypic parameters of crop seeds (such as length, width, thickness, surface area, volume, color, texture, hilum, 100-grain weight, etc.), obtain sufficient phenotypic group information, and then evaluate its morphological traits, nutritional quality, stress resistance and other indicators, can guide the cultivation and screening of excellent crop varieties, thereby transforming the advantages of germplasm resources into actual production advantages.
[0004] Traditional crop seed phenotype collection usually relies on manual testing, which has the disadvantages of high labor cost, low accuracy, rough description, and limited extraction parameters. It is unable to comprehensively and efficiently evaluate the characteristics of crop seeds and is difficult to meet the needs of large-scale and efficient data collection and analysis. A professional breeding researcher uses a vernier caliper or a micrometer to measure crop seed phenotypic parameters at a rate of about 150 to 200 grains per hour, and the data collection efficiency is extremely low. Therefore, the crop breeding industry urgently needs a high-throughput crop seed phenotype collection system and device for artificial intelligence breeding routes, so that high-throughput, low-cost, non-contact collection of crop seed phenotypic group data can become a reality, thereby providing important support for crop breeding, improvement and production, promoting the development of modern agricultural technology, and promoting the sustainable development of agricultural production.
[0005] In recent years, with the rapid development of computer and big data technology, plant phenotyping research based on computer vision and artificial intelligence technology has ushered in a new era of intelligent phenotyping research. The combination of computer vision and artificial intelligence technology has been widely used in the research field of phenotypic extraction and analysis at the plant and organ levels of crops, but there is still a lack of integrated acquisition systems and devices for high-throughput multi-phenotype extraction of crop grains. Summary of the invention
[0006] The purpose of the present invention is to provide a high-throughput crop seed phenotype collection method and device for artificial intelligence breeding, which can perform integrated extraction of high-throughput multi-phenotypes of crop seeds.
[0007] To achieve the above object, the present invention provides the following solutions:
[0008] A high-throughput crop grain phenotype collection method for artificial intelligence breeding includes: determining the hundred-grain weight of crop grains on a grain carrier plate, and simultaneously obtaining a binocular image of all crop grains on the grain carrier plate; using a minimum circumscribed rectangle to select each crop grain in the binocular image, and extracting a two-dimensional morphological phenotype of each crop grain; determining a three-dimensional point cloud model of all crop grains in the binocular image, and extracting a three-dimensional morphological phenotype of each crop grain; segmenting the binocular image into single-grain crop grains to obtain a plurality of single-grain crop grain images; extracting a seed coat color phenotype of each crop grain from each single-grain crop grain image; segmenting a hilum from each single-grain crop grain image, using a minimum circumscribed rectangle to select the hilum, and determining the hilum phenotype of each crop grain.
[0009] A high-throughput crop seed phenotype collection device for artificial intelligence breeding comprises: a binocular camera, a seed carrier plate, an electronic scale, a control board and a computer; the electronic scale is arranged below the seed carrier plate, and the seed carrier plate is used to place a plurality of crop seeds; the output end of the electronic scale is connected to the input end of the control board, and the output end of the control board is connected to the input end of the computer; the electronic scale is used to sense and generate a pressure signal under the pressure of the crop seeds on the seed carrier plate, and determine the 100-grain weight of the crop seeds on the seed carrier plate according to the pressure signal, and at the same time transmit the 100-grain weight to the control board. The board is uploaded to the computer; the control end of the binocular camera is connected to the output end of the computer, and the output end of the binocular camera is connected to the first input end of the computer; the binocular camera is used to take binocular images of all crop grains on the grain carrier plate under the control of the computer, and upload the binocular images to the computer; the computer is used to extract the two-dimensional morphological phenotype, three-dimensional morphological phenotype, seed coat color phenotype and hilum phenotype of each crop grain on the grain carrier plate according to the binocular images and the high-throughput crop grain phenotype collection method for artificial intelligence breeding.
[0010] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: a high-throughput crop grain phenotype collection method and device for artificial intelligence breeding according to the embodiments of the present invention can collect multiple phenotypes of multiple crop grains at one time, and the multiple phenotypes include 100-grain weight, two-dimensional morphological phenotype, three-dimensional morphological phenotype, seed coat color phenotype and hilum phenotype, thereby realizing the integrated extraction of high-throughput multi-phenotypes of crop grains. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 A flow chart of a high-throughput crop seed phenotype collection method for artificial intelligence breeding provided by an embodiment of the present invention;
[0012] Figure 2A schematic diagram of soybean seed phenotypes involved in an embodiment of the present invention;
[0013] Figure 3 The design model diagram of the hardware device part of the present invention;
[0014] Figure 4 A model diagram of a binocular camera holder at the top of the hardware device of the present invention;
[0015] Figure 5 It is an overall effect diagram of the hardware device part of the present invention;
[0016] Figure 6 A 100-grain weight measurement module which is part of the hardware device of the present invention;
[0017] Figure 7 A system architecture diagram of the software system part of the present invention;
[0018] Figure 8 It is the main interface of the software system in the embodiment of the present invention;
[0019] Fig. 9 It is the interface of the two-dimensional morphological phenotype extraction subsystem in the embodiment of the present invention;
[0020] Fig.10 It is the seed coat color phenotype extraction subsystem interface in the embodiment of the present invention;
[0021] Fig.11 It is the interface of the hilum fine phenotype extraction subsystem in the embodiment of the present invention;
[0022] Fig.12 It is the interface of the three-dimensional morphological phenotype extraction subsystem in the embodiment of the present invention;
[0023] Fig.13 It is the interface of the hilum and seed coat crack detection subsystem in the embodiment of the present invention;
[0024] Fig.14 The following is a process for collecting high-throughput binocular images of soybean seeds in an embodiment of the present invention;
[0025] Fig.15 This is a preprocessing process of a high-throughput soybean seed binocular image in an embodiment of the present invention;
[0026] Fig.16 The high-throughput soybean seed two-dimensional morphological phenotype extraction effect in the embodiment of the present invention;
[0027] Fig.17 The single-grain segmentation effect of the high-throughput soybean grain image in the embodiment of the present invention;
[0028] Fig.18The high-throughput extraction effect of soybean seed coat color phenotype in the embodiment of the present invention;
[0029] Fig.19 The high-throughput soybean seed coat texture complexity extraction effect in the embodiment of the present invention;
[0030] Fig. 20 The high-throughput soybean seed hilum fine phenotype extraction effect in the embodiment of the present invention;
[0031] Fig.21 The high-throughput soybean seed hilum texture complexity extraction effect in the embodiment of the present invention;
[0032] Fig. 22 The effect of establishing a high-throughput soybean seed three-dimensional point cloud model in an embodiment of the present invention;
[0033] Fig.23 The calculation principle of high-throughput soybean seed three-dimensional morphological phenotype in the embodiment of the present invention;
[0034] Fig.24 The high-throughput extraction effect of soybean seed three-dimensional morphological phenotype in the embodiment of the present invention;
[0035] Fig.25 : is a structural diagram of a hilum and crack detection model based on YOLOv5 in an embodiment of the present invention;
[0036] Fig.26 This is the effect of high-throughput soybean seed hilum and crack detection in an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0038] Based on computer vision, digital image processing, deep learning and other technologies, the present invention constructs a high-throughput crop seed phenotype collection method and device based on binocular stereo vision, which includes two parts: hardware device and software system. The extractable parameters include important phenotypes such as length, width, thickness, area, volume, seed coat color, hilum length and width, hilum color, 100-grain weight, texture complexity, seed coat cracks, etc. of crop seeds. More than 20 phenotypes can be extracted, such as Figure 2 As shown. Ultimately, the automated standard collection and analysis process will replace manual seed testing, and the collected crop grain phenotypes will provide more sufficient data support for the development and integration of high-throughput phenotyping and germplasm research based on deep learning.
[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0040] Embodiment 1: Figure 1 As shown, an embodiment of the present invention provides a high-throughput crop grain phenotype collection method for artificial intelligence breeding, comprising:
[0041] Step 1: Determine the hundred-grain weight of the crop grains on the grain-carrying plate, and simultaneously obtain the binocular images of all the crop grains on the grain-carrying plate.
[0042] The formula for determining 100-grain weight is:
[0043]
[0044] Among them, M' is the weight of 100 grains, M plate is the weight of the grain bearing plate, M max is the maximum weighing weight of the electronic scale, E is the electromotive force of the electronic scale power supply, R0 is an irrelevant resistor in the circuit, I is the total circuit current during weighing, and R1 is the access resistor of the pressure sensor inside the electronic scale.
[0045] After obtaining the binocular images of all crop grains on the grain carrying plate, the binocular images are preprocessed. The specific preprocessing operations are: cropping the binocular images to remove blank backgrounds and labels in the binocular images; performing RGB three-channel separation on the cropped binocular images to obtain grayscale images; performing histogram equalization on the grayscale images, and selecting the LoG operator for image sharpening to obtain contour-sharpened images; and superimposing the binocular images and the contour-sharpened images at a ratio of 9:1 to obtain enhanced binocular images.
[0046] Step 2: Use the minimum bounding rectangle to select each crop seed in the binocular image, and extract the two-dimensional morphological phenotype of each crop seed. For example, the specific extraction process is as follows:
[0047] (2-1) Grayscale and binarize the binocular image to separate the crop seeds from the background, thereby obtaining a binary image of the crop seeds.
[0048] (2-2) The crop seed binary image is subjected to two consecutive corrosion operations and two dilation operations to eliminate background noise, thereby obtaining a denoised crop seed binary image.
[0049] (2-3) Calculate the number of connected domains of the denoised crop grain binary image to obtain the crop grain count.
[0050] (2-4) Use the minimum bounding rectangle to select each crop grain in the denoised crop grain binary image, and number each crop grain.
[0051] (2-5) The proportional coefficient between the image pixels and the actual size is obtained through the five-point calibration method.
[0052] (2-6) According to the minimum circumscribed rectangle of each crop seed, the proportional coefficient and the crop seed count, according to the formula Calculate the length and width of each crop seed, as well as the average length and average width of all crop seeds on the seed carrier; where L_soybean i and W_soybean i They represent the length and width of the crop grain numbered i, k is the proportional coefficient, L_pixel i and W_pixel i represents the number of pixels of the length and width of the minimum circumscribed rectangle of the crop grain numbered i, L and W represent the average length and average width of all crop grains on the grain supporting plate, and N represents the crop grain count.
[0053] (2-7) The contour circumference and area of each crop seed are calculated using OpenCV functions.
[0054] (2-8) According to the perimeter and area of each crop seed, according to the formula Calculate the cross-sectional perimeter and cross-sectional area of each crop seed, as well as the average cross-sectional perimeter and average cross-sectional area of all crop seeds on the seed carrier; where C 2d _soybean i and S 2d _soybean i represent the cross-sectional perimeter and cross-sectional area of the crop seed numbered i, respectively. 2d _pixel i and S 2d _pixel i Respectively represent the contour circumference and area of the crop grain numbered i, C 2d and S 2d are the average cross-sectional circumference and average cross-sectional area of all crop seeds on the seed-carrying plate, respectively.
[0055] (2-9) A plane rectangular coordinate system is established with the vertex of the upper left corner of the minimum circumscribed rectangle of each crop seed as the coordinate origin. According to the formula Calculate the two-dimensional center coordinates of each crop seed; where (rec_x j , rec_y j ) is the coordinate of the jth vertex in the minimum circumscribed rectangle of the crop grain, (X 2d , Y 2d ) is the two-dimensional center coordinate of the crop grain.
[0056] (2-10) The length and width of each crop grain, the cross-sectional perimeter and cross-sectional area of each crop grain, and the two-dimensional center coordinates of each crop grain together constitute the two-dimensional morphological phenotype of each crop grain.
[0057] Step 3: Determine the 3D point cloud model of all crop seeds in the binocular image, and extract the 3D morphological phenotype of each crop seed. Exemplarily, the 3D morphological phenotype extraction process is:
[0058] (3-1) Perform voxelized grid downsampling on the point cloud in the three-dimensional point cloud model.
[0059] (3-2) Perform normal vector estimation calculation for local surface fitting based on KD tree, and use density-based DBSCAN clustering to cluster the downsampled point cloud so that all point clouds corresponding to each crop grain are clustered into one category.
[0060] (3-3) The cluster center of each class is used as the geometric center of the crop grain to obtain the three-dimensional center coordinates of each crop grain.
[0061] (3-4) According to the three-dimensional center coordinates of each crop seed, the thickness, average diameter, volume and surface area of each crop seed are calculated using the following formula:
[0062] H_soybean i =2·(max(Cloud i (Z))-min(Cloud i (Z)))
[0063]
[0064]
[0065]
[0066] Among them, H_soybean i , R 3d _soybean i 、V 3d _soybean i and S 3d _soybean i They represent the thickness, average diameter, volume and surface area of the crop grain numbered i, respectively. Cloud is the set of all point clouds in the same cluster. N c is the number of points in the same cluster, Cloud r (x r ,y r , z r ) is the coordinate of the rth point cloud in the same cluster, (X3d , Y 3d , Z 3d ) is the three-dimensional center coordinate of the crop grain, Cloud i (Z) represents the point cloud data of the crop grain numbered i in the Z direction.
[0067] (3-5) According to the formula Determine the average seed thickness, average diameter, average volume and average surface area of all crop seeds on the seed carrier; where H, R, V and S 3d They are average kernel thickness, average diameter, average volume and average surface area respectively.
[0068] (3-6) The thickness, average diameter, volume, and surface area of each crop seed together constitute the three-dimensional morphological phenotype of each crop seed.
[0069] Step 4: Segment the binocular image into single-grain crop grains to obtain a plurality of single-grain crop grain images.
[0070] In high-throughput conditions, non-precise parameters such as grain length and width can be extracted. However, if we want to extract the color characteristics and hilum characteristics of each grain, the error will be large if we continue to use high-throughput images (binocular images). Therefore, it is necessary to segment the single grain of the crop in the entire high-throughput image. 2d , Y 2d ) is set as the center point of a single crop grain, and a 100×100 pixel area around the point is intercepted to complete the segmentation of the high-throughput image, which is convenient for subsequent fine parameter extraction.
[0071] Step 5: Extract the seed coat color phenotype of each crop seed from each single crop seed image. Exemplarily, the specific process of seed coat color phenotype extraction is as follows:
[0072] (5-1) The K-Means clustering algorithm is used to cluster the pixels in the single-grain crop grain image into multiple different color clusters.
[0073] (5-2) Based on the multiple different color clusters obtained by clustering, a cluster histogram is drawn to obtain the color spectrum band of a single-grain crop seed image.
[0074] (5-3) The background color is removed from the color spectrum band, and the color category with the largest number of pixels in the color spectrum band after the background color is removed is used as the RGB color of the seed coat of the crop grain.
[0075] (5-4) The mask concept is used to remove the black background with RGB values less than or equal to [50, 50, 50] in the single-grain crop seed image, and the RGB values of the remaining background pixels are set to 0. At the same time, the rest of the background is grayscaled to obtain a single-grain crop seed grayscale image.
[0076] (5-5) According to the formula Calculate the mean of the local variance and the 95% quantile of the local variance of the single-grain crop grain grayscale image; where mean_var is the mean of the local variance, max_var is the 95% quantile of the local variance, var l is the variance of the lth non-zero value, is the mean of the variance of all non-zero values, n is the number of pixels with non-zero variance values, P 95 () is the 95% quantile.
[0077] (5-6) Based on the mean of the local variance and the 95% quantile of the local variance of the single-grain crop grain grayscale image, the formula Calculate the glossiness of the seed coat of crop seeds; where glossiness is the glossiness.
[0078] (5-7) Determine the grayscale co-occurrence matrix of a single crop grain image, and calculate the contrast, homogeneity, correlation, and energy based on the grayscale co-occurrence matrix.
[0079] (5-8) Comprehensive contrast, homogeneity, correlation and energy, according to the formula Calculate the texture complexity of the seed coat of crop seeds; where C is texture complexity, contrast is contrast, homogeneity is homogeneity, correlation is correlation, and energy is energy.
[0080] (5-9) The RGB color of the seed coat of each crop seed, the glossiness of the seed coat of each crop seed, and the texture complexity of the seed coat of each crop seed constitute the seed coat color phenotype of each crop seed.
[0081] Step 6: Segment the hilum from each single crop grain image, select the hilum using the minimum bounding rectangle, and determine the hilum phenotype of each crop grain. Exemplarily, the process of determining the hilum phenotype is as follows:
[0082] (6-1) The single-grain crop grain image is grayed and inverted into a binary image, and then dilated and eroded once respectively when the rectangular structure element size is 2×2 to obtain a single-grain crop grain binary image with noise eliminated;
[0083] (6-2) Use the minimum enclosing rectangle to select the hilum in the binary image of the single-grain crop grain after noise elimination;
[0084] (6-3) Select the minimum circumscribed rectangle of the hilum according to the formula Know Calculate the length, width, perimeter, area and center coordinates of the hilum in a single crop seed; among them, L_hilumfeature, W_hilumfeature, C_hilumfeature, S_hilumfeature and (Xh 2d , Yh 2d ) are the length, width, perimeter, area and center coordinates of the hilum in a single crop seed, Lh_pixel and Wh_pixel are the number of pixels of the length and width of the minimum circumscribed rectangle of the selected hilum, C_hilumfeature and S_hilumfeature are the perimeter and area of the hilum in a single crop seed, k is the proportional coefficient, (rh_x p , rh_y p ) is the coordinate of the pth vertex in the minimum circumscribed rectangle of the selected hilum;
[0085] (6-4) The selection range is (Xh 2d -5, Yh 2d -5) to (Xh 2d +4, Yh 2d +4), using the formula Calculate the RGB value of the hilum color; where [h_red, h_green, h_blue] is the RGB value of the hilum color, [r q , g q , b q ] is the RGB value of the qth pixel in the selected range;
[0086] (6-5) Based on the single-grain crop grain image, the texture complexity extraction algorithm based on 8-neighborhood LBP is used according to the formula Calculate the texture complexity of the hilum; where C h is the texture complexity of the hilum, M and N are the corresponding numbers of rows and columns after the LoG edge detection grayscale image is divided into blocks, Hist m,n (k′) is the k′th binary number in the histogram corresponding to the block in the mth row and nth column of the block image, and block_size is the block size;
[0087] (6-6) The length, width, circumference, area, center coordinates of the hilum, RGB value of the hilum color, and texture complexity of the hilum constitute the hilum phenotype of crop seeds.
[0088] The high-throughput crop seed phenotype collection method for artificial intelligence breeding of the present invention also includes: using YOLOv5-s to detect the hilum and seed coat crack of each crop seed in the binocular image. The two-dimensional morphological phenotype, three-dimensional morphological phenotype, seed coat color phenotype and hilum phenotype of the crop seeds are displayed and exported in a table form.
[0089] The method of the present invention will be described below using high-throughput phenotypic collection of soybean seeds as an example. The method mainly includes the following steps: S1, collecting high-throughput binocular images of soybean seeds; S2, preprocessing the high-throughput binocular images of soybean seeds; S3, calling a series of algorithm models for high-throughput phenotypic extraction of soybean seeds; S4, systematically displaying phenotypic data and exporting them in a table format.
[0090] Furthermore, the process S1 of collecting high-throughput binocular images of soybean seeds includes: placing three labels of variety number, location, and year next to the seed carrier plate, and capturing a binocular image of the soybean seeds, wherein the image includes the three labels of variety number, location, and year.
[0091] Furthermore, the process S2 of preprocessing the high-throughput soybean seed binocular image includes:
[0092] S21. Use a text recognition method based on OCR technology to identify the information in the three labels as an important supplementary index of soybean seed phenotype, so as to classify and name the soybean seed images. Use the CnStd Chinese text detection library and the CnOcr Chinese OCR recognition library, adjust the confidence threshold to 0.8, and realize the recognition of label information.
[0093] S22, cropping a specific area of the initial image to remove the blank useless background and label parts in the original image, so that the subsequent image processing process focuses on the area where the soybean grains are located;
[0094] S23, the cropped image is separated into RGB three channels, and the grayscale image is processed by histogram equalization. In order to make the edge contour of the grain and the hilum contour in the image clearer, the LoG operator is selected for image sharpening, and the original image and the contour sharpening image are superimposed at a ratio of 9:1 to obtain an enhanced image, completing the preprocessing of the high-throughput soybean grain binocular image, such as Fig.15 shown.
[0095] Furthermore, in the process S3, calling a series of algorithm models for high-throughput soybean seed phenotype extraction includes:
[0096] S31. By using the relevant algorithms of Python-OpenCV, the two-dimensional morphological parameters of soybean seeds are extracted. First, the enhanced image is grayed and binarized to separate the soybean seeds from the background, and the seeds are detected and segmented. The binarization threshold is set to 50, and the size of the rectangular structure element is set to 5×5. Two corrosion and two expansion operations are performed continuously to eliminate the tiny background noise, so that the white area in the binary image is the actual outline of the soybean seeds. After morphological processing, the number of connected domains of the binary image is calculated, and the seeds are counted to obtain the number of soybean seeds in the image as N. Each soybean is framed and labeled with a minimum circumscribed rectangle. Suppose the length and width of the soybean seed numbered i are L_soybean respectively. i and W_soybean i The length and width of the corresponding minimum bounding rectangle are L_pixel i and W_pixel i , and then the five-point calibration method is used to obtain the proportional coefficient k between the image pixels and the actual size (unit: mm), which is approximately 0.125. Then, the length and width of each soybean seed and the average length L and average width W of all soybeans are extracted using formula (1).
[0097]
[0098] Since the contour curve of soybean seeds is closed, the cross-sectional perimeter and cross-sectional area of soybean seeds can be calculated by the cv2.arcLength and cv2.contourArea functions in the OpenCV library, as shown in formula (2).
[0099]
[0100] Among them, C 2d _soybean i and S 2d _soybean i are the cross-sectional perimeter (unit: mm) and cross-sectional area (unit: mm) of the soybean seed numbered i. 2 ), C 2d _Pixel i and S 2d _pixel i are the values of soybean seed contour perimeter and area calculated by OpenCV function, C 2d and S 2d are the average cross-sectional circumference and average cross-sectional area of all soybeans, respectively.
[0101] Since the outline of the soybean seed is approximately an ellipse, the approximate center coordinates of the soybean seed can also be obtained by the intersection of the two diagonals of the minimum circumscribed rectangle. A plane rectangular coordinate system is established with the upper left corner of the image as the origin. The center coordinates of the soybean seed are (X 2d , Y 2d ), the coordinates of the four vertices of its minimum circumscribed rectangle are (rec_x1, rec_y1), (rec_x2, rec_y2), (rec_x3, rec_y3) and (rec_x4, rec_y4), respectively, in pixels, as shown in formula (3).
[0102]
[0103] The results of soybean seed two-dimensional morphological phenotype extraction using S31 are as follows: Fig.16 shown.
[0104] S32: In the case of high throughput, non-precise parameters such as seed length and width can be extracted. However, if we want to extract the color characteristics and hilum characteristics of each seed, the error will be large if we continue to use high-throughput images. Therefore, it is necessary to segment the entire high-throughput soybean image into single soybeans. The center coordinates (X 2d , Y 2d ) is set as the center point of the single grain image, and the 100×100 pixel area around the point is intercepted to complete the segmentation of the high-throughput image, which is convenient for subsequent fine parameter extraction. Fig.17 shown.
[0105] S33, read the single grain image obtained in S32, and then use a seed coat color RGB value extraction algorithm based on K-Means clustering to cluster the pixels in the image into K different color clusters. The RGB values between these clusters are used as features, which can better characterize the distribution of the RGB values of the image. Set the number of clusters to 10 and the number of iterations to 200, and obtain the color spectrum of the entire image by drawing a cluster histogram. After removing the background color, the color category with the largest number of pixels in the color spectrum is approximated as the RGB color of the entire soybean seed coat, such as Fig.18 shown.
[0106] Then the glossiness of soybean seeds is obtained by the variance of pixel values in the area. The mask idea is used to remove the nearly black background whose RGB value is not greater than [50, 50, 50], and the RGB value of all background pixels is set to 0 so that they do not participate in the calculation of glossiness, and the remaining part is grayed out. The window size window_size used when calculating the local variance is designed to be 10, the symbol of the local variance value is var, the number of pixels with non-zero variance value is n, and the variance of the i-th non-zero value is var i, the mean variance of all non-zero values is The mean of the local variance is mean_var, and the 95% quantile of the local variance is max_var. The calculation method of mean_var and max_var is shown in formula (4).
[0107]
[0108] Among them, max_var is the upper quantile of the local variance, which can reflect the range of local brightness changes in the image, that is, the brightness change range of the high brightness area in the soybean seed outline. The glossiness is based on the ratio of the standard deviation of the local variance to the upper quantile, which can be calculated using formula (5), as follows: Fig.19 shown.
[0109]
[0110] Finally, the texture complexity of the seed coat is calculated using a method based on the gray-level co-occurrence matrix (GLCM). With the upper left corner of the image as the origin, a rectangular coordinate system is established. Let the number of grayscale levels of the grayscale image I be N (usually N = 256), I (m, n) represents the grayscale value i corresponding to the point when the pixel coordinates in the image are (m, n), and I (m + dcosθ, n + dsinθ) represents the grayscale value j of the corresponding pixel with an angular distance d in the direction of θ from the corresponding point with pixel coordinates (m, n) in the image. Then the GLCM matrix element GLCM i,j The calculation method of (θ, d) is shown in formula (6):
[0111]
[0112] Based on the obtained GLCM matrix, the contrast, homogeneity, correlation, ASM, energy and other features can be calculated. Among them, contrast is the product of the square of the difference between the off-diagonal elements in the gray-level co-occurrence matrix and the sum of the gray-level co-occurrence matrix, which indicates the contrast of pixel pairs with different gray levels in the image; homogeneity is the sum of the diagonal elements in the gray-level co-occurrence matrix, which indicates the frequency of occurrence of the same gray-level pairs; correlation is the sum of the products of each element in the gray-level co-occurrence matrix, which indicates the gray-level change trend between adjacent pixels in the image. ASM is the sum of the squares of each element in the gray-level co-occurrence matrix, which indicates the frequency of occurrence of pixel pairs in the image, and its square root is called energy. Generally speaking, the more complex the texture, the more drastic the gray-level change in the seed coat, the greater the contrast and correlation, and the smaller the homogeneity and ASM. Therefore, the complexity of the surface texture of the soybean seed coat can be comprehensively measured by comparing the above indicators. The calculation methods of contrast, homogeneity, correlation and energy are shown in formulas (7) to (10) respectively.
[0113]
[0114]
[0115]
[0116]
[0117] Where μ and v are the row mean and column mean in the GLCM matrix, σ i and σ j are the row standard deviation and column standard deviation in the GLCM matrix, respectively. By integrating the four indicators of contrast, homogeneity, correlation and energy and evenly distributing the indicator weights, the calculation formula for constructing texture complexity C can be obtained, as shown in formula (11).
[0118]
[0119] S34. The single soybean image obtained in S32 is grayed and inverted to be binarized, and the binarization threshold is set to 180. Then, it is expanded and eroded once when the rectangular structure element size is 2×2 to obtain a binary image with noise eliminated. After morphological processing, the background, grains (excluding the hilum), and hilum in the image can be segmented into three parts, and then the hilum is selected using the minimum circumscribed rectangle. Let the length and width of the hilum be L_hilumfeature and W_hilumfeature respectively, and the number of pixels of the corresponding minimum circumscribed rectangle length and width are Lh_pixel and Wh_pixel respectively, and C_hilumfeature and S_hilumfeature are the circumference (unit: mm) and area (unit: mm) of the hilum respectively. 2 ), Ch_pixel and Sh_pixel are the values of the soybean hilum contour circumference and area calculated by the OpenCV function, respectively. Then, the proportional coefficient k obtained in S31 can be used to calculate the morphological phenotypes of the hilum, such as length, width, circumference and area, as shown in formula (12).
[0120]
[0121] The approximate center coordinates of the hilum can be obtained by the intersection of the diagonals of its minimum circumscribed rectangle. Suppose the center coordinates of the hilum are (Xh 2d , Yh 2d), the coordinates of the four vertices of its minimum circumscribed rectangle are (rh_x1, rh_y1), (rh_x2, rh_y2), (rh_x3, rh_y3) and (rh_x4, rh_y4), respectively, in pixels, as shown in formula (13).
[0122]
[0123] Since the area of the hilum is small and the color of the hilum is more concentrated than the color of the seed coat, and there is no interference from complex textures, the present invention directly calculates the center coordinates of the hilum (Xh 2d , Yh 2d ) is obtained by taking the average value of RGB of all pixels within a certain range around it. To ensure that the range does not contain the corresponding pixels of the seed coat, the range can be (Xh 2d -5, Yh 2d -5) to (Xh 2d +4, Yh 2d +4) are included in the calculation. Assume that the RGB value of the color of the hilum is [h red ,h green ,h blue ], the RGB value of the i-th pixel in the range is [r i , g i , b i ], the calculation method of the hilum color is shown in formula (14).
[0124]
[0125] The results after processing by the above-mentioned hilum morphology and color phenotype extraction algorithm are as follows Fig. 20 shown.
[0126] For the texture complexity of the hilum, C h The present invention uses a texture complexity extraction algorithm based on 8-neighborhood LBP, performs LBP calculation on the single soybean grayscale image after LoG edge detection, sets the distance from the sampling point to the center point to 1, obtains the LBP mode image, then sets the block size block_size to 2, calculates the texture complexity of each block and performs normalization processing, and finally obtains the average texture complexity of all blocks. Texture complexity C of the hilum h The calculation formula of is shown in formula (15).
[0127]
[0128] Among them, M and N are the corresponding numbers of rows and columns after the LoG edge detection grayscale image is divided into blocks, Hist i , j(k) is the kth binary number in the histogram corresponding to the block in the i-th row and j-th column of the block image. The extraction effect of the hilum texture complexity is as follows: Fig.21 shown.
[0129] S35. Read in the corrected and rectified left and right eye images, then perform binocular matching through the SGBM global matching algorithm, perform cost aggregation to obtain a more accurate disparity value, and obtain the final disparity map after disparity optimization. Then, calculate the projection matrix Q based on the internal and external parameters of the camera, and obtain the corresponding depth map after normalization. After obtaining the disparity map and depth map, call the cv2.reprojectImageTo3D() function to convert them into three-dimensional point cloud data. For the initial three-dimensional point cloud data, first perform statistical filtering to remove noise, where the number of domain points is 60 and the standard deviation multiple is 2. Then remove the black background points in the point cloud whose RGB values are close to [255, 255, 255], crop the blank part and the label part, and obtain the processed high-throughput soybean grain three-dimensional point cloud model, as shown in the figure. Fig. 22 shown.
[0130] After obtaining the three-dimensional point cloud model of high-throughput soybean seeds, the three-dimensional morphological phenotype can be extracted. First, the point cloud is voxelized and grid downsampled, and the downsampled voxel size is set to 0.05. The normal vector estimation calculation of the local surface fitting based on the KD tree is performed, and the search radius is set to 0.1 and the maximum number of neighboring points is set to 30. Finally, the point cloud data is clustered using density-based DBSCAN clustering, and the radius of the point cloud density is set to 2.1 and the minimum number of points of the point cloud density is set to 50. In the end, all the point clouds corresponding to each soybean will be clustered into one category, and the number of cluster centers is equal to the number of soybean seeds. Since the seed can be approximately regarded as an oblate ellipsoid, we can approximately consider the cluster center to be the geometric center of the soybean, so that the central coordinates of the soybean seed (X 3d , Y 3d , Z 3d ). Since the generated three-dimensional point cloud is the upper surface of the soybean, its thickness can be estimated by twice the numerical range of the upper surface point cloud in the Z direction, and the numerical range of the XY direction can also be considered as the length and width of the grain, such as Fig.23 shown.
[0131] Let the thickness of soybean seed numbered i be H_soybean i , whose average diameter is R 3d _soybean i , the volume and surface area are V 3d _soybean i and S 3d _soybean iThe set of all point clouds in the same cluster is Cloud, numbered as N, and the number of points is c , the coordinates of the point numbered j are Cloud j (x j ,y j , z j ), the data in the XYZ direction are Cloud(X), Cloud(Y) and Cloud(Z) respectively. The relevant calculation methods are shown in formulas (16) to (21).
[0132] X i =max(Cloud i (X))-min(Cloud i (X)) (16)
[0133] Y i =max(Cloud i (Y))-min(Cloud i (Y)) (17)
[0134] H_soybean i =2·(max(Cloud i (Z))-min(Cloud i (Z))) (18)
[0135]
[0136]
[0137]
[0138] Suppose the number of soybeans is N, the average seed thickness is H, the average diameter is R, the average volume is V, and the average surface area is S. 3d , then the relevant calculation method is shown in formula (22). The extraction effect of high-throughput soybean seed three-dimensional morphological phenotype is as follows Fig.24 shown.
[0139]
[0140] S36. Use YOLOv5-s under the PyTorch framework to detect the hilum and seed coat cracks of the high-throughput soybean seed images obtained in S2. The model structure of YOLOv5-s is as follows: Fig.25As shown, the official yolov5s.pt is used as the pre-training weight, the input image size is set to 640×640, the SGD optimizer is used, the attention mechanism is introduced, the learning rate is set to 0.01, the batch_size is 64, and the number of epochs is 500. In this example, the average mAP of the hilum and crack 50 The detection accuracy of the hilum reached 0.925, the detection accuracy of the seed coat reached 0.995, and the detection accuracy of the seed coat crack reached 0.854, which can realize the extraction of the hilum morphological phenotype and the identification of cracks in most cases, such as Fig.26 shown. Fig.26 Part (a) shows the effect of high-throughput soybean seed hilum detection. Fig.26 Part (b) shows the high-throughput soybean seed hilum morphological phenotype extraction and crack identification.
[0141] Further, the system display of phenotypic data in the process S4 and exporting it in a table form includes: S41, the system automatically saves all phenotypic data involved in S3 into a table in a system-specified folder. S42, after all algorithm call functions are completed in the subsystem, the extracted phenotypic data can also be displayed in a table form below on the interface.
[0142] The crop seed phenotype extraction algorithm involved in the present invention includes four categories of seed 2D morphological phenotype, 3D morphological phenotype, seed coat color phenotype and hilum fine phenotype. Among them, the 2D morphological phenotype includes the length, width, cross-sectional area, cross-sectional perimeter, etc. of the seed; the 3D morphological phenotype includes the thickness, average radius, surface area, volume, etc. of the seed; the seed coat color phenotype includes the RGB value of the seed coat color, surface glossiness, texture complexity, etc.; the hilum fine phenotype includes the length, width, area, perimeter, color, texture complexity, etc. of the hilum.
[0143] Compared with the prior art, the present invention has the following beneficial effects:
[0144] (1) High throughput: The present invention can collect the phenotypes of 100 crop seeds at one time. It only takes about 3 minutes from taking samples to obtaining all data. The efficiency of phenotype collection is about 30 times higher than that of manual work.
[0145] (2) Multiple phenotypes: The present invention provides a method for extracting multiple phenotypes of crop seeds, which can collect phenotypic characteristics in multiple dimensions including morphology, color, hilum, texture, cracks, 100-grain weight, etc. The types of extractable phenotypes are greatly increased compared with ordinary seed testing instruments on the market.
[0146] Embodiment 2: The embodiment of the present invention provides a high-throughput crop grain phenotype collection device for artificial intelligence breeding, including: a binocular camera, a grain carrier plate, an electronic scale, a control board and a computer.
[0147] The electronic scale is arranged below the grain carrier plate, and the grain carrier plate is used to place multiple grains of crops. The output end of the electronic scale is connected to the input end of the control board, and the output end of the control board is connected to the input end of the computer; the electronic scale is used to sense and generate a pressure signal under the pressure of the crop grains on the grain carrier plate, and determine the hundred-grain weight of the crop grains on the grain carrier plate according to the pressure signal, and upload the hundred-grain weight to the computer through the control board. The control end of the binocular camera is connected to the output end of the computer, and the output end of the binocular camera is connected to the first input end of the computer; the binocular camera is used to take binocular images of all crop grains on the grain carrier plate under the control of the computer, and upload the binocular images to the computer. The computer is used to extract the two-dimensional morphological phenotype, three-dimensional morphological phenotype, seed coat color phenotype and hilum phenotype of each crop grain on the grain carrier plate according to the binocular image using the high-throughput crop grain phenotype collection method for artificial intelligence breeding of embodiment 1.
[0148] In one example, the high-throughput crop seed phenotype collection device for artificial intelligence breeding also includes: a seed testing box. The binocular camera, the seed carrying plate, the electronic scale and the control board are all arranged inside the seed testing box; the binocular camera is fixed on the top of the seed testing box, the electronic scale and the seed carrying plate are both located at the bottom of the seed testing box, and the camera of the binocular camera is aimed at the seed carrying plate.
[0149] The high-throughput crop seed phenotype collection device for artificial intelligence breeding constructed in the embodiment of the present invention comprises two parts: a hardware device and a software system. The software system inside the computer is used to execute the high-throughput crop seed phenotype collection method for artificial intelligence breeding in the first embodiment.
[0150] The hardware device uses SolidWorks for model design, such as Figure 3 As shown, the box shell adopts black acrylic plate as the main material. The top cover 1 adopts a detachable embedded design with a binocular camera holder 2, as shown in FIG. Figure 4 As shown, it is convenient to install and fix and disassemble and replace the binocular camera. The box door 4 of the seed test box can rotate around the vertical axis, which is convenient for adjusting the internal accessories and taking and placing crop grains when using the seed test box. Figure 3 The rear side 3 of the test box is also designed with a detachable magnetic outlet plate 5, which is convenient for the USB cable of the binocular camera to be connected to an external computer from the inside of the box. Figure 3 In addition, a ring light strip is installed around the inner wall of the test box to provide stable lighting conditions and prevent external lighting conditions from affecting image acquisition. Figure 5 shown.
[0151] The binocular camera is the core component of the hardware device. The test box is designed with a binocular camera holder located in the center of the top cover inside the box. The holder has 6 equally spaced apertures with a diameter of 6.5 mm, and the binocular camera can be fixed with screws. The binocular camera carried by the present invention is a MV-LD-12-5M-C high-precision industrial lens from MindVision, with a single-lens pixel of 5 million and a focal length of 12 mm. The two lenses are 7 cm apart.
[0152] As for the method of taking crop grains, the method selected by the present invention is to use a grain carrier plate to fix and place the crop grains arranged in a 10×10 standardized manner. The grooves of the grain carrier plate can fix the posture and position of the grains, and the standardized arrangement makes the crop grains non-sticky and clearly arranged, laying the foundation for collecting crop grain phenotypes using computer vision methods.
[0153] The hundred-grain weight of crop grains can be measured by a full-bridge pressure sensor and an HX711 chip. The HX711 collects the voltage difference between the 2nd and 4th lines of the pressure sensor, performs a 128-fold gain, and then performs an AD conversion to obtain the final AD value. Since the full-bridge pressure sensor used is a linear sensor, we can calculate the collected AD data by a one-dimensional linear equation, that is, the weight value = A*AD+B (A is the slope of the linear curve of the pressure sensor, and B is the intercept of the linear curve of the pressure sensor). After the above calculation, an accurate weight value can be obtained. This value is subtracted from the weight of the tray and divided by the crop to obtain the hundred-grain weight of the crop grains. Finally, the Arduino Uno control board is used as a lower computer to read the value measured by the HX711 chip, and the data is transferred to the computer through the USB interface. The electronic scale can also calculate the hundred-grain weight by the determination formula of the hundred-grain weight in step one of Example 1.
[0154] Connect the corresponding nodes of the bridge pressure sensor inside the electronic scale as follows Figure 6 As shown in the figure, connect the corresponding interface of HX711 module to realize the connection and signal transmission between electronic scale and HX711. In addition, the VCC, GND, SCK and DOUT of HX711 module should be connected to 5V, GND, D9 and D10 of Arduino Uno controller respectively, and the USB port of Arduino Uno controller should be connected to the computer to realize data transmission.
[0155] The software system used in this invention is a Windows desktop application software developed based on Python 3.9. The system architecture is shown in the figure below: Figure 7As shown. It mainly uses PyQt5 and Python-OpenCV as auxiliary libraries, and needs to be connected to the hardware device with a USB cable to run. The CPU model of the connected computer is Intel Core i7-9750H, and the operating system is Windows 10. The training environment of the deep learning model involved in some built-in algorithms is based on the PyTorch 1.13.1 framework, using a GPU server of the Linux system. The server is equipped with 2 NVIDIARTX 3090ti GPUs, the CPU model is AMD 3900xt, and the memory is 64G DDR4. The software system of the present invention can mainly realize the following four functions:
[0156] (1) Software function 1: Capture and display high-throughput crop grain images and control the operation of the binocular camera;
[0157] (2) Software function 2: Achieve high-throughput single-grain segmentation of crop grains and save the segmented images;
[0158] (3) Software function 3: Call the high-throughput crop seed phenotype extraction algorithm to complete the extraction of crop seed phenotype data such as morphology, color, hilum, texture, etc., and display and export them in table form;
[0159] (4) Software function 4: Save the crop seed phenotype collection results and exit the system.
[0160] The software system operation demonstration screenshot of the present invention is as follows Figure 8 to Figure 13 shown.
[0161] The operation of the acquisition device of the embodiment of the present invention when acquiring high-throughput soybean seed binocular images in the execution process S1 is as follows:
[0162] S11. Select a soybean seed sample bag. The soybean seed samples in the sample bag should be of the same variety. Pour all the soybean seeds in the bag into a pre-prepared container. The container can be a small plastic bucket or a glass dish.
[0163] S12, placing the soybean seed carrier plate horizontally on the table, aligning the opening of the container with the middle of the seed carrier plate, pouring an appropriate amount of soybean seeds on the seed carrier plate, shaking the carrier plate slowly and smoothing it with hands to ensure that there is a soybean seed in all the grooves of the seed carrier plate, and pouring the excess soybean seeds back into the container to ensure that there are 100 soybean seeds on the seed carrier plate in the end;
[0164] S13, connect the binocular camera USB cable of the hardware device to the computer, and turn on the power of the ring light strip inside the hardware device to provide stable lighting conditions, then transfer the grain carrier plate to the electronic scale at the bottom of the hardware device box, and place three labels of variety number, location, and year next to the grain carrier plate. Since the bottom of the device is covered with black matte velvet, the influence of reflected light can be eliminated;
[0165] S14. Open the software system in the computer, preview the binocular camera shooting picture, fine-tune the position of the seed carrier plate at the bottom of the box, so that the whole picture of the seed carrier plate can be seen in the left and right camera pictures, close the box door after confirming that the position is correct, and use the mouse to click the "Capture Image" button in the software system. The system will save the captured photos to the system folder path, and the high-throughput soybean seed binocular image acquisition can be completed. Fig.14 shown.
[0166] The operation modes of executing the soybean seed phenotype collection method using the software system in the computer of the present invention are as follows:
[0167] Process S31: Click the "2D morphological phenotype extraction" button in the main interface to enter the high-throughput soybean seed 2D morphological phenotype extraction submodule. Click "Import image" to select the cropped image in the image folder, and then display it in the image display area of the interface. Then click the "Grayscale", "Binary", and "Grain Count" buttons respectively to complete the preliminary processing of the image. Then click the "Parameter Extraction" button, and the software can complete the call of the algorithm described in this step.
[0168] Process S33: Click the "Seed Coat Color Phenotype Extraction" button in the main interface to enter the high-throughput soybean seed coat color phenotype extraction submodule. Click "Import Image" to select the cropped image in the image folder, and then display it in the image display area of the interface. Click the "Parameter Extraction" button, and the software can complete the call of the algorithm described in this step.
[0169] Process S34: Click the "Hilum Fine Phenotype Extraction" button in the main interface to enter the high-throughput soybean seed hilum fine phenotype extraction submodule. Click "Import Image" to select the single grain image in the / each / original folder, and then display it in the image display area of the interface. Then click the "Parameter Extraction" button, and the software can complete the call of the algorithm described in this step.
[0170] Process S35: Click the "3D morphological phenotype extraction" button in the main interface to enter the high-throughput soybean seed 3D morphological phenotype extraction submodule. Click "Import left image" and "Import right image" respectively to select the left image left.png and the right image right.png in the image folder to complete the reading of the binocular image, and the path of the read-in image is displayed on the right side of the button. After that, click the "Binocular Image Matching" button, the software can call the SGBM binocular matching algorithm to generate a disparity map and a depth map, and obtain two point cloud data files, ply and pcd, which are saved in the / image / cloud folder, and the general situation of the program operation can be viewed through the progress bar. Click the "View Point Cloud" button to select any point cloud file to open and view. Click the "Parameter Extraction" button, and the software can complete the call of the algorithm described in this step.
[0171] Process S36: Click the "Hillum Crack Phenotype Detection" button in the main interface to enter the high-throughput soybean seed hilum and seed coat crack phenotypic detection submodule. Click the "Import Image" button to select any image to read and display in the image display area above. Then click the "Start Recognition" button, and the software can call the test code of the model trained in this step to complete the phenotypic detection of the hilum and seed coat cracks of the image.
[0172] Process S4: S41. Click the "Parameter Extraction" button on the software system, and the system automatically saves all phenotypic data involved in S3 to a table in the system-specified folder. S42. After all algorithm call functions are completed in the subsystem, click the "Data Display" button in the operation interface to display the extracted phenotypic data in a table below the interface.
[0173] The relevant hardware device structure and crop seed phenotype extraction method are also applicable to the collection of seed phenotypes of different common crops, such as rice, wheat, corn, etc.
[0174] Based on computer binocular vision, digital image processing, deep learning and other technologies, the present invention constructs a high-throughput crop grain phenotype extraction solution for artificial intelligence breeding routes, which can be divided into two parts: hardware devices and software systems. The hardware device includes binocular cameras, boxes, grain carrier plates, single-chip modified electronic scales, annular light strips and other components, which can realize the acquisition of high-throughput crop grain binocular images. The software system is developed based on PythonQt and OpenCV, and uses binocular stereo vision and deep learning technology to realize the extraction of various phenotypic parameters including the morphology, color, hilum, and texture of crop grains, and integrates the above algorithms into system functions, and finally can summarize and export phenotypic data in a tabular form. By using the present invention, breeding researchers can collect the phenotypic group data they need with high throughput, low cost, fast and accurate, helping breeding experts to be liberated from the heavy work of seed testing, thereby further guiding digital and intelligent agricultural production and promoting the healthy development of the entire breeding industry chain.
[0175] The present invention completes the supporting hardware device and software system for high-throughput crop grain phenotype collection for artificial intelligence breeding routes, and successfully constructs an integrated system with good encapsulation through the mutual coordination of software and hardware.
[0176] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A high-throughput crop seed phenotype collection method for artificial intelligence breeding. It is characterized in that The method comprises: Determine the hundred-grain weight of the crop grains on the grain-carrying plate, and simultaneously obtain binocular images of all the crop grains on the grain-carrying plate; Each crop seed in the binocular image is selected using the minimum bounding rectangle, and the two-dimensional morphological phenotype of each crop seed is extracted; Determine the 3D point cloud model of all crop seeds in the binocular image and extract the 3D morphological phenotype of each crop seed; Segmenting the binocular image into single-grain crop grains to obtain a plurality of single-grain crop grain images; Extracting the seed coat color phenotype of each crop kernel from each single crop kernel image; The hilum is segmented from each single crop kernel image, the hilum is selected using a minimum bounding rectangle, and the hilum phenotype of each crop kernel is determined.
2. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that The formula for determining the 100-grain weight is: Among them, M′ is the weight of 100 grains, M plate is the weight of the grain bearing plate, M max is the maximum weighing weight of the electronic scale, E is the electromotive force of the electronic scale power supply, R 0 is an irrelevant resistor in the circuit, I is the total circuit current during weighing, and R1 is the access resistor of the pressure sensor inside the electronic scale.
3. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that Obtain binocular images of all crop grains on the grain carrier plate, followed by: Cropping the binocular image to remove blank background and labels in the binocular image; Separate the RGB three channels of the cropped binocular image to obtain a grayscale image; Perform histogram equalization on the grayscale image and use the LoG operator to sharpen the image to obtain a contour-sharpened image; The binocular image and the contour-sharpened image are superimposed at a ratio of 9:1 to obtain an enhanced binocular image.
4. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that Use the minimum bounding rectangle to select each crop seed in the binocular image, and extract the two-dimensional morphological phenotype of each crop seed, including: Graying and binarizing the binocular image to separate the crop grains from the background, thereby obtaining a crop grain binary image; Continuously performing two corrosion operations and two dilation operations on the crop seed binary image to eliminate background noise and obtain a denoised crop seed binary image; The number of connected domains is calculated for the denoised binary image of crop grains to obtain the crop grain count; Each crop grain in the denoised crop grain binary image is selected using a minimum bounding rectangle, and each crop grain is numbered; The proportional coefficient between the image pixels and the actual size is obtained through the five-point calibration method; According to the minimum circumscribed rectangle of each crop seed, the proportional coefficient and the crop seed count, according to the formula Calculate the length and width of each crop seed, as well as the average length and average width of all crop seeds on the seed carrier; where L_soybean i and W_soybean i They represent the length and width of the crop grain numbered i, k is the proportional coefficient, L_pixel i and W_pixel i represents the number of pixels of the length and width of the minimum circumscribed rectangle of the crop grain numbered i, L and W represent the average length and average width of all crop grains on the grain-carrying plate, and N represents the crop grain count; Calculate the outline perimeter and area of each crop seed through OpenCV function; According to the outline perimeter and area of each crop seed, according to the formula Calculate the cross-sectional perimeter and cross-sectional area of each crop seed, as well as the average cross-sectional perimeter and average cross-sectional area of all crop seeds on the seed carrier; where C 2d _soybean i and S 2d _soybean i represent the cross-sectional perimeter and cross-sectional area of the crop seed numbered i, respectively. 2dd _pixel i and S 2d _pixel i Respectively represent the contour circumference and area of the crop grain numbered i, C 2d and S 2d are the average cross-sectional perimeter and average cross-sectional area of all crop grains on the grain-carrying plate, respectively; A plane rectangular coordinate system is established with the vertex of the upper left corner of the minimum circumscribed rectangle of each crop seed as the coordinate origin. Calculate the two-dimensional center coordinates of each crop seed; where (rec_x j ,rec_y j ) is the coordinate of the jth vertex in the minimum circumscribed rectangle of the crop grain, (X 2d ,Y 2d ) is the two-dimensional center coordinate of the crop grain; The length and width of each crop grain, the cross-sectional perimeter and cross-sectional area of each crop grain, and the two-dimensional center coordinates of each crop grain together constitute the two-dimensional morphological phenotype of each crop grain.
5. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that Determine the 3D point cloud model of all crop seeds in the binocular image and extract the 3D morphological phenotype of each crop seed, including: Performing voxelized grid downsampling on the point cloud in the three-dimensional point cloud model; The normal vector estimation calculation of the local surface fitting based on the KD tree is performed, and the downsampled point cloud is clustered using the density-based DBSCAN clustering, so that all the point clouds corresponding to each crop grain are clustered into one category; The cluster center of each class is used as the geometric center of the crop grain to obtain the three-dimensional center coordinates of each crop grain; According to the three-dimensional center coordinates of each crop seed, the thickness, average diameter, volume and surface area of each crop seed are calculated using the following formula: H_soybean i =2·(max(Cloud i (Z))-min(Cloud i (Z))) Among them, H_soybean i , R 3d _soybean i 、V 3d _soybean i and S 3d _soybean i They represent the thickness, average diameter, volume and surface area of the crop grain numbered i, respectively. Cloud is the set of all point clouds in the same cluster. N c is the number of points in the same cluster, Cloud r (x r ,y r ,z r ) is the coordinate of the rth point cloud in the same cluster, (X 3d ,Y 3d ,Z 3d ) is the three-dimensional center coordinate of the crop grain, Cloud i (Z) represents the point cloud data of the crop grain numbered i in the Z direction; According to the formula Determine the average seed thickness, average diameter, average volume and average surface area of all crop seeds on the seed carrier; where H, R, V and S 3d They are the average kernel thickness, average diameter, average volume and average surface area; The thickness, average diameter, volume and surface area of each crop seed together constitute the three-dimensional morphological phenotype of each crop seed.
6. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that The seed coat color phenotype of each crop grain is extracted from each single crop grain image, specifically including: The K-Means clustering algorithm is used to cluster the pixels in the single-grain crop grain image into multiple different color clusters; According to the multiple different color clusters obtained by clustering, a cluster histogram is drawn to obtain the color spectrum band of the single-grain crop grain image; Eliminate the background color from the color spectrum band, and use the color category with the largest number of pixels in the color spectrum band after the background color is eliminated as the RGB color of the seed coat of the crop grain; The mask concept is used to remove the black background with RGB values less than or equal to [50,50,50] in the single-grain crop grain image, and the RGB values of the remaining background pixels are set to 0, and the rest are grayed to obtain the single-grain crop grain grayscale image; According to the formula Calculate the mean of the local variance and the 95% quantile of the local variance of the single-grain crop grain grayscale image; where mean_var is the mean of the local variance, max_var is the 95% quantile of the local variance, var l is the variance of the lth non-zero value, is the mean of the variance of all non-zero values, n is the number of pixels with non-zero variance values, P 95 () is the 95% quantile; According to the mean of the local variance of the single-grain crop grain grayscale image and the 95% quantile of the local variance, the formula Calculate the glossiness of the seed coat of crop seeds; where glossiness is the glossiness; Determine the gray level co-occurrence matrix of a single crop grain image, and calculate contrast, homogeneity, correlation and energy based on the gray level co-occurrence matrix; Comprehensive contrast, homogeneity, correlation and energy, according to the formula Calculate the texture complexity of the seed coat of crop seeds; where C is texture complexity, contrast is contrast, homogeneity is homogeneity, correlation is correlation, and energy is energy; The RGB color of the seed coat of each crop seed, the glossiness of the seed coat of each crop seed and the texture complexity of the seed coat of each crop seed constitute the seed coat color phenotype of each crop seed.
7. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that The hilum is segmented from each single crop grain image, the hilum is selected using a minimum bounding rectangle, and the hilum phenotype of each crop grain is determined, including: The single-grain crop grain image is grayed and inverted into a binary image, and then dilated and eroded once respectively when the rectangular structure element size is 2×2 to obtain a single-grain crop grain binary image with noise eliminated. Use the minimum bounding rectangle to select the hilum in the binary image of single-grain crop seeds after noise elimination; Select the minimum circumscribed rectangle of the hilum according to the formula and Calculate the length, width, perimeter, area and center coordinates of the hilum in a single crop seed; among them, L_hilumfeature, W_hilumfeature, C_hilumfeature, S_hilumfeature and (Xh 2d , Yh 2d ) are the length, width, perimeter, area and center coordinates of the hilum in a single crop seed, Lh_pixel and Wh_pixel are the number of pixels of the length and width of the minimum circumscribed rectangle of the selected hilum, C_hilumfeature and S_hilumfeature are the perimeter and area of the hilum in a single crop seed, k is the proportional coefficient, (rh_x p ,rh_y p ) is the coordinate of the pth vertex in the minimum circumscribed rectangle of the selected hilum; The selection range is (Xh 2d -5,Yh 2d -5) to (Xh 2d +4,Yh 2d +4), using the formula Calculate the RGB value of the hilum color; where [h_red, h_green, h_blue] is the RGB value of the hilum color, [r q ,g q ,b q ] is the RGB value of the qth pixel in the selected range; According to the single-grain crop grain image, the texture complexity extraction algorithm based on 8-neighborhood LBP is adopted according to the formula Calculate the texture complexity of the hilum; where C h is the texture complexity of the hilum, M and N are the corresponding numbers of rows and columns after the LoG edge detection grayscale image is divided into blocks, Hist m,n (k′) is the k′th binary number in the histogram corresponding to the block in the mth row and nth column of the block image, and block_size is the block size; The length, width, perimeter, area, center coordinates of the hilum, RGB value of the hilum color and the texture complexity of the hilum constitute the hilum phenotype of crop seeds.
8. The high-throughput crop seed phenotype collection method for artificial intelligence breeding according to claim 1, It is characterized in that The method further comprises: YOLOv5-s is used to detect the hilum and seed coat cracks of each crop seed in the binocular image; The two-dimensional morphological phenotype, three-dimensional morphological phenotype, seed coat color phenotype and hilum phenotype of crop kernels are displayed and exported in table form.
9. A high-throughput crop seed phenotype collection device for artificial intelligence breeding. It is characterized in that include: Binocular camera, grain carrier plate, electronic scale, control board and computer; The electronic scale is arranged below the grain carrying plate, and the grain carrying plate is used to place a plurality of crop grains; The output end of the electronic scale is connected to the input end of the control board, and the output end of the control board is connected to the input end of the computer; the electronic scale is used to sense and generate a pressure signal under the pressure of the crop grains on the grain carrying plate, and determine the hundred-grain weight of the crop grains on the grain carrying plate according to the pressure signal, and upload the hundred-grain weight to the computer through the control board; The control end of the binocular camera is connected to the output end of the computer, and the output end of the binocular camera is connected to the first input end of the computer; the binocular camera is used to capture binocular images of all crop grains on the grain carrying plate under the control of the computer, and upload the binocular images to the computer; The computer is used to simultaneously extract the two-dimensional morphological phenotype, three-dimensional morphological phenotype, seed coat color phenotype and hilum phenotype of each crop grain on the grain supporting plate based on the binocular image and the high-throughput crop grain phenotype collection method for artificial intelligence breeding as described in any one of claims 1 to 8.
10. The high-throughput crop seed phenotype collection device for artificial intelligence breeding according to claim 9, It is characterized in that Also includes: Seed test box; The binocular camera, the seed carrier plate, the electronic scale and the control panel are all arranged inside the seed testing box; the binocular camera is fixed on the top of the seed testing box, the electronic scale and the seed carrier plate are both located at the bottom of the seed testing box, and the camera head of the binocular camera is aimed at the seed carrier plate.
Citation Information
Patent Citations
High-throughput detection method for grain form parameters
CN103808263A
Maize kernel phenotype measuring method and system
CN106971393A
Crop high-throughput phenotypic detection method and system
CN107358610A
High-flux plant phenotype measuring device and method
CN111179363A
Multi-lens type wheat grain phenotype detection device and method
CN117191788A
