Rice refined phenotypic characteristic analysis system and method based on deep learning

Abstract

The invention discloses a rice refined phenotypic characteristic analysis system and method based on deep learning, and the system comprises a data image collection module, a digital image processing module, a three-dimensional model reconstruction module, a point cloud data processing module, a point cloud segmentation and feature extraction module, and a phenotypic parameter obtaining module. The data image acquisition module, the digital image processing module, the three-dimensional model reconstruction module, the point cloud data processing module, the point cloud segmentation feature extraction module and the phenotype parameter acquisition module are in communication connection in sequence. According to the method, the rice variety characteristics can be rapidly and effectively evaluated accurately, and the rice phenotype parameters are measured in a dynamic and lossless manner by replacing manual measurement with a three-dimensional reconstruction technology of computer vision; the quantification of complex phenotypic information such as the volume of the minimum bounding box of a single plant is beneficial to accurately analyzing genetic variation and environment interaction in the rice growth process, the variety improvement efficiency is improved, and the breeding process is accelerated.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep image recognition, and specifically relates to a rice refined phenotypic feature analysis system and method based on deep learning. Background Art

[0002] In the field of agriculture, especially in rice research, phenotyping has become a key step in evaluating plant performance. This technology involves measuring the physical characteristics and behaviors of plants, such as plant height, number and size of leaves, root structure, and flowering time, all of which directly affect rice yield. With the advancement of science and technology, traditional manual measurement methods have been replaced by automated and even intelligent technologies. These new technologies can not only collect large amounts of data more quickly and accurately, but also provide more comprehensive information, which is extremely important for the study of rice growth patterns and the management of the growth process.

[0003] For example, in modern phenotyping, computer vision technology combined with image processing algorithms are used to identify and quantify the physical characteristics of plants. Through images captured by high-definition cameras mounted on drones or ground robots, researchers can estimate important indicators such as leaf area, color changes, and growth rates. In addition, various types of sensors are also widely used, such as spectrometers and thermal imaging cameras, which can collect data on the physiological state of plants and help understand the response of plants to environmental conditions.

[0004] Remote sensing technology also plays an important role in this area. Large-scale monitoring using satellites or drones can provide an overall view of farmland, help assess the growth of rice in different areas, and promptly identify potential problem areas. At the same time, with the application of big data analysis and machine learning models, people have begun to try to predict the performance of specific genotypes in specific environments and train models to identify specific phenotypic patterns to accelerate the selection process of new varieties.

[0005] The integration of genomics and phenomics has also brought new breakthroughs to rice research. With the help of gene editing technologies such as CRISPR-Cas9, scientists can modify plant genes more accurately and observe their specific effects on phenotypes. Combining genomic information with phenotypic data can provide a deeper understanding of how gene-environment interactions affect the growth and development of rice. In addition, the application of smart devices and Internet of Things technologies makes it possible to deploy sensor networks in the field. These sensors can monitor changes in environmental factors such as temperature, humidity, and light intensity in real time, and transmit information to a central database for subsequent analysis.

[0006] However, the above characterization methods cannot well capture the subtle characteristics of rice, resulting in the inability to accurately judge the growth status of rice based on refined phenotypic characteristics. Summary of the invention

[0007] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a rice refined phenotypic feature analysis system and method based on deep learning. The present invention uses deep learning technology to analyze subtle features such as rice plant height, number of leaves, leaf area, leaf projection area, etc., and monitor the growth status of rice in real time.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a rice refined phenotypic feature analysis system based on deep learning, comprising a data image acquisition module, a digital image processing module, a three-dimensional model reconstruction module, a point cloud data processing module, a point cloud segmentation and extraction feature module, and a phenotypic parameter acquisition module; the data image acquisition module, the digital image processing module, the three-dimensional model reconstruction module, the point cloud data processing module, the point cloud segmentation and extraction feature module, and the phenotypic parameter acquisition module are sequentially connected in communication;

[0010] The data image acquisition module is used to obtain high-definition images of rice plants;

[0011] The digital image processing module is used to perform image processing on the acquired rice plant image, and to extract and classify the appearance features of the rice plant image;

[0012] The three-dimensional model reconstruction module is used to capture the three-dimensional structural information of the rice plant and reconstruct the three-dimensional point cloud data using the preset three-dimensional model; the three-dimensional reconstruction specifically includes: extracting feature points using a scale-invariant feature conversion algorithm to obtain key point information, reconstructing a sparse point cloud using SFM for the key point information, and reconstructing a dense point cloud using MVS for the sparse point cloud;

[0013] The point cloud data processing module is used for purifying, calibrating, aligning, classifying, identifying, extracting features, reducing dimensions and enhancing data of rice plants, and extracting texture features of rice seeds;

[0014] The point cloud segmentation and feature extraction module is used to accurately identify and segment the three-dimensional point cloud data of the rice plant to extract a series of key phenotypic features;

[0015] The phenotypic parameter acquisition module is used to calculate the phenotypic trait parameters of the rice plant, including plant height, number of leaves, minimum bounding box volume, leaf area, and leaf projection area values.

[0016] As a preferred technical solution, in the digital image processing module, the image processing is specifically:

[0017] The image denoising method based on median filtering is used to perform image denoising;

[0018] The Exg super green method is used to grayscale the denoised image;

[0019] The histogram equalization method is used to count the grayscale values ​​of the image, specifically: calculate the probability density function of each pixel value of the original image, calculate the cumulative distribution function based on the probability density function, perform an equalization transformation on the cumulative distribution function, round the result of the equalization transformation to the nearest integer, and obtain the pixel value of the final equalized image.

[0020] As a preferred technical solution, the scale-invariant feature conversion algorithm is used to extract feature points to obtain key point information, specifically:

[0021] Constructing scale space: The SIFT algorithm uses the original image and the image obtained after different Gaussian convolution processing to perform a difference operation, that is, subtracting the two adjacent layers in each group of the Gaussian pyramid to obtain the Gaussian difference space;

[0022] Extreme point detection: The extreme points of the image are composed of local extreme points in the differential scale space. The DoG value of each pixel is compared with the DoG values ​​of other pixels in the neighborhood space, and the maximum or minimum points in the local space are defined as key points.

[0023] Extreme point direction generation: The direction of the feature point is calculated based on the gradient amplitude and gradient depression angle of all pixels in the neighborhood of the feature point;

[0024] Descriptor generation: After obtaining the key point information, create a descriptor.

[0025] As a preferred technical solution, the sparse point cloud is reconstructed by using SFM on the key point information, specifically:

[0026] For the matched key point pair (p1, p2), let its coordinates on image I1 and image I2 be (x1, y1) and (x2, y2) respectively, the camera intrinsic parameter matrix is ​​K, and the basic matrix is ​​F. Then the three-dimensional coordinates (X, Y, Z) of the point P in the triangulation are calculated by the following formula:

[0027] X=(C2.x-C1.x)*F11+(C2.y-C1.y)*F12+F13

[0028] Y=(C2.x-C1.x)*F21+(C2.y-C1.y)*F22+F23

[0029] Z=(C2.x-C1.x)*F31+(C2.y-C1.y)*F32+F33

[0030] Where C1 and C2 are the optical centers of the first camera and the second camera respectively, and F is the basic matrix, which is calculated by the camera intrinsic matrix K and the essential matrix E;

[0031] The three-dimensional coordinates (X, Y, Z) are then normalized to obtain the three-dimensional coordinates of point P.

[0032] As a preferred technical solution, the sparse point cloud is reconstructed into a dense point cloud using MVS, specifically:

[0033] Initialize feature matching, the calculation formula is:

[0034] Feature matching index = Σ(similarity of feature i * weight of feature i) / Σ(weight of feature i)*100%;

[0035] Mesh generation;

[0036] Patch filtering.

[0037] As a preferred technical solution, in the point cloud data processing module, PCL and CloudCompare software are used to perform registration, filtering, segmentation and correction processing on the point cloud data. The specific processing steps are as follows:

[0038] Point cloud stitching, the specific calculation formula is:

[0039] x_A_transformed=R*xB+t_x

[0040] y_A_transformed=R*yB+t_y

[0041] z_A_transformed=R*zB+t_z

[0042] Among them, (x_A_transformed, y_A_transformed, z_A_transformed) are the new coordinates of the midpoint of B relative to A after transformation; R is a rotation matrix that defines the rotation relationship from B to A; and t_x, t_y, t_z are the three components of the translation vector, representing the translation along the x, y, and z axes;

[0043] After deducting the target point cloud, the rice plant model was segmented from the original model by manual segmentation in the 3D point cloud processing software CloudCompare.

[0044] Point cloud downsampling uses voxel downsampling, and the specific calculation formula is:

[0045] Voxel_Center=(1 / N)*Σ(Pi)

[0046] Among them, Voxel_Center is the center point of the voxel, N is the number of points in the voxel, and Pi is the number of points in the voxel;

[0047] Remove ground point cloud: Use RANSAC-based plane fitting segmentation algorithm to implement ground point cloud segmentation in PCL;

[0048] Point cloud coordinate correction, the specific calculation formula is as follows:

[0049] Translation matrix T = [-t_x,-t_y,-t_z]

[0050] Rotation matrix R = [r11, r12, r13;

[0051] r21, r22, r23;

[0052] r31,r32,r33]

[0053] Corrected coordinate P'=R*P+T;

[0054] P is the original coordinate, P' is the corrected coordinate, t_x, t_y, t_z are the translations, r11,...,r33 are the rotation matrix elements;

[0055] As a preferred technical solution, the specific processing steps of the point cloud segmentation and feature extraction module are as follows:

[0056] The Euclidean clustering algorithm is used to complete the segmentation of individual rice plants. The specific calculation formula is as follows:

[0057] d(I,J)=√[Σ(Xi-Xj)^2+(Yi-Yj)^2]

[0058] Where I and J represent two plant individuals, Xi and Xj are the values ​​of plant individuals I and J on the i-th feature, Yi and Yj are the values ​​of plant individuals I and J on the j-th feature, and d(I,J) represents the Euclidean distance between plant individuals I and J;

[0059] The Euclidean clustering algorithm is used to segment the leaf point cloud for leaf segmentation, specifically: initialization, select one or more points as the center of the initial cluster; traverse each point in the point cloud and calculate the Euclidean distance from the point to all existing cluster centers; assign each point to the cluster center closest to it, so that it is classified into the corresponding cluster; update the center point of each cluster by calculating the average position of all points in the cluster; until the change of the cluster center is less than the preset threshold, or the preset number of iterations is reached.

[0060] As a preferred technical solution, the specific processing steps of the phenotypic parameter acquisition module are as follows:

[0061] The maximum and minimum value traversal algorithm is used to calculate the plant height of rice plants: by traversing all the point clouds of the plants, the maximum value of the z coordinate is obtained, and thus the plant height is calculated;

[0062] The greedy projection triangulation algorithm is used for triangular meshing. The meshed leaf model consists of several spatial triangular meshes. After meshing, the area of a single leaf is calculated by Heron's formula;

[0063] The calculation principle of the minimum bounding box volume of a single plant is adopted. After correction, the maximum x coordinate, y coordinate, and z coordinate, and the minimum x coordinate, y coordinate, and z coordinate of the point cloud of a single plant are obtained, and multiple vertices are obtained. The volume of the cuboid formed by connecting these multiple vertices is the minimum bounding box volume of a single plant;

[0064] The leaf projection area of the rice plant is calculated by summing the areas of individual planar triangular meshes.

[0065] As a preferred technical solution, the maximum and minimum value traversal algorithm is used to calculate the plant height of rice plants: by traversing all the point clouds of the plants, the maximum value of the z coordinate is obtained, and thus the plant height is calculated. Specifically:

[0066] Let the plant height array be H[n], where n is the number of plants, and H[i] represents the height of the i-th plant;

[0067] Initialize the maximum value maxHeight to H[0], and the minimum value minHeight to H[0];

[0068] Traverse the array H. For each H[i] (i from 1 to n - 1), perform the following operations:

[0069] a. If H[i] > maxHeight, then maxHeight = H[i].

[0070] b. If H[i] < minHeight, then minHeight = H[i].

[0071] After the traversal, maxHeight and minHeight represent the maximum plant height and the minimum plant height of the plants respectively;

[0072] To calculate the average plant height, the following formula can be used:

[0073] averageHeight = (sum(H[i]) for i from 0 to n - 1) / n

[0074] where sum(H[i]) represents the sum of the plant heights.

[0075] In a second aspect, the present invention provides a method for analyzing rice phenotypic characteristics based on deep learning, which is applied to the rice phenotypic characteristics analysis system based on deep learning, and is characterized in that it includes the following steps:

[0076] The data image acquisition module collects rice plant image data through drone aerial photography and inputs the data into the spectral data processing module and the spectral image processing module;

[0077] The digital image processing module uses advanced image processing technology to accurately extract and classify the appearance features of rice, including image denoising, contrast enhancement and brightness adjustment to reduce the existing noise, and inputs the data into the 3D model reconstruction module;

[0078] First, create a three-dimensional morphology of the rice plant, then clean, reduce noise and filter the collected point cloud data to improve data quality, and finally segment the point cloud data into different plant parts and transmit the data information to the phenotypic parameter acquisition module;

[0079] The phenotypic parameter acquisition module uses deep learning technology to accurately monitor and evaluate the growth status of rice, extract key phenotypic parameters of rice leaves, plant height, and tiller number, and realize dynamic monitoring of the entire rice growth cycle.

[0080] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0081] The present invention can rapidly and effectively evaluate the characteristics of rice varieties, and dynamically and non-destructively measure rice phenotypic parameters through three-dimensional reconstruction technology of computer vision instead of manual measurement. Quantifying complex phenotypic information such as the minimum bounding box volume of a single plant helps to accurately analyze the genetic variation and environmental interaction during rice growth, improve the efficiency of variety improvement, and accelerate the breeding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0083] Figure 1 A block diagram of a rice phenotypic feature analysis system based on deep learning according to an embodiment of the present invention;

[0084] Figure 2 This is a diagram showing the effect of image preprocessing according to an embodiment of the present invention;

[0085] Figure 3This is a schematic diagram of histogram correction comparison in an embodiment of the present invention;

[0086] Figure 4 The sparse point cloud effect diagram is reconstructed by SFM in the embodiment of the present invention;

[0087] Figure 5 The dense point cloud rendering is reconstructed by MVS in the embodiment of the present invention;

[0088] Figure 6 This is a point cloud segmentation result diagram of a single plant in an embodiment of the present invention;

[0089] Figure 7 A schematic diagram of the process and results of leaf segmentation according to an embodiment of the present invention;

[0090] Figure 8 The present invention is a flowchart of a method for analyzing the refined phenotypic characteristics of rice based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0092] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0093] like Figure 1 As shown, an embodiment of the present invention provides a rice refined phenotypic feature analysis system based on deep learning, including a data image acquisition module, a digital image processing module, a three-dimensional model reconstruction module, a point cloud data processing module, a point cloud segmentation and extraction feature module and a phenotypic parameter acquisition module; the data image acquisition module, the digital image processing module, the three-dimensional model reconstruction module, the point cloud data processing module, the point cloud segmentation and extraction feature module and the phenotypic parameter acquisition module are communicatively connected in sequence.

[0094] The digital image processing module is used to accurately extract and classify the appearance features of rice through advanced image processing technology, including image denoising, contrast enhancement and brightness adjustment, so as to reduce the existing noise and ensure the accuracy of subsequent analysis.

[0095] The three-dimensional model reconstruction module is used to capture the three-dimensional structural information of the rice plant, construct a three-dimensional model, and extract structural features therefrom to provide data for phenotypic analysis.

[0096] The point cloud data processing module is used for data purification, calibration and alignment, classification and recognition, feature extraction, dimension reduction and data enhancement of rice plants, and extraction of rice seed texture features;

[0097] The point cloud segmentation and feature extraction module is used to accurately identify and segment the three-dimensional point cloud data of the rice plant, thereby effectively extracting a series of key phenotypic features.

[0098] The phenotypic parameter acquisition module is used to calculate the phenotypic trait parameters of the rice plant, including plant height, number of leaves, minimum bounding box volume, leaf area, leaf projection area and other values.

[0099] In a specific example, the specific processing steps of the digital image processing module are as follows:

[0100] S1.1. Image denoising is performed using an image denoising method based on median filtering. The calculation formula is:

[0101] f(x,y)=w_(n+1) / 2;

[0102] In the formula, there is a window W containing n pixel values. These values ​​are sorted from small to large and recorded as w_1, w_2, ..., w_n. Then the new value f(x, y) of the center pixel of the window is: when n is an odd number, the median is the (n+1) / 2th value; when n is an even number, the average of the two middle values ​​is usually taken as the median.

[0103] S1.2, use Exg super green method to grayscale the image, and the calculation formula is:

[0104] The calculation formula of the Exg super green method is: Exg=(RG)-0.5(RB), where R, G, and B represent the pixel values ​​of the red, green, and blue channels respectively.

[0105] S1.3, use the histogram equalization method to count the grayscale values ​​of the image, such as Figure 2 As shown, the specific implementation steps are:

[0106] S1.3.1: Calculate the probability density function (PDF) of each pixel value of the original image:

[0107] P(i)=(number of pixels with intensity i) / (total number of pixels)

[0108] S1.3.2: Calculate the cumulative distribution function (CDF):

[0109] CDF(i)=Σ[P(j)](for j from 0to i)

[0110] S1.3.3: Apply equalization transformation:

[0111] S=CDF(i)*(L-1)

[0112] Where S is the pixel value of the output image and L is the total number of pixel values ​​(usually 256 for 8-bit images).

[0113] S1.3.4: Round the result of CDF(i)*(L-1) to the nearest integer to obtain the pixel value of the final equalized image, such as Figure 3 As shown in parts (a) to (d) of FIG.

[0114] In a specific example, the 3D model reconstruction module uses VisualSfM 3D reconstruction, and the specific processing steps are as follows:

[0115] S2.1. Use scale-invariant feature conversion algorithm to extract feature points. The specific implementation steps are as follows:

[0116] S2.1.1: Constructing scale space: The SIFT algorithm uses the original image and the image obtained after different Gaussian convolution processing to perform a difference operation, that is, subtracting the two adjacent layers in each group of the Gaussian pyramid to obtain the Gaussian difference space;

[0117] S2.1.2: Extreme point detection: The extreme points of the image are composed of local extreme points in the differential scale space. The DoG value of each pixel is compared with the DoG values ​​of other pixels in the neighborhood space. The maximum or minimum points in the local space are defined as key points.

[0118] S2.1.3: Extreme point direction generation: The direction of the feature point is calculated based on the gradient amplitude and gradient depression angle of all pixels in the neighborhood of the feature point. The calculation formula for the gradient modulus m(x,y) and gradient amplitude angle (x,y) of the Gaussian image L at point (x,y) is as follows:

[0119]

[0120] Where L is the scale space value of the feature point;

[0121] S2.1.4: Descriptor generation: After obtaining the key point information, create a descriptor. Based on the key point, take a 16×16 rectangular area, divide it into 16 4×4 sub-areas, and calculate the 8-directional gradient histogram of each sub-area to form a seed point. Each feature point is composed of 16 seed points, each seed point has 8 directions, forming a 128-dimensional SIFT feature vector;

[0122] S2.2, use SFM to reconstruct the sparse point cloud, and the calculation formula is as follows:

[0123] For the matched key point pair (p1, p2), let its coordinates on images I1 and I2 be (x1, y1) and (x2, y2) respectively, the camera intrinsic parameter matrix is ​​K, and the basic matrix is ​​F. Then the three-dimensional coordinates (X, Y, Z) of the point P in the triangulation can be calculated by the following formula:

[0124] X=(C2.x-C1.x)*F11+(C2.y-C1.y)*F12+F13

[0125] Y=(C2.x-C1.x)*F21+(C2.y-C1.y)*F22+F23

[0126] Z=(C2.x-C1.x)*F31+(C2.y-C1.y)*F32+F33

[0127] Among them, C1 and C2 are the optical centers of camera 1 and camera 2 respectively, and F is the basic matrix, which can be calculated by the camera intrinsic parameter matrix K and the essential matrix E.

[0128] S2.3, the three-dimensional coordinates (X, Y, Z) also need to be normalized to finally obtain the three-dimensional coordinates of point P, such as Figure 4 shown.

[0129] In a specific implementation, MVS is used to reconstruct a dense point cloud, as shown in the following figure. Figure 5 As shown, the specific implementation steps are as follows:

[0130] S3.1: Initialize feature matching, the calculation formula is:

[0131] Feature matching index = Σ(similarity of feature i * weight of feature i) / Σ(weight of feature i)*100%;

[0132] S3.2: Surface generation;

[0133] S3.3: Patch filtering.

[0134] In a specific implementation, the point cloud data processing module uses PCL and CloudCompare software to perform pre-processing such as registration, filtering, segmentation and correction on the point cloud data. The specific processing steps are as follows:

[0135] S4.1: Point cloud stitching, the specific calculation formula is:

[0136] x_A_transformed=R*xB+t_x

[0137] y_A_transformed=R*yB+t_y

[0138] z_A_transformed=R*zB+t_z

[0139] Among them, (x_A_transformed, y_A_transformed, z_A_transformed) are the new coordinates of the midpoint of B relative to A after transformation. R is a 3x3 rotation matrix that defines the rotation relationship from B to A; and t_x, t_y, t_z are the three components of the translation vector, representing the translation along the x, y, and z axes;

[0140] S4.2: Subtract the target point cloud and use the manual segmentation method to perform a subtraction preprocessing in the 3D point cloud processing software CloudCompare to segment the rice plant model from the original model;

[0141] S4.3: Point cloud downsampling, using voxel downsampling, the specific calculation formula is:

[0142] Voxel_Center=(1 / N)*Σ(Pi)

[0143] Among them, Voxel_Center is the center point of the voxel, N is the number of points in the voxel, and Pi is the number of points in the voxel;

[0144] S4.4: Remove ground point cloud: Use RANSAC-based plane fitting segmentation algorithm to implement ground point cloud segmentation in PCL;

[0145] S4.5: Point cloud coordinate correction, the specific calculation formula is as follows:

[0146] Translation matrix T = [-t_x,-t_y,-t_z]

[0147] Rotation matrix R = [r11, r12, r13;

[0148] r21, r22, r23;

[0149] r31,r32,r33]

[0150] Corrected coordinates P' = R*P + T

[0151] P is the original coordinate, P' is the corrected coordinate, t_x, t_y, t_z are the translations, and r11,...,r33 are the rotation matrix elements.

[0152] In a specific implementation, the point cloud segmentation and feature extraction module has the following specific processing steps:

[0153] S5.1. Use the Euclidean clustering algorithm to complete the segmentation of individual rice plants. The specific calculation formula is as follows:

[0154] d(I,J)=√[Σ(Xi-Xj)^2+(Yi-Yj)^2];

[0155] Among them, I and J represent two plant individuals, Xi and Xj are the values ​​of plant individuals I and J on the i-th feature, Yi and Yj are the values ​​of plant individuals I and J on the j-th feature, d(I,J) represents the Euclidean distance between plant individuals I and J, and the point cloud segmentation result of a single plant is as follows: Figure 6 shown.

[0156] S5.2. Use the Euclidean clustering algorithm to segment the leaf point cloud for leaf segmentation. The process and results of leaf segmentation are shown in Figure 7 As shown, the specific implementation steps are as follows:

[0157] S5.2.1: Initialization: Select one or more points as the centers of the initial clusters.

[0158] S5.2.2: Traverse each point in the point cloud and calculate the Euclidean distance from the point to the center of all existing clusters.

[0159] S5.2.3: Assign each point to the cluster center that is closest to it, so that it belongs to the corresponding cluster.

[0160] S5.2.4: Update the center point of each cluster, usually by calculating the average position of all points in the cluster.

[0161] S5.2.5: Repeat steps S5.2.2 to S5.2.4 until the change in cluster center is less than a preset threshold or a preset number of iterations is reached.

[0162] In a specific implementation, the specific processing steps of the phenotypic parameter acquisition module are as follows:

[0163] S6.1. Calculate the height of rice plants using the maximum value traversal algorithm: traverse all point clouds of the plants to obtain the maximum value of the z coordinate, thereby calculating the height of the plants. The specific implementation steps are as follows:

[0164] S6.1.1: Let the plant height array be H[n], where n is the number of plants, and H[i] represents the height of the i-th plant.

[0165] S6.1.2: Initialize the maximum value maxHeight as H[0], and the minimum value minHeight as H[0].

[0166] S6.1.3: Traverse the array H. For each H[i] (i from 1 to n - 1), perform the following operations:

[0167] a. If H[i] > maxHeight, then maxHeight = H[i].

[0168] b. If H[i] < minHeight, then minHeight = H[i].

[0169] S6.1.4: After the traversal, maxHeight and minHeight represent the maximum plant height and the minimum plant height respectively.

[0170] S6.1.5: Calculate the average plant height using the following formula:

[0171] averageHeight = (sum(H[i]) for i from 0 to n - 1) / n

[0172] where sum(H[i]) represents the sum of the plant heights;

[0173] S6.2: Use the greedy projection triangulation algorithm for triangulation. The triangulated leaf model consists of several spatial triangular patches. After triangulation, calculate the area of a single leaf using Heron's formula. The specific calculation formula is as follows:

[0174] S = √[p(p - a)(p - b)(p - c)]

[0175] The three side lengths of the leaf are a, b, and c, the semi-perimeter p = (a + b + c) / 2, and the leaf area is S;

[0176] S6.3: Adopt the calculation principle of the minimum bounding box volume of a single plant. After correction, obtain the maximum x-coordinate, y-coordinate, and z-coordinate, and the minimum x-coordinate, y-coordinate, and z-coordinate of the single-plant point cloud, getting 8 vertices. The volume of the cuboid formed by connecting these 8 vertices is the minimum bounding box volume of a single plant. The specific calculation formula is as follows:

[0177] V = l × w × h, where l represents the length of the bounding box, w represents the width of the bounding box, and h represents the height of the bounding box;

[0178] S6.4. Calculate the projected area of ​​rice plant leaves by summing the areas of individual plane triangle patches.

[0179] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0180] like Figure 8 As shown, in another embodiment of the present invention, a method for analyzing rice phenotypic characteristics in a refined manner based on deep learning is provided, comprising the following steps:

[0181] The data image acquisition module collects rice plant image data through drone aerial photography and inputs the data into the spectral data processing module and the spectral image processing module;

[0182] The digital image processing module uses advanced image processing technology to accurately extract and classify the appearance features of rice, including image denoising, contrast enhancement and brightness adjustment to reduce the existing noise, and inputs the data into the 3D model reconstruction module;

[0183] First, create a three-dimensional morphology of the rice plant, then clean, reduce noise and filter the collected point cloud data to improve data quality, and finally segment the point cloud data into different plant parts and transmit the data information to the phenotypic parameter acquisition module;

[0184] The phenotypic parameter acquisition module uses deep learning technology to accurately monitor and evaluate the growth status of rice, extract key phenotypic parameters of rice leaves, plant height, and tiller number, and realize dynamic monitoring of the entire rice growth cycle.

[0185] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0186] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0187] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A rice phenotypic feature analysis system based on deep learning, characterized in that: It includes a data image acquisition module, a digital image processing module, a three-dimensional model reconstruction module, a point cloud data processing module, a point cloud segmentation and extraction feature module and a phenotype parameter acquisition module; the data image acquisition module, the digital image processing module, the three-dimensional model reconstruction module, the point cloud data processing module, the point cloud segmentation and extraction feature module and the phenotype parameter acquisition module are sequentially connected in communication; The data image acquisition module is used to obtain high-definition images of rice plants; The digital image processing module is used to perform image processing on the acquired rice plant image, and to extract and classify the appearance features of the rice plant image; The three-dimensional model reconstruction module is used to capture the three-dimensional structural information of the rice plant and reconstruct the three-dimensional point cloud data using the preset three-dimensional model; the three-dimensional reconstruction specifically includes: extracting feature points using a scale-invariant feature conversion algorithm to obtain key point information, reconstructing a sparse point cloud using SFM for the key point information, and reconstructing a dense point cloud using MVS for the sparse point cloud; The point cloud data processing module is used for purifying, calibrating, aligning, classifying, identifying, extracting features, reducing dimensions and enhancing data of rice plants, and extracting texture features of rice seeds; The point cloud segmentation and feature extraction module is used to accurately identify and segment the three-dimensional point cloud data of the rice plant to extract a series of key phenotypic features; The phenotypic parameter acquisition module is used to calculate the phenotypic trait parameters of the rice plant, including plant height, number of leaves, minimum bounding box volume, leaf area, and leaf projection area values.

2. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: In the digital image processing module, the image processing is specifically as follows: The image denoising method based on median filtering is used to perform image denoising; The Exg super green method is used to grayscale the denoised image; The histogram equalization method is used to count the grayscale values ​​of the image, specifically: calculate the probability density function of each pixel value of the original image, calculate the cumulative distribution function based on the probability density function, perform an equalization transformation on the cumulative distribution function, round the result of the equalization transformation to the nearest integer, and obtain the pixel value of the final equalized image.

3. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: The scale-invariant feature conversion algorithm is used to extract feature points to obtain key point information, specifically: Constructing scale space: The SIFT algorithm uses the original image and the image obtained after different Gaussian convolution processing to perform a difference operation, that is, subtracting the two adjacent layers in each group of the Gaussian pyramid to obtain the Gaussian difference space; Extreme point detection: The extreme points of the image are composed of local extreme points in the differential scale space. The DoG value of each pixel is compared with the DoG values ​​of other pixels in the neighborhood space, and the maximum or minimum points in the local space are defined as key points. Extreme point direction generation: The direction of the feature point is calculated based on the gradient amplitude and gradient depression angle of all pixels in the neighborhood of the feature point; Descriptor generation: After obtaining the key point information, create a descriptor.

4. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: The sparse point cloud is reconstructed by using SFM on the key point information, specifically: For the matched key point pair (p1, p2), let its coordinates on image I1 and image I2 be (x1, y1) and (x2, y2) respectively, the camera intrinsic parameter matrix is ​​K, and the basic matrix is ​​F. Then the three-dimensional coordinates (X, Y, Z) of the point P in the triangulation are calculated by the following formula: X=(C2.x-C1.x)*F11+(C2.y-C1.y)*F12+F13 Y=(C2.x-C1.x)*F21+(C2.y-C1.y)*F22+F23 Z=(C2.x-C1.x)*F31+(C2.y-C1.y)*F32+F33 Among them, C1 and C2 are the optical centers of the first camera and the second camera respectively, and F is the basic matrix, which is calculated by the camera intrinsic parameter matrix K and the essential matrix E; The three-dimensional coordinates (X, Y, Z) are then normalized to obtain the three-dimensional coordinates of point P.

5. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: The method of reconstructing a dense point cloud using MVS on a sparse point cloud is as follows: Initialize feature matching, the calculation formula is: Feature matching index = Σ(similarity of feature i * weight of feature i) / Σ(weight of feature i)*100%; Mesh generation; Patch filtering.

6. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: In the point cloud data processing module, PCL and CloudCompare software are used to perform registration, filtering, segmentation and correction processing on the point cloud data. The specific processing steps are as follows: Point cloud stitching, the specific calculation formula is: x_A_transformed=R*xB+t_x y_A_transformed=R*yB+t_y z_A_transformed=R*zB+t_z Among them, (x_A_transformed, y_A_transformed, z_A_transformed) are the new coordinates of the midpoint of B relative to A after transformation; R is a rotation matrix that defines the rotation relationship from B to A; and t_x, t_y, t_z are the three components of the translation vector, representing the translation along the x, y, and z axes; After deducting the target point cloud, the rice plant model was segmented from the original model using a manual segmentation method and a pre-processing step in the 3D point cloud processing software CloudCompare. Point cloud downsampling uses voxel downsampling, and the specific calculation formula is: Voxel_Center=(1 / N)*Σ(Pi) Among them, Voxel_Center is the center point of the voxel, N is the number of points in the voxel, and Pi is the number of points in the voxel; Remove ground point cloud: Use RANSAC-based plane fitting segmentation algorithm to implement ground point cloud segmentation in PCL; Point cloud coordinate correction, the specific calculation formula is as follows: Translation matrix T = [-t_x,-t_y,-t_z] Rotation matrix R = [r11, r12, r13; r21,r22,r23； r31,r32,r33] Corrected coordinate P'=R*P+T; P is the original coordinate, P' is the corrected coordinate, t_x, t_y, t_z are the translations, and r11,...,r33 are the rotation matrix elements.

7. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: The specific processing steps of the point cloud segmentation and feature extraction module are as follows: The Euclidean clustering algorithm is used to complete the segmentation between individual rice plants. The specific calculation formula is as follows: d(I,J)=√[Σ(Xi-Xj)^2+(Yi-Yj)^2] Where I and J represent two plant individuals, Xi and Xj are the values of plant individuals I and J on the i-th feature, Yi and Yj are the values of plant individuals I and J on the j-th feature, and d(I,J) represents the Euclidean distance between plant individuals I and J; The Euclidean clustering algorithm is used to segment the leaf point cloud for leaf segmentation. Specifically: Initialize by selecting one or more points as the centers of the initial clusters; traverse each point in the point cloud and calculate the Euclidean distance from this point to all existing cluster centers; assign each point to the cluster center closest to it, thus belonging to the corresponding cluster; update the center point of each cluster by calculating the average position of all points within the cluster; until the change in the cluster center is less than a preset threshold or the preset number of iterations is reached.

8. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 1, characterized in that: The specific processing steps of the phenotype parameter acquisition module are as follows: The maximum value traversal algorithm is used to calculate the plant height of rice plants: the maximum value of the z coordinate is obtained by traversing all the point clouds of the plant, thereby calculating the plant height; The greedy projection triangulation algorithm is used for triangulation. The triangulated leaf model consists of several spatial triangular patches. After triangulation, the area of a single leaf is calculated using Heron's formula; Using the calculation principle of the minimum bounding box volume of a single plant, after correction, the maximum x coordinate, y coordinate, and z coordinate, and the minimum x coordinate, y coordinate, and z coordinate of the single-plant point cloud are obtained, resulting in multiple vertices. The volume of the cuboid formed by connecting these multiple vertices is the minimum bounding box volume of a single plant; The leaf projection area of rice plants is calculated by summing the areas of individual planar triangular patches.

9. The method for analyzing rice phenotypic characteristics based on deep learning according to claim 8, characterized in that: The maximum value traversal algorithm is used to calculate the plant height of rice plants: the maximum value of the z coordinate is obtained by traversing all the point clouds of the plant, thereby calculating the plant height. Specifically: Let the plant height array be H[n], where n is the number of plants, and H[i] represents the height of the i-th plant; Initialize the maximum value maxHeight as H[0] and the minimum value minHeight as H[0]; Traverse the array H. For each H[i] (i from 1 to n-1), perform the following operations: a. If H[i]>maxHeight, then maxHeight = H[i]. b. If H[i]<minHeight, then minHeight = H[i]. After the traversal, maxHeight and minHeight represent the maximum plant height and minimum plant height of the plants respectively; The average plant height can be calculated using the following formula: averageHeight=(sum(H[i])for i from 0to n-1) / n Where sum(H[i]) represents the sum of the plant heights.

10. A rice refined phenotypic characteristic analysis method based on deep learning, applied to a rice refined phenotypic characteristic analysis system based on deep learning according to any one of claims 1 to 9, characterized in that: Including the following steps: The data image acquisition module inputs the rice plant image data collected by drone aerial photography into the spectral data processing module and the spectral image processing module; The digital image processing module uses advanced image processing technology to accurately extract and classify the appearance features of rice, including image denoising, contrast enhancement and brightness adjustment to reduce the existing noise, and inputs the data into the 3D model reconstruction module; First, create a three-dimensional morphology of the rice plant, then clean, reduce noise and filter the collected point cloud data to improve data quality, and finally segment the point cloud data into different plant parts and transmit the data information to the phenotypic parameter acquisition module; The phenotypic parameter acquisition module uses deep learning technology to accurately monitor and evaluate the growth status of rice, extract key phenotypic parameters of rice leaves, plant height, and tiller number, and realize dynamic monitoring of the entire rice growth cycle.

Images