Corn plant leaf dip angle field in-situ extraction method based on depth camera

Abstract

The invention discloses a corn plant leaf dip angle field in-situ extraction method based on a depth camera. The method belongs to the technical field of intelligent phenotype monitoring of farmland crops in intelligent agriculture. The defect of low measurement precision caused by a single corn leaf inclination angle measurement mode in the prior art is overcome. According to the method, the calculation amount is simplified, high detection precision can still be guaranteed, an existing U-Net deep learning model is improved so that the method can be more accurately suitable for extraction of a corn plant skeleton, a technical means of fitting a main stem is added, the real form of a corn plant is fully considered, and the method is suitable for large-scale popularization and application. The calculation accuracy of the blade inclination angle is improved; in the method for performing 3D point cloud processing on the 3D point cloud data of the depth image to obtain the leaf inclination angle of each leaf of the corn plant, the calculation of the leaf inclination angle is more accurate by performing PCA analysis on the point cloud of each leaf, redefining the flatness condition of the leaf and performing processing according to the conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent phenotypic monitoring of farmland crops in smart agriculture, and particularly relates to a method for in-situ extraction of maize plant leaf inclination angles in the field using a depth camera. Background Art

[0002] In recent years, smart agriculture has become an important approach to sustainable agricultural development, and its core lies in achieving intelligent monitoring and management of farmland crops through advanced technical means. Intelligent phenotypic monitoring of farmland crops is one of the important research fields of precision agriculture, and efficient crop phenotypic monitoring technology is a key foundation for research in breeding, genomics, phenomics, and intelligent farmland management. Phenotypic analysis plays a crucial role in breeding and agricultural management. It can not only help understand the relationship between gene function and environmental effects, but also guide germplasm screening and variety evaluation in the breeding process, thereby accelerating the breeding process. Phenotypic monitoring provides important data support for resource regulation and management strategy formulation in precision agriculture, and is therefore regarded as one of the core driving forces for the development of agricultural science and technology. However, most traditional methods for measuring phenotypic parameters rely on manual operations, and have problems such as low measurement accuracy, high cost, and long time consumption, making it difficult to meet the requirements of modern agriculture for efficient and precise monitoring. With the rapid development of sensor technology and artificial intelligence, intelligent phenotypic monitoring technology based on multi-source data fusion has gradually become a research hotspot, providing new ideas and technical paths for solving the limitations of traditional methods.

[0003] Maize, recognized as the world's golden food, is one of the important food crops in China, especially in major producing areas such as Jilin Province, with significant economic value. Maize is not only an important raw material for food, medicine, and industry, but also has broad development and application prospects. By monitoring the growth of organs such as the stem, leaves, and plants of maize, key indicators such as the growth trend, pest and disease resistance, lodging resistance, and yield of maize can be indirectly reflected. Therefore, accurate measurement of maize phenotypic parameters has important scientific significance and application value. However, most phenotypic parameters during the maize growth period (such as stem diameter, leaf area, leaf inclination angle, etc.) are small-size morphological parameters, and their measurement is difficult, and traditional methods are difficult to meet the accuracy and efficiency requirements. For example, the leaf inclination angle is an important indicator reflecting the photosynthetic efficiency and lodging resistance of maize, but its measurement usually relies on manual operation, with problems such as strong subjectivity and low efficiency. To address these problems, developing an efficient and accurate method for measuring maize phenotypic parameters has become an important topic in precision agriculture and phenomics research.

[0004] In the field of phenotypic monitoring, a variety of sensors and technologies have been applied to the measurement of crop parameters. Taking the measurement of maize leaf inclination angle as an example, the methods include manual measurement, binocular stereo vision system, radar / laser sensor, and depth camera, etc. Although the manual measurement method is simple and intuitive, it has low efficiency, strong subjectivity, and is difficult to meet the needs of large-scale monitoring. The binocular stereo vision system realizes parameter extraction through image matching and three-dimensional reconstruction, but it is sensitive to natural lighting conditions and has high computational complexity, which limits its application in outdoor environments. Radar / laser sensors can provide high-precision depth information, but they are costly and perform poorly in complex scenarios (such as leaf closure or shadows). In recent years, the technology based on depth cameras has gradually become a research hotspot, which can synchronously acquire color images and depth information, providing a new solution for phenotypic parameter measurement. However, the existing depth camera technology still has the defect of low measurement accuracy caused by a single measurement method in the measurement of the phenotypic parameter leaf inclination angle during the growth period of maize. Summary of the Invention

[0005] In order to solve the defect of low measurement accuracy caused by a single measurement method in the measurement of the phenotypic parameter leaf inclination angle during the growth period of maize by the existing depth camera technology, the present invention provides a method for in-situ extraction of maize plant leaf inclination angle based on a depth camera. The method includes the following steps: S1. Use a depth camera to collect depth images and 3D point cloud data of the depth images during the mid-growth period of maize; S2. Perform multi-dimensional processing on the depth images to obtain the leaf inclination angles of each leaf of the maize plant; S3. Perform 3D point cloud processing on the 3D point cloud data of the depth images to obtain the leaf inclination angles of each leaf of the maize plant; S4. Calculate the determination coefficients of the regression curves of the leaf inclination angles of each leaf of the maize plant obtained after processing the depth images and the 3D point cloud data of the depth images of the same plant in steps S2 and S3 respectively and ; If and are both greater than or equal to 0.88, then go to step S5; if and one of the values is greater than or equal to 0.88, then go to step S6; if and are both less than 0.88, then go to step S7; S5. Perform weighted fusion on the processing results of steps S2 and S3 to obtain the leaf inclination angles of each leaf of the maize plant; S6. Use the result obtained by the method corresponding to the determination coefficient of the regression curve being greater than or equal to 0.88 as the leaf inclination angle of each leaf of the maize plant; S7. Determine the occlusion and overlap of the leaves, the fracture of the skeleton, and the missing or error of the point cloud, and transmit the data back to guide the experimenters to optimize steps S2 and S3. After optimization, return to step S1 to continue.

[0006] Furthermore, the middle growth stage of maize includes the large trumpet mouth stage and the tasseling stage.

[0007] Furthermore, the specific steps of step S3 are as follows: S31. Image segmentation: Perform color-based threshold segmentation on the maize plants in the depth image, and screen the pixels of the image so that the image can automatically distinguish the maize plants and the background area; S32. Image binarization: After completing the image segmentation, perform binarization on the image so that the pixel points in the image only present two colors, black and white; S33. Morphological processing: First perform morphological opening operation on the image and then perform morphological closing operation; S34. Skeletonization processing: Use the U-Net-ACS model to extract the skeletons of maize plants in the image; S35. Corner detection: Detect the corners in the maize plant skeleton through the Harris corner detection algorithm, and judge the stem-leaf connection points in the maize plant skeleton. Divide the main stem corners of the maize plant and the main stem corners of the maize leaves with the stem-leaf connection points as the boundary; S36. Angle calculation: Fit the main stem line of the maize plant through the main stem corners of the maize plant, judge the end points of the maize leaves through the main stem corners of the maize leaves, and calculate the leaf inclination angle of the maize plant leaves through the main stem line of the maize plant, the stem-leaf connection points and the end points of the maize leaves.

[0008] Furthermore, the U-Net-ACS model is obtained by improving on the basis of the U-Net model. Specifically: Introduce the CBAM attention mechanism after the convolution operation in the encoding process of the U-Net model, introduce the ASPP module after the convolution operation in the decoding process of the U-Net model, and adopt the deep supervision mechanism in the training process of the U-Net-ACS model.

[0009] Furthermore, the specific method for judging the stem-leaf connection points in the maize plant skeleton is as follows: For each corner point, use a neighborhood window to traverse the corner points in its neighborhood. If the number of corner points in its neighborhood is greater than or equal to three, then it is considered as a stem-leaf connection point. Furthermore, fitting the main stem line of the maize plant through the main stem corners of the maize plant is obtained by: obtained, where represents the calculation of weighted least squares, is the weight value, , and are the linear coefficients for fitting, corresponding to the intercept and slope of the main stem respectively, represent the coordinates of the corner points, represents the number of corner points, represents the mean coordinate of the corner points.

[0010] Furthermore, the leaf inclination angle of the maize plant is calculated through the main stem line of the maize plant, the stem-leaf connection point, and the end point of the maize leaf by: obtained, where, represents the leaf inclination angle, , represents the slope of the main direction of the leaf, represents the coordinates of the stem-leaf connection point corresponding to the leaf, represents the coordinates of the end point corresponding to the leaf.

[0011] Furthermore, the specific steps of step S4 are as follows: S41. Use CloudCompare software to preprocess the 3D point cloud data obtained by the depth camera to remove noise; S42. Use the PointNeXt algorithm to segment the preprocessed point cloud to separate the main body and the leaves; S43. Use the Laplace transform to extract the skeleton of the separated leaves and extract its skeleton information; S44. Extract the stem-leaf connection points in the skeleton; S45. Calculate the leaf inclination angle of the maize plant.

[0012] Furthermore, the specific steps of step S45 are as follows: First, establish a local coordinate system for each leaf, use the stem-leaf connection point as the origin, extract the main direction of the leaf point cloud through the PCA method, and define the leaf extension direction as the Z-axis; Perform PCA analysis on the point cloud of each leaf. If the leaf point cloud is flat, use the Hough transform to fit the plane model of the point cloud of this leaf to obtain the main plane equation describing the spatial attitude of the leaf, and extract the normal vector of the main plane as the leaf normal vector. If the leaf point cloud is not flat, extract the eigenvector corresponding to the minimum eigenvalue as the leaf normal vector; Calculate the angle between the leaf normal vector and the Z-axis of the local coordinate system to obtain the leaf inclination angle of the leaf.

[0013] Furthermore, determining whether the leaf point cloud is flat is specifically as follows: Calculate the covariance matrix of the leaf point cloud and solve the eigenvalues. If , it is considered that the leaf point cloud is flat, otherwise, it is considered that the leaf point cloud is not flat, where, represents the maximum eigenvalue, denotes the minimum eigenvalue, is an empirical threshold, .

[0014] The beneficial effects of the method of the present invention are as follows: In the method of obtaining the leaf inclination angles of each leaf of a corn plant by performing multi-dimensional processing on a depth image, while streamlining the calculation amount, high detection accuracy can still be ensured, and the existing U-Net deep learning model is improved so that it can be more accurately applied to the extraction of the corn plant skeleton, and a technical means of fitting the main stem is added, fully considering the true morphology of the corn plant, and improving the calculation accuracy of the leaf inclination angle; In the method of obtaining the leaf inclination angles of each leaf of a corn plant by performing 3D point cloud processing on the 3D point cloud data of the depth image, by performing PCA analysis on the point cloud of each leaf, redefining the flatness of the leaf, and dealing with different situations, the calculation of the leaf inclination angle is made more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flowchart of the method described in the embodiment of the present invention; Figure 2 is a structural diagram of the U-Net-ACS model in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] Embodiment 1 This embodiment provides a method for in-situ extraction of the leaf inclination angle of a corn plant based on a depth camera, including the following steps: S1. Use a depth camera to collect the depth image and the 3D point cloud data of the depth image during the mid-growth stage of the corn; S2. Perform multi-dimensional processing on the depth image to obtain the leaf inclination angles of each leaf of the corn plant; S3. Perform 3D point cloud processing on the 3D point cloud data of the depth image to obtain the leaf inclination angles of each leaf of the corn plant; S4. Calculate the determination coefficients and of the regression curves of the leaf inclination angles of each leaf of the corn plant obtained after performing the processing of step S2 and step S3 on the depth image and the 3D point cloud data of the depth image of the same plant respectively; If and are both greater than or equal to 0.88, then proceed to step S5; If and If one of the values is greater than or equal to 0.88, then proceed to step S6; if and are both less than 0.88, then proceed to step S7; S5. Perform weighted fusion on the processing results of step S2 and step S3 to obtain the leaf inclination angles of each leaf of the corn plant; Through obtain the leaf inclination angle after weighted fusion, wherein, is the final angle result after fusion, is the angle calculated by using step S2, is the angle calculated by using step S3, , .

[0018] S6. Use the result obtained by the method corresponding to the coefficient of determination of the regression curve being greater than or equal to 0.88 as the leaf inclination angle of each leaf of the corn plant; S7. Determine the occlusion and overlap of the leaves, the fracture of the skeleton, and the missing or error of the point cloud, and transmit the data back to guide the experimenter to optimize steps S2 and S3, and after optimization, return to step S1 to continue.

[0019] In this embodiment, the RGB-D camera Intel RealSense is used as the image acquisition device to obtain the small-scale phenotypic parameters in the middle growth stage of corn. The RGB-D camera Intel Realsense simultaneously obtains color images, depth images, and 3D point cloud data; when the device takes pictures, it is parallel to the upper-middle part of the corn plant, and the preset distance from the plant is 120 cm. The middle growth stage of corn includes the large trumpet mouth stage and the tasseling stage; the small-scale phenotypic parameters include the parameters corresponding to the stems, leaves, and stalks in the middle growth stage of corn.

[0020] Embodiment 2, This embodiment further limits Embodiment 1 and further describes step S3 in Embodiment 1.

[0021] As Figure 1 shown, step S3 is specifically as follows: S31. Image segmentation: Perform color-based threshold segmentation processing on the corn plant in the depth image, and screen the pixels of the image so that the image can automatically distinguish the corn plant and the background area; S32. Image binarization processing: After completing the image segmentation, perform binarization processing on the image so that the pixel points in the image only present two colors, black and white; S33. Morphological processing: First perform morphological opening operation processing on the image and then perform morphological closing operation processing; S34. Skeletonization processing: Use the U-Net-ACS model to extract the skeleton of the corn plants in the image; As Figure 2 shown, the U-Net-ACS model is obtained by improving on the basis of the U-Net model. Specifically, in the encoding process of the U-Net model, the CBAM attention mechanism is introduced after the convolutional operation, in the decoding process of the U-Net model, the ASPP module is introduced after the convolutional operation, and in the training process of the U-Net-ACS model, the deep supervision mechanism is adopted.

[0022] CBAM consists of channel attention (CA) and spatial attention (SA).

[0023] The role of spatial attention is to focus on important regions in the image in the spatial dimension. By calculating the importance of different positions in the feature map, it improves the model's response ability to the target region.

[0024] In CBAM, the spatial attention module usually performs global information extraction in the channel dimension through max pooling and average pooling, then uses convolutional operations to generate spatial attention weights, and normalizes them through the Sigmoid activation function, thereby adjusting the weights of the feature map.

[0025] In the present invention, this mechanism helps the U-Net model to focus more on the plant skeleton region and improve the segmentation accuracy.

[0026] The deep supervision mechanism is a training strategy and has nothing to do with spatial attention. Its core idea is to add auxiliary supervision signals (auxiliary losses) at multiple stages of the decoder, that is, to output auxiliary segmentation results at different levels of the U-Net, rather than only supervising at the final output layer.

[0027] In the present invention, the deep supervision mechanism is optimized in combination with the U-Net structure, that is, additional prediction branches are added at multiple stages (feature maps of different resolutions) of the decoder, and auxiliary losses such as cross-entropy loss or Dice loss are calculated. This way ensures that features at different scales can obtain effective supervision, making the model more accurate in restoring fine structures, and ultimately improving the accuracy and robustness of skeleton extraction.

[0028] CBAM = Channel Attention (CA) + Spatial Attention (SA), which is used to enhance the feature extraction ability.

[0029] ASPP (Atrous Spatial Pyramid Pooling) is used for multi-scale feature extraction to improve the segmentation ability for targets of different sizes.

[0030] The deep supervision mechanism conducts auxiliary supervision at multiple stages of the decoder to improve the segmentation stability and convergence speed.

[0031] These three mechanisms are independent of each other but can be used in combination, and their combination helps to improve the segmentation effect of the U-Net variant.

[0032] S35, Corner Detection: Detect the corners in the corn plant skeleton through the Harris corner detection algorithm, and determine the stem-leaf connection points in the corn plant skeleton. Divide the main stem corners of the corn plant and the main stem corners of the corn leaf with the stem-leaf connection points as the boundary; The determination of the stem-leaf connection points in the corn plant skeleton is specifically as follows: For each corner, use a neighborhood window to traverse the corners in its neighborhood. If the number of corners in its neighborhood is greater than or equal to three, it is considered as a stem-leaf connection point. S36, Angle Calculation: Fit the main stem line of the corn plant through the main stem corners of the corn plant, determine the end points of the corn leaf through the main stem corners of the corn leaf, and calculate the leaf inclination angle of the corn plant leaf through the main stem line of the corn plant, the stem-leaf connection point, and the end point of the corn leaf.

[0033] Fitting the main stem line of the corn plant through the main stem corners of the corn plant: Obtained, where, represents the calculation by weighted least squares method, is the weight value, , and are the linear coefficients to be fitted, corresponding to the intercept and slope of the main stem respectively, represents the coordinates of the corner, represents the number of corners, represents the coordinate mean of the corner.

[0034] Calculating the leaf inclination angle of the corn plant leaf through the main stem line of the corn plant, the stem-leaf connection point, and the end point of the corn leaf: Obtained, where, represents the leaf inclination angle, , represents the slope of the main direction of the leaf, represents the coordinates of the stem-leaf connection point corresponding to the leaf, represents the coordinates of the end point corresponding to the leaf.

[0035] Example 3 This embodiment further limits Embodiment 1 and further explains step S4 in Embodiment 1.

[0036] As Figure 1 shown, step S4 is specifically as follows: S41. Use CloudCompare software to preprocess the 3D point cloud data obtained by the depth camera to remove noise; The specific steps include: Select a suitable filter (such as mean filtering, median filtering, etc.), remove noise through filtering and improve the quality of the point cloud. Adjust the sampling density parameter to ensure that the point cloud is evenly distributed and the sampling density is appropriate, avoiding areas that are too dense or too sparse from affecting subsequent analysis. Remove outliers. Use the built-in outlier detection tool in CloudCompare (such as based on statistical values, curvature, or plane fitting methods, etc.) to remove points that deviate significantly or do not conform to the data distribution. Through the above preprocessing steps, the quality of the point cloud can be significantly improved, laying a foundation for subsequent analysis.

[0037] S42. Use the PointNeXt algorithm to segment the preprocessed point cloud and separate the main body and the leaves; During the setup of PointNeXt: Select a suitable feature extraction method (such as curvature, gradient, etc.) according to the geometric characteristics of the leaves, and these features can effectively distinguish the leaves from other parts. Adjust the segmentation parameters to ensure that the main body and the leaves can be accurately separated. For example, by adjusting the curvature threshold or plane fitting parameters, optimize the segmentation effect.

[0038] S43. Use the Laplace transform to extract the skeleton of the separated leaves and extract their skeleton information; The Laplace transform can effectively capture the main structural features of the point cloud and generate a connected skeleton framework. Further optimize and streamline according to the skeleton results. By adjusting the skeleton simplification parameters or using geometric filtering methods, ensure that the skeleton representation is accurate and concise.

[0039] S44. Extract the stem-leaf connection points in the skeleton; Taking each skeleton point as the center, construct a three-dimensional spherical neighborhood. If there are three or more skeleton extension directions with large direction differences in the neighborhood, then this point is identified as a stem-leaf connection point.

[0040] S45. Calculate the leaf inclination angle of the maize plant leaves: First, establish a local coordinate system for each leaf, with the stem-leaf connection point as the origin, extract the main direction of the leaf point cloud through the PCA (Principal Component Analysis) method, and define the leaf extension direction among them as the Z-axis; Perform PCA analysis on the point cloud of each blade. If the point cloud of the blade is flat, use the Hough transform to fit a plane model to the point cloud of the blade to obtain the main plane equation describing the spatial attitude of the blade, and extract the normal vector of the main plane as the blade normal vector. If the point cloud of the blade is not flat, extract the eigenvector corresponding to the minimum eigenvalue as the blade normal vector; Calculate the angle between the blade normal vector and the Z-axis of the local coordinate system to obtain the blade leaf inclination angle.

[0041] The so-called "minimum eigenvalue" specifically refers to: when performing principal component analysis (PCA) on the blade point cloud, the smallest one among the eigenvalues obtained by calculating its covariance matrix.

[0042] More specifically: the covariance matrix of the point cloud reflects the distribution variances in each direction; the three eigenvalues of the covariance matrix correspond to the variances of the point cloud in the three principal axis directions; the eigenvector corresponding to the minimum eigenvalue represents the normal vector direction in the point cloud (that is, the direction with the smallest change); this direction is usually perpendicular to the local plane where the point cloud is mainly distributed and can be approximately regarded as the normal direction of the blade.

[0043] Determining whether the blade point cloud is flat is specifically as follows: Calculate the covariance matrix of the blade point cloud and solve for the eigenvalues. If , it is considered that the blade point cloud is flat, otherwise, it is considered that the blade point cloud is not flat, where represents the maximum eigenvalue, represents the minimum eigenvalue, is an empirical threshold, .

[0044] Example 4 This example further limits step S7 in Example 1. To verify the accuracy and robustness of the phenotypic parameter precise acquisition method, verification is required. Verify the feasibility and accuracy of the verification method. If the expected accuracy is not achieved, perform optimization and result analysis. The overall process is as Figure 1 shown.

[0045] Verification of the accuracy of the method, including the influence of the camera placement angle, the bracket placement height, and the distance between the camera and the plant on the result accuracy. In the field experiment, repeatedly optimize the above preset data according to the results to reduce the systematic error and cumulative error that may be caused in the measurement.

[0046] Verification of the robustness of the method. Select the mid-growth stage of corn in Northeast China and perform tests under different working conditions such as sunny days, cloudy days, and rainy days to verify the robustness of the algorithm; verify the accuracy by comparing with the results of actual manual measurement. According to the test results, repeatedly optimize the method to meet the requirements for the acquisition accuracy of small-size phenotypic parameters in the mid-growth stage of corn.

[0047] The specific repeated optimization method is as follows: In S2, the key points of optimization include image segmentation, skeleton extraction, and corner detection. For image segmentation, the main optimized parameter is the color threshold range to more accurately distinguish corn plants from the background and improve the integrity of the target area. In skeleton extraction, the optimized parameters include the loss function weights of the U-Net-ACS network, the hyperparameters of the attention module, and the data augmentation strategy adopted during the training process to improve the accuracy and robustness of skeleton recognition. For corner detection, the response function threshold and non-maximum suppression parameters of the Harris algorithm are optimized to ensure the accuracy of corners and reduce false detections and missed detections.

[0048] In S3, the key points of optimization include point cloud preprocessing, leaf segmentation, and skeleton extraction. In point cloud preprocessing, the window size and filtering method (such as mean filtering or bilateral filtering) of the filter, as well as the point cloud sampling density, are optimized to improve the quality of the point cloud. During leaf segmentation, the curvature threshold, feature extraction method, and classification loss function of PointNeXt are optimized to ensure the accurate separation of the main body from the leaves. For skeleton extraction, the parameters of the Laplace transform are optimized to make the skeleton more coherent, and the skeleton simplification parameters are adjusted to reduce redundant points and improve the calculation efficiency. Finally, when calculating the leaf inclination angle, the PCA calculation window size and the fitting accuracy of the Hough transform are optimized to make the leaf normal vector more accurate and improve the stability and reliability of leaf inclination angle measurement.

[0049] In step S7, when guiding cloud preprocessing, leaf segmentation, and skeleton extraction, it is carried out according to the feedback of leaf occlusion and overlap, skeleton breakage, and point cloud missing or error. The specific determination for leaf occlusion and overlap, skeleton breakage, and point cloud missing or error is as follows: Occlusion and overlap: The number of skeleton breaks is greater than 2 or the average boundary gradient of the connected region is lower than the threshold; Skeleton breakage: The length of the connected segment of the main leaf skeleton is less than 60% of the total length; Point cloud missing or error: There is a sudden change in the Z-direction depth or the proportion of 0-value points exceeds 10%, and the PCA main direction fails.

[0050] The above determination will be used as an indicator to guide the repeated optimization method.

Claims

1. A method for in-situ extraction of maize plant leaf inclination angle based on a depth camera, characterized in that, The method includes the following steps: S1. Use a depth camera to collect depth images and 3D point cloud data of the depth images in the mid - growth stage of corn; S2. Perform multi - dimensional processing on the depth images to obtain the leaf inclination angles of each leaf of the corn plant; S3. Perform 3D point cloud processing on the 3D point cloud data of the depth images to obtain the leaf inclination angles of each leaf of the corn plant; S4. Calculate the coefficient of determination of the regression curves of the leaf angles of each leaf of the corn plant obtained after performing the processing in step S2 and step S3 on the depth image of the same plant and the 3D point cloud data of the depth image respectively. and ; if and are both greater than or equal to 0.88, then proceed to step S5; if and one of the values is greater than or equal to 0.88, then proceed to step S6; if and are both less than 0.88, then proceed to step S7; S5. Perform weighted fusion on the processing results of steps S2 and S3 to obtain the leaf inclination angles of each leaf of the corn plant; S6. Use the result obtained by the method corresponding to the determination coefficient of the regression curve being greater than or equal to 0.88 as the leaf inclination angles of each leaf of the corn plant; S7. Judge the presence of occlusion and overlap of leaves, skeleton breakage, and point cloud missing or error, and transmit data back to guide the experimenter to optimize steps S2 and S3. After optimization, return to step S1 to continue.

2. The method for in-situ extraction of maize plant leaf inclination angle based on a depth camera according to claim 1, characterized in that, The mid - growth stage of corn includes the large trumpet - mouth stage and the tasseling stage.

3. The method for in-situ extraction of maize plant leaf inclination angle based on a depth camera according to claim 2, characterized in that, The specific content of step S3 is as follows: S31. Image segmentation: Perform color - based threshold segmentation processing on the corn plants in the depth images, and screen the pixels of the images so that the images can automatically distinguish the corn plants and the background areas; S32. Image binarization processing: After completing image segmentation, perform binarization processing on the images so that the pixel points in the images only present two colors, black and white; S33. Morphological processing: First perform morphological opening operation processing on the images and then perform morphological closing operation processing; S34. Skeletonization processing: Use the U - Net - ACS model to extract the skeletons of the corn plants in the images; S35. Corner point detection: Detect the corner points in the skeletons of the corn plants through the Harris corner detection algorithm, and judge the stem - leaf connection points in the skeletons of the corn plants. Divide the main stem corner points of the corn plants and the main stem corner points of the corn leaves with the stem - leaf connection points as the boundary; S36. Angle calculation: Fit the main stem line of the corn plant through the main stem corner points of the corn plant, judge the end points of the corn leaves through the main stem corner points of the corn leaves, and calculate the leaf inclination angles of the corn plant leaves through the main stem line of the corn plant, the stem - leaf connection points, and the end points of the corn leaves.

4. The method for in-situ extraction of maize plant leaf inclination angle based on a depth camera according to claim 3, wherein The U - Net - ACS model is obtained by improving on the basis of the U - Net model. Specifically, in the encoding process of the U - Net model, the CBAM attention mechanism is introduced after the convolution operation, in the decoding process of the U - Net model, the ASPP module is introduced after the convolution operation, and in the training process of the U - Net - ACS model, the deep supervision mechanism is adopted.

5. The method for in-situ extraction of maize plant leaf inclination angle based on a depth camera according to claim 4, wherein The specific method of judging the stem - leaf connection points in the skeletons of the corn plants is as follows: For each corner point, use a neighborhood window to traverse the corner points in its neighborhood. If the number of corner points in its neighborhood is greater than or equal to three, then it is considered as a stem - leaf connection point.

6. The in-situ extraction method of maize plant leaf inclination angle based on a depth camera according to claim 5, characterized in that, Fitting the main stem line of the corn plant through the main stem corner points of the corn plant by: obtained, where, represents the weighted least squares calculation, is the weight value, , and are the linear coefficients to be fitted, corresponding to the intercept and slope of the main stem respectively, represents the coordinates of the corner points, represents the number of corner points, represents the mean value of the corner point coordinates.

7. The method for in-situ extraction of maize plant leaf inclination angle based on a depth camera according to claim 6, characterized in that, Calculating the leaf inclination angles of the corn plant leaves through the main stem line of the corn plant, the stem - leaf connection points, and the end points of the corn leaves by: obtained, wherein, represents the leaf inclination angle, , represents the main direction slope of the leaf, represents the coordinate of the stem-leaf connection point corresponding to the leaf, represents the coordinate of the end point corresponding to the leaf.

8. The in-situ extraction method of maize plant leaf inclination based on a depth camera according to claim 7, characterized in that The specific content of step S4 is as follows: S41. Use CloudCompare software to pre - process the 3D point cloud data obtained by the depth camera to remove noise; S42. Use the PointNeXt algorithm to segment the preprocessed point cloud and separate the main body and leaves; S43. Use the Laplace transform to extract the skeleton of the separated leaves and extract their skeleton information; S44. Extract the stem-leaf connection points in the skeleton; S45. Calculate the leaf inclination angle of the maize plant leaves.

9. The method for in-situ extraction of maize plant leaf inclination angle based on a depth camera according to claim 8, characterized in that, The specific steps of step S45 are as follows: First, establish a local coordinate system for each leaf, use the stem-leaf connection point as the origin, extract the main direction of the leaf point cloud through the PCA method, and define the leaf extension direction as the Z axis; Perform PCA analysis on the point cloud of each leaf. If the leaf point cloud is flat, use the Hough transform to fit the plane model of the leaf point cloud to obtain the main plane equation describing the spatial attitude of the leaf, and extract the normal vector of the main plane as the leaf normal vector. If the leaf point cloud is not flat, extract the eigenvector corresponding to the minimum eigenvalue as the leaf normal vector; Calculate the angle between the leaf normal vector and the Z axis of the local coordinate system to obtain the leaf inclination angle.

10. The in-situ extraction method of maize plant leaf inclination angle based on a depth camera according to claim 9, wherein, Determining whether the leaf point cloud is flat is specifically: Calculate the covariance matrix of the blade point cloud and solve for the eigenvalues. If , it is considered that the blade point cloud is flat; otherwise, it is considered that the blade point cloud is uneven, where represents the maximum eigenvalue, represents the minimum eigenvalue, is an empirical threshold, .

Images