Rice plant segmentation method and system based on view angle transformation and dense point cloud

Abstract

The invention discloses a rice plant segmentation method and system based on view angle transformation and dense point clouds, and the method comprises the steps: scanning a rice field through employing an unmanned plane laser radar system, and obtaining the dense point clouds of a rice canopy; performing angle transformation processing on the dense point cloud by adopting an overlooking angle to obtain a two-dimensional image of the dense point cloud at the overlooking angle; calculating a homography matrix from the dense point cloud to the two-dimensional image, and establishing an accurate mapping relation between the three-dimensional point cloud and the two-dimensional image; for the two-dimensional image, carrying out segmentation marking on the rice plant by adopting a region marking method so as to realize accurate segmentation and individual marking of the rice plant on a two-dimensional plane; and endowing each point in the point cloud with a corresponding region mark number by using a mapping relationship between the three-dimensional point cloud and the two-dimensional image, and further obtaining a rice plant segmentation result of the whole dense point cloud. According to the method, high-precision rice plant segmentation is realized, and more reliable and efficient technical support is provided for precision agricultural management and phenotype analysis of rice.

Description

Technical Field

[0001] The present invention relates to the technical field of crop phenotype and unmanned aerial vehicle remote sensing, and in particular to a rice plant segmentation method and system based on perspective transformation and dense point cloud. Background Art

[0002] With the rapid development of precision agriculture, the use of UAV remote sensing technology to monitor rice phenotypic traits has become an important means of agricultural management. In recent years, UAVs equipped with lidar systems have acquired high-density three-dimensional point cloud data, providing a new technical approach and a richer data source for high-throughput rice phenotyping.

[0003] In rice phenotyping monitoring, accurate identification and segmentation of individual rice plants is a key step, which is crucial for subsequent growth parameter extraction and agricultural situation analysis. However, due to the morphological changes of rice plants during growth and complex environmental factors in the field, it is still a huge challenge to accurately segment individual rice plants only from 3D point cloud data.

[0004] The current mainstream plant point cloud segmentation methods are mainly based on spatial clustering algorithms. These methods work well when dealing with sparsely distributed plants, but it is often difficult to accurately distinguish adjacent plants in densely planted crops such as rice. In addition, these methods usually require a lot of manual parameter adjustments and are difficult to adapt to rice plants of different growth stages and varieties.

[0005] In the existing technology, the processing and analysis of three-dimensional point cloud data mainly have the following deficiencies: 1. Lack of effective perspective transformation methods: Three-dimensional point cloud data usually needs to be converted into two-dimensional images from a bird's-eye view for easy analysis, but existing methods often find it difficult to retain sufficient spatial information.

[0006] 2. Insufficient data utilization: Most methods only use the spatial information of point clouds and fail to fully utilize the two-dimensional image information generated by point clouds.

[0007] 3. Poor adaptability of segmentation algorithms: Existing methods are often difficult to adapt to the changes in morphological characteristics of rice at different growth stages. Summary of the invention

[0008] The first purpose of the present invention is to solve the problems of low accuracy and poor efficiency of traditional rice plant segmentation technology when dealing with high-density planting and serious plant overlap, and to provide a rice plant segmentation method based on perspective transformation and dense point cloud, which makes full use of the high-density point cloud data collected by the unmanned aerial vehicle laser radar system, and converts the three-dimensional point cloud data into a two-dimensional image from a top-down angle through innovative perspective transformation technology. At the same time, the homography matrix is ​​used to establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image, and combined with the regional labeling method, high-precision rice plant segmentation is achieved, providing more reliable and efficient technical support for precision agricultural management and phenotypic analysis of rice.

[0009] A second object of the present invention is to provide a rice plant segmentation system based on viewpoint transformation and dense point cloud.

[0010] The first objective of the present invention is achieved by the following technical solution: A rice plant segmentation method based on perspective transformation and dense point cloud, comprising: S1. Use the UAV-mounted LiDAR system to scan the rice field and obtain a dense point cloud of the rice canopy. S2, performing angle transformation processing on the dense point cloud at a top-view angle to obtain a two-dimensional image of the dense point cloud at a top-view angle; wherein the top-view angle is vertically downward to maximize the separability between plants; S3, calculating the homography matrix from the dense point cloud to the two-dimensional image, and establishing an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; S4. For the two-dimensional image, a region marking method is used to segment and mark the rice plants, so as to achieve accurate segmentation and individual marking of the rice plants on the two-dimensional plane; S5. Using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, a corresponding region marking number is assigned to each point in the point cloud, thereby obtaining a rice plant segmentation result of the entire dense point cloud.

[0011] Further, the step S1 comprises: S101, on the UAV ground station, the rice field is set as the scanning area, and the heading overlap rate and the lateral overlap rate of the scanning are set, and the UAV ground station automatically generates a route according to the operation information; wherein the heading overlap rate is set to 60%-80%, and the lateral overlap rate is set to 50%-70%, so as to ensure that the required dense point cloud data is obtained; S102, the UAV autonomously flies according to the planned route, automatically performs laser radar scanning according to the set overlap rate, and records the position and attitude information of the scan data; wherein the position and attitude information includes longitude, latitude, altitude, heading angle, pitch angle and roll angle; the flight altitude of the UAV is set to 20-50 meters, and the flight speed is 3-8 meters / second to balance data collection efficiency and point cloud density; S103, the drone uploads the collected dense point cloud data to the server in real time based on the data link; wherein the real-time uploaded data includes both the point cloud coordinates and the reflection intensity information of each point; the drone and the server establish a secure data transmission channel based on a preset communication protocol; the data link adopts a 4G / 5G network or a dedicated wireless communication module, and the transmission rate is not less than 10Mbps; S104, after receiving the point cloud data, the server performs a real-time data quality check, including evaluation of point cloud density, coverage, and data integrity; if it is found that the data quality does not meet the requirements, a rescanning instruction is automatically sent to the drone; S105, preprocessing the received original point cloud data, including noise removal, ground point filtering and point cloud registration; wherein, the noise removal adopts a statistical outlier filtering algorithm, the ground point filtering adopts a progressive morphological filtering algorithm, and the point cloud registration adopts an iterative closest point algorithm; S106, storing the pre-processed point cloud data in a standard format and establishing a spatial index structure for subsequent rapid access and processing; S107, using adaptive point cloud density control algorithm to dynamically adjust the laser radar scanning frequency and the flight speed of the drone ,satisfy: ; In the formula, and are the minimum and maximum point cloud densities set, is the scanning bandwidth; S108, realize multi-sensor data fusion, combine RTK-GPS, IMU and lidar data, use extended Kalman filter algorithm to optimize point cloud positioning accuracy, the state estimation equation is: ; In the formula, is the optimal state estimate, is the Kalman gain, is the observation vector, is the observation matrix, It's time A priori state estimate of .

[0012] Further, the step S2 comprises: S201. Based on the principal component analysis method, determine the main plane of the dense point cloud and calculate the covariance matrix C of the point cloud: ; In the formula, It is the first Points, is the centroid of the point cloud, N is the number of point clouds; S202, using the covariance matrix C to calculate the normal vector of the main plane: Perform eigenvalue decomposition on the covariance matrix C; Select the eigenvector corresponding to the smallest eigenvalue as the normal vector of the principal plane ; The normal vector As the initial looking down direction; If the normal vector If the angle with the vertical direction exceeds the preset threshold, the following optimization is performed: Adopt iterative plane fitting algorithm; set the maximum number of iterations and convergence threshold; iterative process: filter point set, recalculate covariance matrix and update normal vector; stop when convergence condition is met or maximum number of iterations is reached; finally optimize normal vector as determined top view direction; S203: According to the determined overlooking direction, coordinate transformation is performed on the dense point cloud, as follows: Calculate the rotation matrix , rotate the normal vector to Axis parallel; For point cloud Apply the rotation matrix Among them, is the normal vector, is the Z-axis unit vector, and the rotation matrix is ​​calculated using the quaternion or Euler angle method , so that ; Calculate the bounding box of the transformed point cloud and determine the new coordinate origin; Translate the point cloud so that the center of its bounding box coincides with the new coordinate origin; S204, projecting the point cloud after coordinate transformation onto a horizontal plane to generate a two-dimensional grid image, wherein the value of each grid is determined by the number of points falling into the grid or the height statistics, as follows: Determine the grid size and set it to 1-2 times the average spacing of the point cloud; Create a blank raster image, sized to cover the entire point cloud projection area; For each point, calculate its position in the raster image; Update grid values. The statistics you can choose include maximum height, average height or point density. Normalize the generated raster image and map the pixel values ​​to the range of 0-255; S205, post-processing the generated two-dimensional raster image to enhance image quality and analyzability, as follows: Apply Gaussian filtering to remove noise; Use adaptive histogram equalization to enhance contrast; Morphological operations are used to remove small isolated areas; S206, saving the processed two-dimensional raster image in a common image format, and recording the geographic reference information of the image so as to maintain consistency with the original point cloud data; S207, introduce adaptive multi-scale point cloud downsampling algorithm, according to the local point cloud density Dynamically resize voxels ; In the formula, It is an adjustable parameter used to balance downsampling efficiency and detail preservation; S208. Apply principal component analysis to enhance the structural features of the point cloud.

[0013] Further, the step S3 comprises: S301, selecting at least 4 pairs of corresponding points in the 3D point cloud and the 2D image to form a point pair set, wherein the perspective projection transformation from 3D to 2D has 8 degrees of freedom, and each pair of corresponding points provides two constraint equations, therefore, at least 4 pairs of corresponding points are required to uniquely determine the transformation; Select points with significant features in the 3D point cloud; Find the pixel positions corresponding to the points selected in the above three-dimensional point cloud in the two-dimensional image to form point pairs; Make sure the selected point pairs are evenly distributed and cover the entire area of ​​interest; S302, using a direct linear transformation algorithm to calculate the initial homography matrix H, as follows: Construct a system of linear equations, providing two equations for each pair of corresponding points; Use singular value decomposition to solve the linear equations and obtain the initial homography matrix H; S303, using a random sample consistency algorithm to optimize the initial homography matrix H to improve mapping accuracy, as follows: S3031. Randomly select a point pair subset with the minimum number of points, which belongs to the point pair set; S3032, calculating a candidate homography matrix using the point pairs in the selected point pair subset; S3033, calculating the projection errors of all point pairs under the candidate homography matrix; S3034, counting the number of inliers whose errors are within a preset threshold; S3035, repeat the above steps S3031-S3034 multiple times, and select the model with the most inliers; S3036, recalculate the final homography matrix using all the inliers in the model with the most inliers ; S304, using the Levenberg-Marquardt algorithm Nonlinear optimization is performed to further improve the accuracy of the homography matrix, as follows: Define the objective function as the sum of squared reprojection errors of a set of point pairs; Based on this objective function, iterative optimization is performed with the matrix optimized by RANSAC as the initial value; In each iteration, the damping factor is adjusted to balance convergence speed and stability; When the error change is less than the preset threshold or the maximum number of iterations is reached, the optimization is stopped to obtain the homography matrix with the best accuracy. ; S305, evaluate the final optimized homography matrix The accuracy is as follows: Compute the average reprojection error for a set of point pairs; Calculate the spatial distribution consistency of the mapped points; If the average reprojection error and spatial distribution consistency accuracy do not meet the requirements, consider increasing the number of corresponding points or reselecting corresponding points; S306, introduce deep learning-assisted feature matching algorithm, use convolutional neural network to extract feature descriptors of point cloud and image, network loss function Defined as: ; In the formula, is the descriptor loss, is the detector loss, is the balance factor; S307, using adaptive RANSAC algorithm to dynamically adjust the number of iterations : ; In the formula, is the expected probability of success, is the interior point ratio, is the minimum sample set size.

[0014] Further, the step S4 comprises: S401, preprocessing the two-dimensional image, including denoising and contrast enhancement; S402, applying a watershed algorithm to perform preliminary segmentation on the preprocessed two-dimensional image; S403, optimizing the segmentation result using morphological operations, including opening operations and closing operations; S404, using connected region analysis to assign a unique label number to each segmented region; S405, introduce a multi-scale superpixel segmentation algorithm, combine color and spatial information, and define a superpixel similarity metric : ; In the formula, and are color distance and spatial distance respectively, and is the corresponding normalization factor; S406, applying the graph cut algorithm to optimize the segmentation results and minimize the energy function : ; In the formula, Assign labels, and Represents a pixel in an image; Is a data item, indicating that the label Assign to pixel the cost; is a smoothing term, representing adjacent pixels and Penalty for being assigned a different label; Represents a set of adjacent pixel pairs.

[0015] Further, the step S5 comprises: S501, using homography matrix Each point in the 3D point cloud Projected onto the two-dimensional image plane, it is represented as follows: ; in, , , Represents the 3D coordinates of each point in the 3D point cloud, , represents the horizontal and vertical coordinates of pixels in a two-dimensional image, is the final projection coordinate, is the corresponding scaling factor; S502, determining the segmentation area to which the projection point belongs according to the position of the projection point; S503, assigning the marking numbers of the corresponding segmented regions to the points in the three-dimensional point cloud; S504: For the points whose projection positions fall on the boundary of the segmented area, the nearest neighbor rule is used to determine the belonging, as follows: Identify the points whose projection positions fall on the boundary of the segmented regions; for each such point, calculate its distance to the adjacent segmented regions; assign the point to the segmented region with the closest distance; if there are multiple segmented regions with equal distances, select the region with the largest area to determine its affiliation.

[0016] The second object of the present invention is achieved by the following technical solution: a rice plant segmentation system based on perspective transformation and dense point cloud, used to implement the above-mentioned rice plant segmentation method based on perspective transformation and dense point cloud, comprising: A point cloud acquisition unit is used to scan the rice field using a UAV-mounted laser radar system to obtain a dense point cloud of the rice canopy; A perspective transformation unit, configured to perform angle transformation processing on the dense point cloud at a top-view angle to obtain a two-dimensional image of the dense point cloud at a top-view angle; A mapping establishment unit, used to calculate the homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; A two-dimensional segmentation unit, used for segmenting and marking rice plants in the two-dimensional image using a region marking method; A three-dimensional marking unit, used to assign a corresponding region marking number to each point in the point cloud by using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, so as to obtain a segmentation result of the entire dense point cloud; Adaptive optimization unit, used to realize point cloud density control, multi-sensor data fusion, multi-scale feature extraction and temporal consistency constraint functions.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. Innovative method of perspective transformation and fusion of dense point cloud: The present invention creatively converts 3D dense point cloud data into a 2D image from a top-down perspective and establishes an accurate 3D to 2D mapping relationship. This method cleverly combines the rich information of 3D data with the high efficiency of 2D image processing, providing a new idea for rice plant segmentation and effectively solving the segmentation problem in cases of high-density planting and severe plant overlap.

[0018] 2. Selection of the optimal overlooking angle: The present invention uses a vertically downward overlooking angle to perform angle transformation processing on the dense point cloud. This method can maximize the separability between plants. By selecting the optimal overlooking angle, the accuracy and efficiency of the subsequent segmentation process are significantly improved. It is particularly suitable for densely planted rice field environments.

[0019] 3. Accurate 3D to 2D mapping technology: This invention proposes an innovative homography matrix calculation method to establish an accurate mapping relationship between 3D point clouds and 2D images. This high-precision mapping technology not only improves the conversion accuracy of 3D point clouds to 2D images, but also provides a reliable foundation for subsequent plant segmentation and labeling.

[0020] 4. Efficient two-dimensional image segmentation method: The present invention adopts the region labeling method to segment and label the rice plants in the two-dimensional image, realizing the accurate segmentation and individual labeling of the rice plants in the two-dimensional plane. This method makes full use of the high efficiency of two-dimensional image processing while ensuring the accuracy of the segmentation results.

[0021] 5. Three-dimensional point cloud precise labeling technology: The present invention innovatively proposes a precise labeling method from two-dimensional segmentation results to three-dimensional point clouds. By utilizing the established mapping relationship between three-dimensional point clouds and two-dimensional images, a corresponding area labeling number is assigned to each point in the point cloud, thereby achieving high-precision segmentation and labeling of the entire dense point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the process of the present invention.

[0023] Figure 2 Schematic diagram of dense point cloud processing flow.

[0024] Figure 3 Schematic diagram of the rice plant segmentation process.

[0025] Figure 4 Schematic diagram of the composition of the system of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0027] Example 1 like Figures 1 to 3 As shown, this embodiment discloses a rice plant segmentation method based on perspective transformation and dense point cloud, and the specific situation is as follows: S1. Use the UAV-mounted LiDAR system to scan the rice field and obtain the dense point cloud of the rice canopy, as follows: S101, on the UAV ground station, the rice field is set as the scanning area, and the heading overlap rate and the lateral overlap rate of the scanning are set, and the UAV ground station automatically generates a route according to the operation information; wherein the heading overlap rate is set to 60%-80%, and the lateral overlap rate is set to 50%-70%, so as to ensure that the required dense point cloud data is obtained; S102, the UAV autonomously flies according to the planned route, automatically performs laser radar scanning according to the set overlap rate, and records the position and attitude information of the scan data; wherein the position and attitude information includes longitude, latitude, altitude, heading angle, pitch angle and roll angle; the flight altitude of the UAV is set to 20-50 meters, and the flight speed is 3-8 meters / second to balance data collection efficiency and point cloud density; S103, the drone uploads the collected dense point cloud data to the server in real time based on the data link; wherein the real-time uploaded data includes both the point cloud coordinates and the reflection intensity information of each point; the drone and the server establish a secure data transmission channel based on a preset communication protocol; the data link adopts a 4G / 5G network or a dedicated wireless communication module, and the transmission rate is not less than 10Mbps; S104, after receiving the point cloud data, the server performs a real-time data quality check, including evaluation of point cloud density, coverage, and data integrity; if it is found that the data quality does not meet the requirements, a rescanning instruction is automatically sent to the drone; S105, preprocessing the received original point cloud data, including noise removal, ground point filtering and point cloud registration; wherein, the noise removal adopts a statistical outlier filtering algorithm, the ground point filtering adopts a progressive morphological filtering algorithm, and the point cloud registration adopts an iterative closest point algorithm; S106, storing the pre-processed point cloud data in a standard format and establishing a spatial index structure for subsequent rapid access and processing; S107, using adaptive point cloud density control algorithm to dynamically adjust the laser radar scanning frequency and the flight speed of the drone ,satisfy: ; In the formula, and are the minimum and maximum point cloud densities set, is the scanning bandwidth; S108, realize multi-sensor data fusion, combine RTK-GPS, IMU and lidar data, use extended Kalman filter algorithm to optimize point cloud positioning accuracy, the state estimation equation is: ; In the formula, is the optimal state estimate, is the Kalman gain, is the observation vector, is the observation matrix, It's time A priori state estimate of .

[0028] S2. Performing angle transformation processing on the dense point cloud at a top-view angle to obtain a two-dimensional image of the dense point cloud at a top-view angle; wherein the top-view angle is vertically downward to maximize the separability between plants; specifically as follows: S201. Based on the principal component analysis method, determine the main plane of the dense point cloud and calculate the covariance matrix C of the point cloud: ; In the formula, It is the first Points, is the centroid of the point cloud, N is the number of point clouds; S202, using the covariance matrix C to calculate the normal vector of the main plane: Perform eigenvalue decomposition on the covariance matrix C; Select the eigenvector corresponding to the smallest eigenvalue as the normal vector of the principal plane ; The normal vector As the initial looking down direction; If the normal vector If the angle with the vertical direction exceeds the preset threshold, the following optimization is performed: Adopt iterative plane fitting algorithm; set the maximum number of iterations and convergence threshold; iterative process: filter point set, recalculate covariance matrix and update normal vector; stop when convergence condition is met or maximum number of iterations is reached; finally optimize normal vector as determined top view direction; S203: According to the determined overlooking direction, coordinate transformation is performed on the dense point cloud, as follows: Calculate the rotation matrix , rotate the normal vector to Axis parallel; For point cloud Apply the rotation matrix Among them, is the normal vector, is the Z-axis unit vector, and the rotation matrix is ​​calculated using the quaternion or Euler angle method , so that ; Calculate the bounding box of the transformed point cloud and determine the new coordinate origin; Translate the point cloud so that the center of its bounding box coincides with the new coordinate origin; S204, projecting the point cloud after coordinate transformation onto a horizontal plane to generate a two-dimensional grid image, wherein the value of each grid is determined by the number of points falling into the grid or the height statistics, as follows: Determine the grid size and set it to 1-2 times the average spacing of the point cloud; Create a blank raster image, sized to cover the entire point cloud projection area; For each point, calculate its position in the raster image; Update grid values. The statistics you can choose include maximum height, average height or point density. Normalize the generated raster image and map the pixel values ​​to the range of 0-255; S205, post-processing the generated two-dimensional raster image to enhance image quality and analyzability, as follows: Apply Gaussian filtering to remove noise; Use adaptive histogram equalization to enhance contrast; Morphological operations are used to remove small isolated areas; S206, saving the processed two-dimensional raster image in a common image format, and recording the geographic reference information of the image so as to maintain consistency with the original point cloud data; S207, introduce adaptive multi-scale point cloud downsampling algorithm, according to the local point cloud density Dynamically resize voxels ; In the formula, It is an adjustable parameter used to balance downsampling efficiency and detail preservation; S208. Apply principal component analysis to enhance the structural features of the point cloud.

[0029] S3, calculating the homography matrix from the dense point cloud to the two-dimensional image, and establishing an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image, as follows: S301, selecting at least 4 pairs of corresponding points in the 3D point cloud and the 2D image to form a point pair set, wherein the perspective projection transformation from 3D to 2D has 8 degrees of freedom, and each pair of corresponding points provides two constraint equations, therefore, at least 4 pairs of corresponding points are required to uniquely determine the transformation; Select points with significant features in the 3D point cloud; Find the pixel positions corresponding to the points selected in the above three-dimensional point cloud in the two-dimensional image to form point pairs; Make sure the selected point pairs are evenly distributed and cover the entire area of ​​interest; S302, using a direct linear transformation algorithm to calculate the initial homography matrix H, as follows: Construct a system of linear equations, providing two equations for each pair of corresponding points; Use singular value decomposition to solve the linear equations and obtain the initial homography matrix H; S303, using a random sample consistency algorithm to optimize the initial homography matrix H to improve mapping accuracy, as follows: S3031. Randomly select a point pair subset with the minimum number of points, which belongs to the point pair set; S3032, calculating a candidate homography matrix using the point pairs in the selected point pair subset; S3033, calculating the projection errors of all point pairs under the candidate homography matrix; S3034, counting the number of inliers whose errors are within a preset threshold; S3035, repeat the above steps S3031-S3034 multiple times, and select the model with the most inliers; S3036, recalculate the final homography matrix using all the inliers in the model with the most inliers ; S304, using the Levenberg-Marquardt algorithm Nonlinear optimization is performed to further improve the accuracy of the homography matrix, as follows: Define the objective function as the sum of squared reprojection errors of a set of point pairs; Based on this objective function, iterative optimization is performed with the matrix optimized by RANSAC as the initial value; In each iteration, the damping factor is adjusted to balance convergence speed and stability; When the error change is less than the preset threshold or the maximum number of iterations is reached, the optimization is stopped to obtain the homography matrix with the best accuracy. ; S305, evaluate the final optimized homography matrix The accuracy is as follows: Compute the average reprojection error for a set of point pairs; Calculate the spatial distribution consistency of the mapped points; If the average reprojection error and spatial distribution consistency accuracy do not meet the requirements, consider increasing the number of corresponding points or reselecting corresponding points; S306, introduce deep learning-assisted feature matching algorithm, use convolutional neural network to extract feature descriptors of point cloud and image, network loss function Defined as: ; In the formula, is the descriptor loss, is the detector loss, is the balance factor; S307, using adaptive RANSAC algorithm to dynamically adjust the number of iterations : ; In the formula, is the expected probability of success, is the interior point ratio, is the minimum sample set size.

[0030] S4. For the two-dimensional image, a region marking method is used to segment and mark the rice plants, so as to achieve accurate segmentation and individual marking of the rice plants on the two-dimensional plane, as follows: S401, preprocessing the two-dimensional image, including denoising and contrast enhancement; S402, applying a watershed algorithm to perform preliminary segmentation on the preprocessed two-dimensional image; S403, optimizing the segmentation result using morphological operations, including opening operations and closing operations; S404, using connected region analysis to assign a unique label number to each segmented region; S405, introduce a multi-scale superpixel segmentation algorithm, combine color and spatial information, and define a superpixel similarity metric : ; In the formula, and are color distance and spatial distance respectively, and is the corresponding normalization factor; S406, applying the graph cut algorithm to optimize the segmentation results and minimize the energy function : ; In the formula, Assign labels, and Represents a pixel in an image; Is a data item, indicating that the label Assign to pixel the cost; is a smoothing term, representing adjacent pixels and Penalty for being assigned a different label; Represents a set of adjacent pixel pairs.

[0031] S5. Using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, assign a corresponding region marking number to each point in the point cloud, thereby obtaining a rice plant segmentation result of the entire dense point cloud; specifically, as follows: S501, using homography matrix Each point in the 3D point cloud Projected onto the two-dimensional image plane, it is represented as follows: ; in, , , Represents the 3D coordinates of each point in the 3D point cloud, , represents the horizontal and vertical coordinates of pixels in a two-dimensional image, is the final projection coordinate, is the corresponding scaling factor; S502, determining the segmentation area to which the projection point belongs according to the position of the projection point; S503, assigning the marking numbers of the corresponding segmented regions to the points in the three-dimensional point cloud; S504: For the points whose projection positions fall on the boundary of the segmented area, the nearest neighbor rule is used to determine the belonging, as follows: Identify the points whose projection positions fall on the boundary of the segmented regions; for each such point, calculate its distance to the adjacent segmented regions; assign the point to the segmented region with the closest distance; if there are multiple segmented regions with equal distances, select the region with the largest area to determine its affiliation.

[0032] Example 2 This embodiment discloses a rice plant segmentation system based on perspective transformation and dense point cloud, which is used to implement the rice plant segmentation method based on perspective transformation and dense point cloud described in Example 1. Figure 4 As shown, it includes: A point cloud acquisition unit is used to scan the rice field using a UAV-mounted laser radar system to obtain a dense point cloud of the rice canopy; A perspective transformation unit, configured to perform angle transformation processing on the dense point cloud at a top-view angle to obtain a two-dimensional image of the dense point cloud at a top-view angle; A mapping establishment unit, used to calculate the homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; A two-dimensional segmentation unit, used for segmenting and marking rice plants in the two-dimensional image using a region marking method; A three-dimensional marking unit, used to assign a corresponding region marking number to each point in the point cloud by using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, so as to obtain a segmentation result of the entire dense point cloud; Adaptive optimization unit, used to realize point cloud density control, multi-sensor data fusion, multi-scale feature extraction and temporal consistency constraint functions.

[0033] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A rice plant segmentation method based on perspective transformation and dense point cloud, characterized in that: include: S1. Use the UAV-mounted LiDAR system to scan the rice field and obtain a dense point cloud of the rice canopy. S2, performing angle transformation processing on the dense point cloud at a top-view angle to obtain a two-dimensional image of the dense point cloud at a top-view angle; wherein the top-view angle is vertically downward to maximize the separability between plants; S3, calculating the homography matrix from the dense point cloud to the two-dimensional image, and establishing an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; S4. For the two-dimensional image, a region marking method is used to segment and mark the rice plants, so as to achieve accurate segmentation and individual marking of the rice plants on the two-dimensional plane; S5. Using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, a corresponding region marking number is assigned to each point in the point cloud, thereby obtaining a rice plant segmentation result of the entire dense point cloud.

2. The rice plant segmentation method based on perspective transformation and dense point cloud according to claim 1, characterized in that: The step S1 comprises: S101, on the UAV ground station, the rice field is set as the scanning area, and the heading overlap rate and the lateral overlap rate of the scanning are set, and the UAV ground station automatically generates a route according to the operation information; wherein the heading overlap rate is set to 60%-80%, and the lateral overlap rate is set to 50%-70%, so as to ensure that the required dense point cloud data is obtained; S102, the UAV autonomously flies according to the planned route, automatically performs laser radar scanning according to the set overlap rate, and records the position and attitude information of the scan data; wherein the position and attitude information includes longitude, latitude, altitude, heading angle, pitch angle and roll angle; the flight altitude of the UAV is set to 20-50 meters, and the flight speed is 3-8 meters / second to balance data collection efficiency and point cloud density; S103, the drone uploads the collected dense point cloud data to the server in real time based on the data link; wherein the real-time uploaded data includes both the point cloud coordinates and the reflection intensity information of each point; the drone and the server establish a secure data transmission channel based on a preset communication protocol; the data link adopts a 4G / 5G network or a dedicated wireless communication module, and the transmission rate is not less than 10Mbps; S104, after receiving the point cloud data, the server performs a real-time data quality check, including evaluation of point cloud density, coverage, and data integrity; if it is found that the data quality does not meet the requirements, a rescanning instruction is automatically sent to the drone; S105, preprocessing the received original point cloud data, including noise removal, ground point filtering and point cloud registration; wherein, the noise removal adopts a statistical outlier filtering algorithm, the ground point filtering adopts a progressive morphological filtering algorithm, and the point cloud registration adopts an iterative closest point algorithm; S106, storing the pre-processed point cloud data in a standard format and establishing a spatial index structure for subsequent rapid access and processing; S107, using adaptive point cloud density control algorithm to dynamically adjust the laser radar scanning frequency and the flight speed of the drone ,satisfy: ； In the formula, and are the minimum and maximum point cloud densities set, is the scanning bandwidth; S108, realize multi-sensor data fusion, combine RTK-GPS, IMU and lidar data, use extended Kalman filter algorithm to optimize point cloud positioning accuracy, the state estimation equation is: ； In the formula, is the optimal state estimate, is the Kalman gain, is the observation vector, is the observation matrix, It's time A priori state estimate of .

3. The rice plant segmentation method based on perspective transformation and dense point cloud according to claim 2, characterized in that: The step S2 comprises: S201. Based on the principal component analysis method, determine the main plane of the dense point cloud and calculate the covariance matrix C of the point cloud: ； In the formula, It is the first Points, is the centroid of the point cloud, N is the number of point clouds; S202, using the covariance matrix C to calculate the normal vector of the main plane: Perform eigenvalue decomposition on the covariance matrix C; Select the eigenvector corresponding to the smallest eigenvalue as the normal vector of the principal plane ; The normal vector As the initial looking down direction; If the normal vector If the angle with the vertical direction exceeds the preset threshold, the following optimization is performed: Adopt iterative plane fitting algorithm; set the maximum number of iterations and convergence threshold; iterative process: filter point set, recalculate covariance matrix and update normal vector; stop when convergence condition is met or maximum number of iterations is reached; finally optimize normal vector as determined top view direction; S203: According to the determined overlooking direction, coordinate transformation is performed on the dense point cloud, as follows: Calculate the rotation matrix , rotate the normal vector to Axis parallel; For point cloud Apply the rotation matrix Among them, is the normal vector, is the Z-axis unit vector, and the rotation matrix is ​​calculated using the quaternion or Euler angle method , so that ; Calculate the bounding box of the transformed point cloud and determine the new coordinate origin; Translate the point cloud so that the center of its bounding box coincides with the new coordinate origin; S204, projecting the point cloud after coordinate transformation onto a horizontal plane to generate a two-dimensional grid image, wherein the value of each grid is determined by the number of points falling into the grid or the height statistics, as follows: Determine the grid size and set it to 1-2 times the average spacing of the point cloud; Create a blank raster image, sized to cover the entire point cloud projection area; For each point, calculate its position in the raster image; Update grid values. The statistics you can choose include maximum height, average height or point density. Normalize the generated raster image and map the pixel values ​​to the range of 0-255; S205, post-processing the generated two-dimensional raster image to enhance image quality and analyzability, as follows: Apply Gaussian filtering to remove noise; Use adaptive histogram equalization to enhance contrast; Morphological operations are used to remove small isolated areas; S206, saving the processed two-dimensional raster image in a common image format, and recording the geographic reference information of the image so as to maintain consistency with the original point cloud data; S207, introduce adaptive multi-scale point cloud downsampling algorithm, according to the local point cloud density Dynamically resize voxels ; In the formula, It is an adjustable parameter used to balance downsampling efficiency and detail preservation; S208. Apply principal component analysis to enhance the structural features of the point cloud.

4. The rice plant segmentation method based on perspective transformation and dense point cloud according to claim 3, characterized in that: The step S3 comprises: S301, selecting at least 4 pairs of corresponding points in the 3D point cloud and the 2D image to form a point pair set, wherein the perspective projection transformation from 3D to 2D has 8 degrees of freedom, and each pair of corresponding points provides two constraint equations, therefore, at least 4 pairs of corresponding points are required to uniquely determine the transformation; Select points with significant features in the 3D point cloud; Find the pixel positions corresponding to the points selected in the above three-dimensional point cloud in the two-dimensional image to form point pairs; Make sure the selected point pairs are evenly distributed and cover the entire area of ​​interest; S302, using a direct linear transformation algorithm to calculate the initial homography matrix H, as follows: Construct a system of linear equations, providing two equations for each pair of corresponding points; Use singular value decomposition to solve the linear equations and obtain the initial homography matrix H; S303, using a random sample consistency algorithm to optimize the initial homography matrix H to improve mapping accuracy, as follows: S3031. Randomly select a point pair subset with the minimum number of points, which belongs to the point pair set; S3032, calculating a candidate homography matrix using the point pairs in the selected point pair subset; S3033, calculating the projection errors of all point pairs under the candidate homography matrix; S3034, counting the number of inliers whose errors are within a preset threshold; S3035, repeat the above steps S3031-S3034 multiple times to select the model with the most inliers; S3036, recalculate the final homography matrix using all the inliers in the model with the most inliers ; S304, using the Levenberg-Marquardt algorithm Nonlinear optimization is performed to further improve the accuracy of the homography matrix, as follows: Define the objective function as the sum of squared reprojection errors of a set of point pairs; Based on this objective function, iterative optimization is performed with the matrix optimized by RANSAC as the initial value; In each iteration, the damping factor is adjusted to balance convergence speed and stability; When the error change is less than the preset threshold or the maximum number of iterations is reached, the optimization is stopped to obtain the homography matrix with the best accuracy. ; S305, evaluate the final optimized homography matrix The accuracy is as follows: Compute the average reprojection error for a set of point pairs; Calculate the spatial distribution consistency of the mapped points; If the average reprojection error and spatial distribution consistency accuracy do not meet the requirements, consider increasing the number of corresponding points or reselecting corresponding points; S306, introduce deep learning-assisted feature matching algorithm, use convolutional neural network to extract feature descriptors of point cloud and image, network loss function Defined as: ； In the formula, is the descriptor loss, is the detector loss, is the balance factor; S307, using adaptive RANSAC algorithm to dynamically adjust the number of iterations : ； In the formula, is the expected probability of success, is the interior point ratio, is the minimum sample set size.

5. The rice plant segmentation method based on perspective transformation and dense point cloud according to claim 4, characterized in that: The step S4 comprises: S401, preprocessing the two-dimensional image, including denoising and contrast enhancement; S402, applying a watershed algorithm to perform preliminary segmentation on the preprocessed two-dimensional image; S403, optimizing the segmentation result using morphological operations, including opening operations and closing operations; S404, using connected region analysis to assign a unique label number to each segmented region; S405, introduce a multi-scale superpixel segmentation algorithm, combine color and spatial information, and define a superpixel similarity metric : ； In the formula, and are color distance and spatial distance respectively, and is the corresponding normalization factor; S406, applying the graph cut algorithm to optimize the segmentation results and minimize the energy function : ； In the formula, Assign labels, and Represents a pixel in an image; Is a data item, indicating that the label Assign to pixel the cost; is a smoothing term, representing adjacent pixels and Penalty for being assigned a different label; Represents a set of adjacent pixel pairs.

6. The rice plant segmentation method based on perspective transformation and dense point cloud according to claim 1, characterized in that: The step S5 comprises: S501, using homography matrix Each point in the 3D point cloud Projected onto the two-dimensional image plane, it is represented as follows: ； in, , , Represents the 3D coordinates of each point in the 3D point cloud, , represents the horizontal and vertical coordinates of pixels in a two-dimensional image, is the final projection coordinate, is the corresponding scaling factor; S502, determining the segmentation area to which the projection point belongs according to the position of the projection point; S503, assigning the marking numbers of the corresponding segmented regions to the points in the three-dimensional point cloud; S504: For the points whose projection positions fall on the boundary of the segmented area, the nearest neighbor rule is used to determine the belonging, as follows: Identify the points whose projection positions fall on the boundary of the segmented regions; for each such point, calculate its distance to the adjacent segmented regions; assign the point to the segmented region with the closest distance; if there are multiple segmented regions with equal distances, select the region with the largest area to determine its affiliation.

7. A rice plant segmentation system based on perspective transformation and dense point cloud, characterized in that: The rice plant segmentation method based on perspective transformation and dense point cloud for implementing any one of claims 1 to 6 comprises: A point cloud acquisition unit is used to scan the rice field using a UAV-mounted laser radar system to obtain a dense point cloud of the rice canopy; A perspective transformation unit, configured to perform angle transformation processing on the dense point cloud at a top-view angle to obtain a two-dimensional image of the dense point cloud at a top-view angle; A mapping establishment unit, used to calculate the homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; A two-dimensional segmentation unit, used for segmenting and marking rice plants in the two-dimensional image using a region marking method; A three-dimensional marking unit, used to assign a corresponding region marking number to each point in the point cloud by using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, so as to obtain a segmentation result of the entire dense point cloud; Adaptive optimization unit, used to realize point cloud density control, multi-sensor data fusion, multi-scale feature extraction and temporal consistency constraint functions.

Images