Rice plant segmentation method and system based on viewpoint transformation and dense point cloud

Abstract

The present invention discloses a rice plant segmentation method and system based on perspective transformation and dense point cloud. The method comprises: using an unmanned aerial vehicle (UAV) laser radar system to scan a rice field to obtain a dense point cloud of the rice canopy; performing angle transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at a top-down perspective; calculating a homography matrix from the dense point cloud to the two-dimensional image to establish a precise mapping relationship between the three-dimensional point cloud and the two-dimensional image; using a region labeling method to segment and label rice plants in the two-dimensional image to achieve precise segmentation and individual labeling of rice plants on a two-dimensional plane; and using the mapping relationship between the three-dimensional point cloud and the two-dimensional image to assign a corresponding region label number to each point in the point cloud, thereby obtaining a rice plant segmentation result for the entire dense point cloud. The present invention achieves high-precision rice plant segmentation, providing more reliable and efficient technical support for precision agricultural management and phenotyping of rice.

Description

Technical Field

[0001] The present invention relates to the technical field of crop phenotyping and unmanned aerial vehicle (UAV) remote sensing, and in particular to a rice plant segmentation method and system based on perspective transformation and dense point cloud. Background Art

[0002] With the rapid development of precision agriculture, monitoring rice phenotypic traits using drone remote sensing technology has become an important agricultural management tool. In recent years, drones equipped with lidar systems have acquired high-density three-dimensional point cloud data, providing a new technical approach and a richer data source for high-throughput rice phenotyping.

[0003] Accurately identifying and segmenting individual rice plants is a key step in rice phenotyping, crucial for subsequent growth parameter extraction and agricultural analysis. However, due to the morphological changes of rice plants during growth and the complex environmental factors in the field, accurately segmenting individual rice plants from 3D point cloud data alone remains a significant challenge.

[0004] Current mainstream plant point cloud segmentation methods are primarily based on spatial clustering algorithms. These methods work well for sparsely distributed plants, but often struggle to accurately distinguish adjacent plants in densely planted crops like rice. Furthermore, these methods often require extensive manual parameter adjustments, making them difficult to adapt to rice plants of varying growth stages and varieties.

[0005] In the existing technology, the processing and analysis of 3D point cloud data mainly have the following deficiencies:

[0006] 1. Lack of effective perspective transformation methods: 3D point cloud data usually needs to be converted into 2D images from a bird's-eye view for analysis, but existing methods often find it difficult to retain sufficient spatial information.

[0007] 2. Insufficient data utilization: Most methods only use the spatial information of point clouds and fail to fully utilize the two-dimensional image information generated by point clouds.

[0008] 3. Poor adaptability of segmentation algorithms: Existing methods are often difficult to adapt to the changes in morphological characteristics of rice at different growth stages. Summary of the Invention

[0009] The first purpose of the present invention is to solve the problems of low accuracy and poor efficiency of traditional rice plant segmentation technology when dealing with high-density planting and serious plant overlap. It provides a rice plant segmentation method based on perspective transformation and dense point cloud, which makes full use of the high-density point cloud data collected by the drone-mounted lidar system. Through innovative perspective transformation technology, the three-dimensional point cloud data is converted into a two-dimensional image from a top-down angle. At the same time, the homography matrix is used to establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image, and combined with the region labeling method, high-precision rice plant segmentation is achieved, providing more reliable and efficient technical support for precision agricultural management and phenotypic analysis of rice.

[0010] A second object of the present invention is to provide a rice plant segmentation system based on viewpoint transformation and dense point cloud.

[0011] The first objective of the present invention is achieved by the following technical solution: a rice plant segmentation method based on perspective transformation and dense point cloud, comprising:

[0012] S1. Use a UAV-mounted LiDAR system to scan the rice field and obtain a dense point cloud of the rice canopy.

[0013] S2. performing angle transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at the top-down perspective; wherein the top-down perspective is vertically downward to maximize the separability between plants;

[0014] S3, calculating the homography matrix from the dense point cloud to the two-dimensional image, and establishing an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image;

[0015] S4. For the two-dimensional image, segment and mark the rice plants using a region marking method to achieve accurate segmentation and individual marking of the rice plants on the two-dimensional plane;

[0016] S5. Using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, assign a corresponding region label number to each point in the point cloud, thereby obtaining a rice plant segmentation result of the entire dense point cloud.

[0017] Furthermore, the step S1 includes:

[0018] S101. On the UAV ground station, the rice field is set as the scanning area, and the heading overlap rate and lateral overlap rate of the scan are set. The UAV ground station automatically generates a route based on the operation information; wherein the heading overlap rate is set to 60%-80%, and the lateral overlap rate is set to 50%-70% to ensure that the required dense point cloud data is obtained;

[0019] S102: The UAV autonomously flies according to the planned route, automatically performs lidar scanning according to a set overlap rate, and records the position and attitude information of the scan data; wherein the position and attitude information includes longitude, latitude, altitude, heading angle, pitch angle, and roll angle; the UAV's flight altitude is set to 20-50 meters and the flight speed is set to 3-8 meters per second to balance data collection efficiency and point cloud density;

[0020] S103. The drone uploads the collected dense point cloud data to the server in real time based on a data link. The uploaded data includes both the point cloud coordinates and the reflection intensity information of each point. The drone and the server establish a secure data transmission channel based on a preset communication protocol. The data link utilizes a 4G / 5G network or a dedicated wireless communication module with a transmission rate of no less than 10 Mbps.

[0021] S104. After receiving the point cloud data, the server performs a real-time data quality check, including an assessment of point cloud density, coverage, and data integrity. If the data quality does not meet the requirements, a rescan command is automatically sent to the drone.

[0022] S105, preprocessing the received raw point cloud data, including noise removal, ground point filtering, and point cloud registration; wherein, noise removal uses a statistical outlier filtering algorithm, ground point filtering uses a progressive morphological filtering algorithm, and point cloud registration uses an iterative closest point algorithm;

[0023] S106, storing the pre-processed point cloud data in a standard format and establishing a spatial index structure for subsequent rapid access and processing;

[0024] S107, using adaptive point cloud density control algorithm to dynamically adjust the laser radar scanning frequency and drone flight speed ,satisfy:

[0025] ;

[0026] Where, and are the minimum and maximum point cloud densities set, is the scanning bandwidth;

[0027] S108, realize multi-sensor data fusion, combine RTK-GPS, IMU and lidar data, and use extended Kalman filter algorithm to optimize point cloud positioning accuracy. The state estimation equation is:

[0028] ;

[0029] Where, is the optimal state estimate, is the Kalman gain, is the observation vector, is the observation matrix, It's time A priori state estimate of .

[0030] Further, the step S2 includes:

[0031] S201. Based on the principal component analysis method, determine the main plane of the dense point cloud and calculate the covariance matrix C of the point cloud:

[0032] ;

[0033] Where, The first Points, is the centroid of the point cloud, N is the number of point clouds;

[0034] S202. Calculate the normal vector of the main plane using the covariance matrix C:

[0035] Perform eigenvalue decomposition on the covariance matrix C;

[0036] Select the eigenvector corresponding to the smallest eigenvalue as the normal vector of the main plane ;

[0037] The normal vector As the initial looking down direction;

[0038] If the normal vector If the angle with the vertical direction exceeds the preset threshold, the following optimization is performed:

[0039] An iterative plane fitting algorithm is used; the maximum number of iterations and the convergence threshold are set; the iterative process includes filtering the point set, recalculating the covariance matrix, and updating the normal vector; the algorithm stops when the convergence condition is met or the maximum number of iterations is reached; the final optimized normal vector is used as the determined top-down direction;

[0040] S203: Perform coordinate transformation on the dense point cloud according to the determined overlooking direction, as follows:

[0041] Calculate the rotation matrix , rotate the normal vector to axis parallel;

[0042] Point cloud Apply the rotation matrix Among them, is the normal vector, is the Z-axis unit vector, and the rotation matrix is calculated using the quaternion or Euler angle method. , making ;

[0043] Calculate the bounding box of the transformed point cloud and determine the new coordinate origin;

[0044] Translate the point cloud so that the center of its bounding box coincides with the new coordinate origin;

[0045] S204: Project the point cloud after coordinate transformation onto a horizontal plane to generate a two-dimensional grid image, where the value of each grid is determined by the number of points falling into the grid or the height statistics, as follows:

[0046] Determine the grid size and set it to 1-2 times the average spacing of the point cloud;

[0047] Create a blank raster image with a size covering the entire point cloud projection area;

[0048] For each point, calculate its position in the raster image;

[0049] Update grid values. The statistics you can choose include maximum height, average height or point density.

[0050] Normalize the generated raster image and map the pixel values to the range of 0-255;

[0051] S205: Post-process the generated two-dimensional raster image to enhance image quality and analyzability, as follows:

[0052] Apply Gaussian filtering to remove noise;

[0053] Use adaptive histogram equalization to enhance contrast;

[0054] Morphological operations are used to remove small isolated areas;

[0055] S206, saving the processed two-dimensional raster image into a common image format and recording the geographic reference information of the image so as to maintain consistency with the original point cloud data;

[0056] S207, introduce adaptive multi-scale point cloud downsampling algorithm, according to the local point cloud density Dynamic voxel resizing ;

[0057] Where, It is an adjustable parameter used to balance downsampling efficiency and detail preservation;

[0058] S208. Apply principal component analysis to enhance the structural features of the point cloud.

[0059] Further, the step S3 includes:

[0060] S301. Select at least four pairs of corresponding points in the three-dimensional point cloud and the two-dimensional image to form a point pair set. The three-dimensional to two-dimensional perspective projection transformation has eight degrees of freedom, and each pair of corresponding points provides two constraint equations. Therefore, at least four pairs of corresponding points are required to uniquely determine the transformation.

[0061] Select points with significant features in the 3D point cloud;

[0062] Find the pixel positions in the 2D image corresponding to the points selected in the 3D point cloud to form point pairs;

[0063] Make sure the selected point pairs are evenly distributed and cover the entire area of interest;

[0064] S302: Calculate the initial homography matrix H using a direct linear transformation algorithm, as follows:

[0065] Construct a system of linear equations, providing two equations for each pair of corresponding points;

[0066] Use singular value decomposition to solve the linear equations and obtain the initial homography matrix H;

[0067] S303: Use the random sample consistency algorithm to optimize the initial homography matrix H to improve the mapping accuracy, as follows:

[0068] S3031. Randomly select a minimum number of point pair subsets, which belong to the above point pair set;

[0069] S3032. Calculate a candidate homography matrix using the point pairs in the selected point pair subset;

[0070] S3033. Calculate the projection errors of all point pairs under the candidate homography matrix;

[0071] S3034. Counting the number of inliers whose errors are within a preset threshold;

[0072] S3035. Repeat the above steps S3031-S3034 multiple times to select the model with the most inliers;

[0073] S3036, recalculate the final homography matrix using all the inliers in the model with the most inliers ;

[0074] S304, using the Levenberg-Marquardt algorithm Perform nonlinear optimization to further improve the accuracy of the homography matrix as follows:

[0075] Define the objective function as the sum of squares of the reprojection errors of the point pair set;

[0076] Based on this objective function, iterative optimization is performed with the matrix optimized by RANSAC as the initial value;

[0077] In each iteration, the damping factor is adjusted to balance convergence speed and stability;

[0078] When the error change is less than the preset threshold or the maximum number of iterations is reached, the optimization is stopped and the homography matrix with the best accuracy is obtained. ;

[0079] S305, evaluate the final optimized homography matrix The accuracy is as follows:

[0080] Calculate the average reprojection error for a set of point pairs;

[0081] Calculate the spatial distribution consistency of the mapped points;

[0082] If the average reprojection error and spatial distribution consistency accuracy do not meet the requirements, consider increasing the number of corresponding points or reselecting corresponding points;

[0083] S306, introduce deep learning assisted feature matching algorithm, use convolutional neural network to extract feature descriptors of point cloud and image, network loss function Defined as:

[0084] ;

[0085] Where, is the descriptor loss, is the detector loss, is the balance factor;

[0086] S307, using adaptive RANSAC algorithm to dynamically adjust the number of iterations :

[0087] ;

[0088] Where, is the expected probability of success, is the interior point ratio, is the minimum sample set size.

[0089] Further, the step S4 includes:

[0090] S401, preprocessing the two-dimensional image, including denoising and contrast enhancement;

[0091] S402, applying a watershed algorithm to perform preliminary segmentation on the pre-processed two-dimensional image;

[0092] S403, optimizing the segmentation result using morphological operations, including opening and closing operations;

[0093] S404, using connected component analysis to assign a unique label number to each segmented region;

[0094] S405. Introduce a multi-scale superpixel segmentation algorithm, combine color and spatial information, and define a superpixel similarity metric. :

[0095] ;

[0096] Where, and are color distance and spatial distance respectively, and is the corresponding normalization factor;

[0097] S406, apply the graph cut algorithm to optimize the segmentation results and minimize the energy function :

[0098] ;

[0099] Where, Assign labels, and Represents a pixel in an image; Is a data item, indicating that the label Assign to pixels the price; is a smoothing term, representing adjacent pixels and Penalties for being assigned a different label; Represents a set of adjacent pixel pairs.

[0100] Further, the step S5 includes:

[0101] S501, using the homography matrix Each point in the 3D point cloud Projected onto the two-dimensional image plane, it is represented as follows:

[0102] ;

[0103] in, 、 、 Represents the three-dimensional coordinates of each point in the three-dimensional point cloud, 、 represents the horizontal and vertical coordinates of the pixels in the two-dimensional image, is the final projection coordinate, is the corresponding scaling factor;

[0104] S502: Determine the segmentation area to which the projection point belongs based on its position;

[0105] S503, assigning label numbers of corresponding segmented regions to points in the three-dimensional point cloud;

[0106] S504: For points whose projected positions fall on the boundary of the segmented area, the nearest neighbor rule is used to determine the belonging of the points, as follows:

[0107] Identify the points whose projection positions fall on the boundary of the segmented regions; for each such point, calculate its distance to the adjacent segmented regions; assign the point to the segmented region with the closest distance; if there are multiple segmented regions with equal distances, choose the region with the largest area to determine its affiliation.

[0108] The second objective of the present invention is achieved through the following technical solution: a rice plant segmentation system based on viewpoint transformation and dense point cloud, for implementing the above-mentioned rice plant segmentation method based on viewpoint transformation and dense point cloud, comprising:

[0109] A point cloud acquisition unit is used to scan rice fields using a UAV-mounted lidar system to obtain a dense point cloud of the rice canopy;

[0110] a perspective transformation unit, configured to perform perspective transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at a top-down perspective;

[0111] A mapping establishment unit, configured to calculate a homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image;

[0112] a two-dimensional segmentation unit, configured to segment and mark rice plants in the two-dimensional image using a region marking method;

[0113] a three-dimensional marking unit, configured to assign a corresponding region marking number to each point in the point cloud by using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, thereby obtaining a segmentation result of the entire dense point cloud;

[0114] Adaptive optimization unit, used to realize point cloud density control, multi-sensor data fusion, multi-scale feature extraction and temporal consistency constraint functions.

[0115] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0116] 1. Innovative method of perspective transformation and dense point cloud fusion: This invention uniquely converts 3D dense point cloud data into a 2D image from a top-down perspective and establishes a precise 3D-to-2D mapping relationship. This method cleverly combines the rich information of 3D data with the high efficiency of 2D image processing, providing a new approach to rice plant segmentation and effectively solving the segmentation problem in cases of high-density planting and severe plant overlap.

[0117] 2. Optimal top-down angle selection: The present invention uses a vertically downward top-down angle to perform angle transformation processing on dense point clouds. This method can maximize the separability between plants. By selecting the optimal top-down angle, the accuracy and efficiency of the subsequent segmentation process are significantly improved. It is particularly suitable for densely planted rice field environments.

[0118] 3. Accurate 3D to 2D mapping technology: This invention proposes an innovative homography matrix calculation method to establish an accurate mapping relationship between 3D point clouds and 2D images. This high-precision mapping technology not only improves the conversion accuracy of 3D point clouds to 2D images, but also provides a reliable foundation for subsequent plant segmentation and labeling.

[0119] 4. Efficient two-dimensional image segmentation method: The present invention uses a region labeling method to segment and label rice plants in a two-dimensional image, achieving accurate segmentation and individual labeling of rice plants on a two-dimensional plane. This method fully utilizes the high efficiency of two-dimensional image processing while ensuring the accuracy of the segmentation results.

[0120] 5. 3D point cloud precise labeling technology: This invention uniquely proposes a precise labeling method from 2D segmentation results to 3D point clouds. By utilizing the established mapping relationship between 3D point clouds and 2D images, each point in the point cloud is assigned a corresponding region label number, achieving high-precision segmentation and labeling of the entire dense point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] Figure 1 Schematic diagram of the process of the present invention.

[0122] Figure 2 Schematic diagram of dense point cloud processing flow.

[0123] Figure 3 Schematic diagram of the rice plant segmentation process.

[0124] Figure 4 Schematic diagram of the composition of the system of the present invention. DETAILED DESCRIPTION

[0125] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0126] Example 1

[0127] like Figures 1 to 3 As shown, this embodiment discloses a rice plant segmentation method based on perspective transformation and dense point cloud, the details of which are as follows:

[0128] S1. Use a UAV-mounted LiDAR system to scan the rice field and obtain a dense point cloud of the rice canopy, as follows:

[0129] S101. On the UAV ground station, the rice field is set as the scanning area, and the heading overlap rate and lateral overlap rate of the scan are set. The UAV ground station automatically generates a route based on the operation information; wherein the heading overlap rate is set to 60%-80%, and the lateral overlap rate is set to 50%-70% to ensure that the required dense point cloud data is obtained;

[0130] S102: The UAV autonomously flies according to the planned route, automatically performs lidar scanning according to a set overlap rate, and records the position and attitude information of the scan data; wherein the position and attitude information includes longitude, latitude, altitude, heading angle, pitch angle, and roll angle; the UAV's flight altitude is set to 20-50 meters and the flight speed is set to 3-8 meters per second to balance data collection efficiency and point cloud density;

[0131] S103. The drone uploads the collected dense point cloud data to the server in real time based on a data link. The uploaded data includes both the point cloud coordinates and the reflection intensity information of each point. The drone and the server establish a secure data transmission channel based on a preset communication protocol. The data link utilizes a 4G / 5G network or a dedicated wireless communication module with a transmission rate of no less than 10 Mbps.

[0132] S104. After receiving the point cloud data, the server performs a real-time data quality check, including an assessment of point cloud density, coverage, and data integrity. If the data quality does not meet the requirements, a rescan command is automatically sent to the drone.

[0133] S105, preprocessing the received raw point cloud data, including noise removal, ground point filtering, and point cloud registration; wherein, noise removal uses a statistical outlier filtering algorithm, ground point filtering uses a progressive morphological filtering algorithm, and point cloud registration uses an iterative closest point algorithm;

[0134] S106, storing the pre-processed point cloud data in a standard format and establishing a spatial index structure for subsequent rapid access and processing;

[0135] S107, using adaptive point cloud density control algorithm to dynamically adjust the laser radar scanning frequency and drone flight speed ,satisfy:

[0136] ;

[0137] Where, and are the minimum and maximum point cloud densities set, is the scanning bandwidth;

[0138] S108, realize multi-sensor data fusion, combine RTK-GPS, IMU and lidar data, and use extended Kalman filter algorithm to optimize point cloud positioning accuracy. The state estimation equation is:

[0139] ;

[0140] Where, is the optimal state estimate, is the Kalman gain, is the observation vector, is the observation matrix, It's time A priori state estimate of .

[0141] S2. Performing angle transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at the top-down perspective; wherein the top-down perspective is vertically downward to maximize the separability between plants; specifically, as follows:

[0142] S201. Based on the principal component analysis method, determine the main plane of the dense point cloud and calculate the covariance matrix C of the point cloud:

[0143] ;

[0144] Where, The first Points, is the centroid of the point cloud, N is the number of point clouds;

[0145] S202. Calculate the normal vector of the main plane using the covariance matrix C:

[0146] Perform eigenvalue decomposition on the covariance matrix C;

[0147] Select the eigenvector corresponding to the smallest eigenvalue as the normal vector of the main plane ;

[0148] The normal vector As the initial looking down direction;

[0149] If the normal vector If the angle with the vertical direction exceeds the preset threshold, the following optimization is performed:

[0150] An iterative plane fitting algorithm is used; the maximum number of iterations and the convergence threshold are set; the iterative process includes filtering the point set, recalculating the covariance matrix, and updating the normal vector; the algorithm stops when the convergence condition is met or the maximum number of iterations is reached; the final optimized normal vector is used as the determined top-down direction;

[0151] S203: Perform coordinate transformation on the dense point cloud according to the determined overlooking direction, as follows:

[0152] Calculate the rotation matrix , rotate the normal vector to axis parallel;

[0153] Point cloud Apply the rotation matrix Among them, is the normal vector, is the Z-axis unit vector, and the rotation matrix is calculated using the quaternion or Euler angle method. , making ;

[0154] Calculate the bounding box of the transformed point cloud and determine the new coordinate origin;

[0155] Translate the point cloud so that the center of its bounding box coincides with the new coordinate origin;

[0156] S204: Project the point cloud after coordinate transformation onto a horizontal plane to generate a two-dimensional grid image, where the value of each grid is determined by the number of points falling into the grid or the height statistics, as follows:

[0157] Determine the grid size and set it to 1-2 times the average spacing of the point cloud;

[0158] Create a blank raster image with a size covering the entire point cloud projection area;

[0159] For each point, calculate its position in the raster image;

[0160] Update grid values. The statistics you can choose include maximum height, average height or point density.

[0161] Normalize the generated raster image and map the pixel values to the range of 0-255;

[0162] S205: Post-process the generated two-dimensional raster image to enhance image quality and analyzability, as follows:

[0163] Apply Gaussian filtering to remove noise;

[0164] Use adaptive histogram equalization to enhance contrast;

[0165] Morphological operations are used to remove small isolated areas;

[0166] S206, saving the processed two-dimensional raster image into a common image format and recording the geographic reference information of the image so as to maintain consistency with the original point cloud data;

[0167] S207, introduce adaptive multi-scale point cloud downsampling algorithm, according to the local point cloud density Dynamic voxel resizing ;

[0168] Where, It is an adjustable parameter used to balance downsampling efficiency and detail preservation;

[0169] S208. Apply principal component analysis to enhance the structural features of the point cloud.

[0170] S3. Calculate the homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image, as follows:

[0171] S301. Select at least four pairs of corresponding points in the three-dimensional point cloud and the two-dimensional image to form a point pair set. The three-dimensional to two-dimensional perspective projection transformation has eight degrees of freedom, and each pair of corresponding points provides two constraint equations. Therefore, at least four pairs of corresponding points are required to uniquely determine the transformation.

[0172] Select points with significant features in the 3D point cloud;

[0173] Find the pixel positions in the 2D image corresponding to the points selected in the 3D point cloud to form point pairs;

[0174] Make sure the selected point pairs are evenly distributed and cover the entire area of interest;

[0175] S302: Calculate the initial homography matrix H using a direct linear transformation algorithm, as follows:

[0176] Construct a system of linear equations, providing two equations for each pair of corresponding points;

[0177] Use singular value decomposition to solve the linear equations and obtain the initial homography matrix H;

[0178] S303: Use the random sample consistency algorithm to optimize the initial homography matrix H to improve the mapping accuracy, as follows:

[0179] S3031. Randomly select a minimum number of point pair subsets, which belong to the above point pair set;

[0180] S3032. Calculate a candidate homography matrix using the point pairs in the selected point pair subset;

[0181] S3033. Calculate the projection errors of all point pairs under the candidate homography matrix;

[0182] S3034. Counting the number of inliers whose errors are within a preset threshold;

[0183] S3035. Repeat the above steps S3031-S3034 multiple times to select the model with the most inliers;

[0184] S3036, recalculate the final homography matrix using all the inliers in the model with the most inliers ;

[0185] S304, using the Levenberg-Marquardt algorithm Perform nonlinear optimization to further improve the accuracy of the homography matrix as follows:

[0186] Define the objective function as the sum of squares of the reprojection errors of the point pair set;

[0187] Based on this objective function, iterative optimization is performed with the matrix optimized by RANSAC as the initial value;

[0188] In each iteration, the damping factor is adjusted to balance convergence speed and stability;

[0189] When the error change is less than the preset threshold or the maximum number of iterations is reached, the optimization is stopped and the homography matrix with the best accuracy is obtained. ;

[0190] S305, evaluate the final optimized homography matrix The accuracy is as follows:

[0191] Calculate the average reprojection error for a set of point pairs;

[0192] Calculate the spatial distribution consistency of the mapped points;

[0193] If the average reprojection error and spatial distribution consistency accuracy do not meet the requirements, consider increasing the number of corresponding points or reselecting corresponding points;

[0194] S306, introduce deep learning assisted feature matching algorithm, use convolutional neural network to extract feature descriptors of point cloud and image, network loss function Defined as:

[0195] ;

[0196] Where, is the descriptor loss, is the detector loss, is the balance factor;

[0197] S307, using adaptive RANSAC algorithm to dynamically adjust the number of iterations :

[0198] ;

[0199] Where, is the expected probability of success, is the interior point ratio, is the minimum sample set size.

[0200] S4. For the two-dimensional image, segment and label the rice plants using a region labeling method to achieve accurate segmentation and individual labeling of the rice plants on the two-dimensional plane, as follows:

[0201] S401, preprocessing the two-dimensional image, including denoising and contrast enhancement;

[0202] S402, applying a watershed algorithm to perform preliminary segmentation on the pre-processed two-dimensional image;

[0203] S403, optimizing the segmentation result using morphological operations, including opening and closing operations;

[0204] S404, using connected component analysis to assign a unique label number to each segmented region;

[0205] S405. Introduce a multi-scale superpixel segmentation algorithm, combine color and spatial information, and define a superpixel similarity metric. :

[0206] ;

[0207] Where, and are color distance and spatial distance respectively, and is the corresponding normalization factor;

[0208] S406, apply the graph cut algorithm to optimize the segmentation results and minimize the energy function :

[0209] ;

[0210] Where, Assign labels, and Represents a pixel in an image; Is a data item, indicating that the label Assign to pixels the price; is a smoothing term, representing adjacent pixels and Penalties for being assigned a different label; Represents a set of adjacent pixel pairs.

[0211] S5. Using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, assign a corresponding region label number to each point in the point cloud, thereby obtaining a rice plant segmentation result for the entire dense point cloud; specifically, as follows:

[0212] S501, using the homography matrix Each point in the 3D point cloud Projected onto the two-dimensional image plane, it is represented as follows:

[0213] ;

[0214] in, 、 、 Represents the three-dimensional coordinates of each point in the three-dimensional point cloud, 、 represents the horizontal and vertical coordinates of the pixels in the two-dimensional image, is the final projection coordinate, is the corresponding scaling factor;

[0215] S502: Determine the segmentation area to which the projection point belongs based on its position;

[0216] S503, assigning label numbers of corresponding segmented regions to points in the three-dimensional point cloud;

[0217] S504: For points whose projected positions fall on the boundary of the segmented area, the nearest neighbor rule is used to determine the belonging of the points, as follows:

[0218] Identify the points whose projection positions fall on the boundary of the segmented regions; for each such point, calculate its distance to the adjacent segmented regions; assign the point to the segmented region with the closest distance; if there are multiple segmented regions with equal distances, choose the region with the largest area to determine its affiliation.

[0219] Example 2

[0220] This embodiment discloses a rice plant segmentation system based on perspective transformation and dense point cloud, which is used to implement the rice plant segmentation method based on perspective transformation and dense point cloud described in Example 1. Figure 4 As shown, it includes:

[0221] A point cloud acquisition unit is used to scan rice fields using a UAV-mounted lidar system to obtain a dense point cloud of the rice canopy;

[0222] a perspective transformation unit, configured to perform perspective transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at a top-down perspective;

[0223] A mapping establishment unit, configured to calculate a homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image;

[0224] a two-dimensional segmentation unit, configured to segment and mark rice plants in the two-dimensional image using a region marking method;

[0225] a three-dimensional labeling unit, configured to assign a corresponding region label number to each point in the point cloud using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, thereby obtaining a segmentation result of the entire dense point cloud;

[0226] Adaptive optimization unit, used to realize point cloud density control, multi-sensor data fusion, multi-scale feature extraction and temporal consistency constraint functions.

[0227] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A rice plant segmentation method based on perspective transformation and dense point cloud, characterized in that: include: S1. Use a UAV-mounted LiDAR system to scan the rice field and obtain a dense point cloud of the rice canopy. S2. performing angle transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at the top-down perspective; wherein the top-down perspective is vertically downward to maximize the separability between plants, including: S201. Based on the principal component analysis method, determine the main plane of the dense point cloud and calculate the covariance matrix C of the point cloud: Where, The first Points, is the centroid of the point cloud, N is the number of point clouds; S202. Calculate the normal vector of the main plane using the covariance matrix C: Perform eigenvalue decomposition on the covariance matrix C; Select the eigenvector corresponding to the smallest eigenvalue as the normal vector of the main plane ; The normal vector As the initial looking down direction; If the normal vector If the angle with the vertical direction exceeds the preset threshold, the following optimization is performed: An iterative plane fitting algorithm is used; the maximum number of iterations and the convergence threshold are set; the iterative process includes filtering the point set, recalculating the covariance matrix, and updating the normal vector; the algorithm stops when the convergence condition is met or the maximum number of iterations is reached; the final optimized normal vector is used as the determined top-down direction; S203: Perform coordinate transformation on the dense point cloud according to the determined overlooking direction, as follows: Calculate the rotation matrix , rotate the normal vector to axis parallel; Point cloud Apply the rotation matrix Among them, is the normal vector, is the Z-axis unit vector, and the rotation matrix is calculated using the quaternion or Euler angle method. , making ; Calculate the bounding box of the transformed point cloud and determine the new coordinate origin; Translate the point cloud so that the center of its bounding box coincides with the new coordinate origin; S204: Project the point cloud after coordinate transformation onto a horizontal plane to generate a two-dimensional grid image, where the value of each grid is determined by the number of points falling into the grid or the height statistics, as follows: Determine the grid size and set it to 1-2 times the average spacing of the point cloud; Create a blank raster image with a size covering the entire point cloud projection area; For each point, calculate its position in the raster image; Update grid values. The statistics you can choose include maximum height, average height or point density. Normalize the generated raster image and map the pixel values to the range of 0-255; S205: Post-process the generated two-dimensional raster image to enhance image quality and analyzability, as follows: Apply Gaussian filtering to remove noise; Use adaptive histogram equalization to enhance contrast; Morphological operations are used to remove small isolated areas; S206, saving the processed two-dimensional raster image into a common image format and recording the geographic reference information of the image so as to maintain consistency with the original point cloud data; S207, introduce adaptive multi-scale point cloud downsampling algorithm, according to the local point cloud density Dynamic voxel resizing ; Where, It is an adjustable parameter used to balance downsampling efficiency and detail preservation; S208. Applying principal component analysis to enhance point cloud structural features S3, calculating the homography matrix from the dense point cloud to the two-dimensional image, and establishing an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; S4. For the two-dimensional image, segment and mark the rice plants using a region marking method to achieve accurate segmentation and individual marking of the rice plants on the two-dimensional plane; S5. Using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, assign a corresponding region label number to each point in the point cloud, thereby obtaining a rice plant segmentation result of the entire dense point cloud.

2. The rice plant segmentation method based on viewpoint transformation and dense point cloud according to claim 1, characterized in that: The step S1 comprises: S101. On the UAV ground station, the rice field is set as the scanning area, and the heading overlap rate and lateral overlap rate of the scan are set. The UAV ground station automatically generates a route based on the operation information; wherein the heading overlap rate is set to 60%-80%, and the lateral overlap rate is set to 50%-70% to ensure that the required dense point cloud data is obtained; S102: The UAV autonomously flies according to the planned route, automatically performs lidar scanning according to a set overlap rate, and records the position and attitude information of the scan data; wherein the position and attitude information includes longitude, latitude, altitude, heading angle, pitch angle, and roll angle; the UAV's flight altitude is set to 20-50 meters and the flight speed is set to 3-8 meters per second to balance data collection efficiency and point cloud density; S103. The drone uploads the collected dense point cloud data to the server in real time based on a data link. The uploaded data includes both the point cloud coordinates and the reflection intensity information of each point. The drone and the server establish a secure data transmission channel based on a preset communication protocol. The data link utilizes a 4G / 5G network or a dedicated wireless communication module with a transmission rate of no less than 10 Mbps. S104. After receiving the point cloud data, the server performs a real-time data quality check, including an assessment of point cloud density, coverage, and data integrity. If the data quality does not meet the requirements, a rescan command is automatically sent to the drone. S105, preprocessing the received raw point cloud data, including noise removal, ground point filtering, and point cloud registration; wherein, noise removal uses a statistical outlier filtering algorithm, ground point filtering uses a progressive morphological filtering algorithm, and point cloud registration uses an iterative closest point algorithm; S106, storing the pre-processed point cloud data in a standard format and establishing a spatial index structure for subsequent rapid access and processing; S107, using adaptive point cloud density control algorithm to dynamically adjust the laser radar scanning frequency and drone flight speed ,satisfy: Where, and are the minimum and maximum point cloud densities set, is the scanning bandwidth; S108, realize multi-sensor data fusion, combine RTK-GPS, IMU and lidar data, and use extended Kalman filter algorithm to optimize point cloud positioning accuracy. The state estimation equation is: Where, is the optimal state estimate, is the Kalman gain, is the observation vector, is the observation matrix, It's time A priori state estimate of .

3. The rice plant segmentation method based on viewpoint transformation and dense point cloud according to claim 2, characterized in that: The step S3 comprises: S301. Select at least four pairs of corresponding points in the three-dimensional point cloud and the two-dimensional image to form a point pair set. The three-dimensional to two-dimensional perspective projection transformation has eight degrees of freedom, and each pair of corresponding points provides two constraint equations. Therefore, at least four pairs of corresponding points are required to uniquely determine the transformation. Select points with significant features in the 3D point cloud; Find the pixel positions in the 2D image corresponding to the points selected in the 3D point cloud to form point pairs; Make sure the selected point pairs are evenly distributed and cover the entire area of interest; S302: Calculate the initial homography matrix H using a direct linear transformation algorithm, as follows: Construct a system of linear equations, providing two equations for each pair of corresponding points; Use singular value decomposition to solve the linear equations and obtain the initial homography matrix H; S303: Use the random sample consistency algorithm to optimize the initial homography matrix H to improve the mapping accuracy, as follows: S3031. Randomly select a minimum number of point pair subsets, which belong to the above point pair set; S3032. Calculate a candidate homography matrix using the point pairs in the selected point pair subset; S3033. Calculate the projection errors of all point pairs under the candidate homography matrix; S3034. Counting the number of inliers whose errors are within a preset threshold; S3035. Repeat the above steps S3031-S3034 multiple times to select the model with the most inliers; S3036, recalculate the final homography matrix using all the inliers in the model with the most inliers ; S304, using the Levenberg-Marquardt algorithm Perform nonlinear optimization to further improve the accuracy of the homography matrix as follows: Define the objective function as the sum of squares of the reprojection errors of the point pair set; Based on this objective function, iterative optimization is performed with the matrix optimized by RANSAC as the initial value; In each iteration, the damping factor is adjusted to balance convergence speed and stability; When the error change is less than the preset threshold or the maximum number of iterations is reached, the optimization is stopped and the homography matrix with the best accuracy is obtained. ; S305, evaluate the final optimized homography matrix The accuracy is as follows: Calculate the average reprojection error for a set of point pairs; Calculate the spatial distribution consistency of the mapped points; If the average reprojection error and spatial distribution consistency accuracy do not meet the requirements, consider increasing the number of corresponding points or reselecting corresponding points; S306, introduce deep learning assisted feature matching algorithm, use convolutional neural network to extract feature descriptors of point cloud and image, network loss function Defined as: Where, is the descriptor loss, is the detector loss, is the balance factor; S307, using adaptive RANSAC algorithm to dynamically adjust the number of iterations : Where, is the expected probability of success, is the interior point ratio, is the minimum sample set size.

4. The rice plant segmentation method based on viewpoint transformation and dense point cloud according to claim 3, characterized in that: The step S4 comprises: S401, preprocessing the two-dimensional image, including denoising and contrast enhancement; S402, applying a watershed algorithm to perform preliminary segmentation on the pre-processed two-dimensional image; S403, optimizing the segmentation result using morphological operations, including opening and closing operations; S404, using connected component analysis to assign a unique label number to each segmented region; S405. Introduce a multi-scale superpixel segmentation algorithm, combine color and spatial information, and define a superpixel similarity metric. : Where, and are color distance and spatial distance respectively, and is the corresponding normalization factor; S406, apply the graph cut algorithm to optimize the segmentation results and minimize the energy function : Where, Assign labels, and Represents a pixel in an image; Is a data item, indicating that the label Assign to pixels the price; is a smoothing term, representing adjacent pixels and Penalties for being assigned a different label; Represents a set of adjacent pixel pairs.

5. The rice plant segmentation method based on viewpoint transformation and dense point cloud according to claim 1, characterized in that: The step S5 comprises: S501, using the homography matrix Each point in the 3D point cloud Projected onto the two-dimensional image plane, it is represented as follows: in, 、 、 Represents the three-dimensional coordinates of each point in the three-dimensional point cloud, 、 represents the horizontal and vertical coordinates of the pixels in the two-dimensional image, is the final projection coordinate, is the corresponding scaling factor; S502: Determine the segmentation area to which the projection point belongs based on its position; S503, assigning label numbers of corresponding segmented regions to points in the three-dimensional point cloud; S504: For points whose projected positions fall on the boundary of the segmented area, the nearest neighbor rule is used to determine the belonging of the points, as follows: Identify the points whose projection positions fall on the boundary of the segmented regions; for each such point, calculate its distance to the adjacent segmented regions; assign the point to the segmented region with the closest distance; if there are multiple segmented regions with equal distances, choose the region with the largest area to determine its affiliation.

6. A rice plant segmentation system based on view transformation and dense point cloud, characterized by: The method for implementing rice plant segmentation based on perspective transformation and dense point cloud according to any one of claims 1 to 5 comprises: A point cloud acquisition unit is used to scan rice fields using a UAV-mounted lidar system to obtain a dense point cloud of the rice canopy; a perspective transformation unit, configured to perform perspective transformation processing on the dense point cloud using a top-down perspective to obtain a two-dimensional image of the dense point cloud at a top-down perspective; A mapping establishment unit, configured to calculate a homography matrix from the dense point cloud to the two-dimensional image, and establish an accurate mapping relationship between the three-dimensional point cloud and the two-dimensional image; a two-dimensional segmentation unit, configured to segment and mark rice plants in the two-dimensional image using a region marking method; a three-dimensional marking unit, configured to assign a corresponding region marking number to each point in the point cloud by using the mapping relationship between the three-dimensional point cloud and the two-dimensional image, thereby obtaining a segmentation result of the entire dense point cloud; Adaptive optimization unit, used to realize point cloud density control, multi-sensor data fusion, multi-scale feature extraction and temporal consistency constraint functions.

Images