A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion

Abstract

A kind of corn stubble height measurement method based on depth camera point cloud reconstruction and feature fusion, acquires farmland multi-view RGB-D data, based on adaptive key frame and pose graph optimization technology, realizes high-precision splicing reconstruction of full-field three-dimensional point cloud by coordinate transformation and point cloud registration formula;For microtopography fluctuation, construct microtopography adaptive segmentation algorithm, realize accurate extraction of real ground surface through local terrain envelope surface;Introduce anisotropic geometric constraint density clustering algorithm, combine PCA multidimensional feature and crop row linear constraint to accurately extract stubble, filter out weed and lodging straw interference, realize accurate segmentation of single corn stubble point cloud;Finally, accurate stubble height is calculated by local reference surface fitting equation and percentile statistical method.The present application does not need to rely on GPU and deep learning pre-training, significantly improves the measurement accuracy, real-time performance and robustness in complex farmland environment, belongs to the field of precision agriculture and crop phenotype measurement technology.

Description

Technical Field

[0001] This invention relates to the fields of precision agriculture and crop phenotyping technology, specifically to a point cloud data processing method based on a consumer-grade RGB-D camera. This method is suitable for in-situ, precise, and portable measurement of stubble height after corn harvest, and can provide key data support for harvesting operation quality assessment and intelligent header contour control. Background Technology

[0002] As a crucial global food and energy crop, the quality of mechanized corn harvesting directly impacts the economic benefits and ecological security of subsequent industrial chains. Stubble height, a core indicator for evaluating harvesting quality, is of significant agronomic importance for precise control: excessively low stubble heights easily lead to root impurities carrying soil pathogens contaminating feed, increasing the risk of silage mold; while excessively high stubble heights result in biomass waste and provide overwintering host environments for pests and diseases. Therefore, achieving rapid and accurate in-situ measurement of corn stubble height in the field is not only a necessary means to assess the performance of current harvesting machinery but also a key sensing foundation for future automated contour-following control of intelligent headers. However, post-harvest cornfields typically exhibit unstructured characteristics, with undulating furrows, abundant lodged straw, weeds, and clods of soil, posing a significant challenge to the accurate measurement of stubble height.

[0003] Traditional stubble height measurement relies primarily on manual sampling using steel rulers. This method is inefficient, highly subjective, and fails to meet the demands of precision agriculture for large-area, high-time data acquisition. While remote sensing technology based on UAV platforms is widely used, airborne point cloud density is insufficient for describing fine structural features. Corn stubble diameters are typically only 2-4 cm, easily perceived as noise or confused with background textures under the large field of view of airborne systems, making individual-level accurate identification and measurement difficult. Therefore, it is impossible to perform precise individual-level identification and measurement of such tiny stubble diameters of only 2-4 cm.

[0004] Near-ground sensing technologies, especially consumer-grade RGB-D cameras, offer new possibilities for solving the aforementioned problems. However, existing technologies still have the following shortcomings when dealing with complex post-harvest farmland scenes:

[0005] (1) Difficulty in extracting ground reference: Traditional global plane fitting algorithms (such as RANSAC) are prone to model fitting bias on surfaces with significant micro-topographic undulations and straw cover, resulting in ground reference drift.

[0006] (2) Low accuracy in identifying small targets: Depth observation from a single perspective is easily obstructed, resulting in incomplete stubble point cloud data. At the same time, background artifacts such as weeds and soil clods that are similar to the stubble shape are very likely to cause false detections, which traditional spatial clustering methods cannot effectively distinguish.

[0007] (3) High dependence on computing power: Although existing high-precision deep learning methods (such as PointNet++) have good segmentation results, they rely on a large amount of labeled data and GPU computing power, making it difficult to deploy on low-cost portable terminals or agricultural machinery edge computing platforms.

[0008] Therefore, there is an urgent need for a lightweight method that does not rely on high computing power, is robust to the environment, and can achieve high-precision individual identification and height measurement. Summary of the Invention

[0009] To address the technical problems existing in the prior art, the purpose of this invention is to provide a method for measuring maize stubble height based on point cloud reconstruction and feature fusion using a depth camera. This method solves the data integrity problem through multi-view reconstruction, utilizes local statistical and geometric features to cope with complex terrain and background interference, and ultimately achieves high-precision, robust measurement without the need for a deep learning model, providing technical support for rapid field assessment and the development of intelligent equipment.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion includes the following steps:

[0012] Step 1: Acquire a continuous sequence of color and depth images of unstructured farmland using a handheld or vehicle-mounted RGB-D camera;

[0013] Step 2: Based on the image sequence acquired in Step 1, a high-precision and high-completeness three-dimensional point cloud of the field is reconstructed through adaptive keyframe filtering and optimization.

[0014] Step 3: Based on the three-dimensional point cloud of the field obtained in Step 2, a micro-topography adaptive segmentation algorithm is used to construct a local terrain envelope surface, which accurately segments the point cloud into ground point cloud and non-ground point cloud.

[0015] Step 4: Based on the non-ground point cloud obtained in Step 3, an anisotropic geometric constraint density clustering algorithm is used, combined with PCA multidimensional features and crop row linear constraints, to accurately extract individual point cloud clusters of corn stubble.

[0016] Step 5: Based on the ground point cloud obtained in Step 3 and the stubble point cloud cluster extracted in Step 4, search for ground points within a preset range at the base of each stubble, fit a local reference plane, and calculate the vertical distance from the stubble point cloud cluster to the plane, which is used as the height measurement value of the stubble.

[0017] As a preferred option, in step 1, the RGB-D camera slowly moves along an "S"-shaped path along the direction of the corn planting rows to acquire multi-angle, full-coverage image sequences of the target area.

[0018] As a preferred option, in step 2, the adaptive keyframe selection and multi-view point cloud reconstruction follow the following steps:

[0019] Step 2-1: For the acquired image sequence, calculate the Laplacian variance of each frame as a measure of sharpness, and calculate the optical flow variation between adjacent frames;

[0020] Step 2-2: Based on the sharpness, inter-frame differences and thresholds obtained in Step 2-1, a set of key frames with high texture quality and significant pose changes is intelligently selected according to the thresholds, and blurry and redundant frames are removed.

[0021] Steps 2-3: Perform bilateral filtering noise reduction on the selected keyframe depth maps, and use camera intrinsic parameters to backproject the depth maps into 3D space to generate local point clouds; convert the pixel coordinates (u,v) of the k-th frame and the corresponding depth value into 3D points in the camera coordinate system. = The conversion formula is:

[0022] X = Y = , =d,

[0023] in, The focal length of the camera in the horizontal and vertical directions. Here are the coordinates of the camera's principal point, and d represents the depth value corresponding to pixel (u, v) in the depth map.

[0024] Steps 2-4: Extract Fast Feature Histogram (FPFH) descriptors from the local point clouds of adjacent keyframes, perform coarse registration using the RANSAC algorithm, and obtain the initial transformation matrix; calculate the pose transformation matrix between the current frame and the previous keyframe, including the translation vector. and rotation matrix Set translation threshold and rotation threshold When satisfied or When this happens, the current frame is identified as a keyframe and included in the global stitching.

[0025] Steps 2-5: Based on the initial transformation in Step 2-4, a Point-to-Plane (ICP) algorithm is used for fine registration to optimize the relative pose of adjacent frames. The Iterative Nearest Point (IPP) algorithm is then used to register adjacent keyframes. The optimal rotation matrix R and translation vector t are solved by minimizing the objective function E, achieving a unified stitching of the point cloud to the world coordinate system.

[0026] E = ,

[0027] in, For points in the current frame source point cloud, N represents the nearest point in the target frame's point cloud, where N is the number of matching points.

[0028] Steps 2-6: Construct a pose graph containing the poses of all keyframes. By minimizing the global pose residual, the Levenberg-Marquardt algorithm is used for global optimization to eliminate cumulative drift and finally generate a complete dense point cloud of the field scene.

[0029] As a preferred option, the micro-topography adaptive segmentation algorithm in step 3 has the following specific steps:

[0030] Step 3-1: Project the stitched and reconstructed point cloud onto the XY 2D plane, and divide it into sections with dimensions of... A regular grid;

[0031] Step 3-2, for each grid Calculate the p-th percentile of the Z-axis height of the internal point cloud, and use it as the local ground height of the grid. The principle formula is as follows:

[0032] ;

[0033] Step 3-3: For empty grids caused by occlusion, use 3 Interpolation fill is performed on the median of the 3-grid neighborhood;

[0034] Steps 3-4: Height Normalization: Calculate the difference in height between each point and the ground level of its corresponding grid. The principle formula is shown below:

[0035] ,

[0036] in, This represents the set of Z-axis height values ​​for all point clouds within the current grid, where z is the original Z-axis height value of a single point.

[0037] As a preferred option, in step 4, the anisotropic geometrically constrained density clustering algorithm, combined with PCA multidimensional features and crop row linear constraints, accurately extracts stubble retention. The specific steps are as follows:

[0038] Step 4-1, Density Clustering: Perform DBSCAN density clustering on the non-ground point cloud obtained in Step 3 to divide the spatially discrete points into independent initial object clusters. The clustering radius Eps and the minimum number of points MinPts are set according to the point cloud density;

[0039] Step 4-2, PCA Feature Extraction: For each initial object cluster Construct its covariance matrix and perform singular value decomposition to obtain eigenvalues. and the corresponding feature vector Calculate the following geometric features:

[0040] ① Linearity: Describe whether the object is long and thin;

[0041] ② Verticality: This describes whether the main direction is perpendicular to the ground surface.

[0042] ③ Height-to-diameter ratio: ;

[0043] in, Represents the unit vector in the vertical direction. This represents the length of the point cloud cluster along the Z-axis. It refers to the maximum span in the horizontal direction, i.e., the XY plane.

[0044] Step 4-3, Weighted scoring and discrimination: Constructing a multidimensional feature scoring function S( When the total score At that time, it was determined that it might be corn stubble. It is a scoring threshold used to determine whether a candidate cluster is a corn stubble;

[0045] Step 4-4: Crop Row Linear Constraints: Extract the centroid coordinates of all candidate stubble clusters and fit them to a straight line using the RANSAC algorithm to simulate the crop row lines in mechanized planting; calculate the distance from the centroid of each candidate stubble cluster to the nearest fitted crop row line. If this distance is greater than the inter-row distance threshold, the line will be considered closed. If any weeds are found between rows, they are identified as weeds and removed. The remaining clusters are the precisely identified corn stubble individuals.

[0046] As a preferred option, the multidimensional feature scoring function S( in step 4-3) Specifically:

[0047] ,

[0048] in, This is an indicator function; its value is 1 if the condition is true, and 0 otherwise. , , , , where is the weight of the corresponding feature; , , These are the threshold values ​​for linearity, verticality, and height ratio, respectively. This refers to a reasonable range of values ​​for the stubble height.

[0049] As a preferred option, the stubble height calculation in step 5 is based on a local benchmark, and the specific steps are as follows:

[0050] Step 5-1: For each cluster of corn stubble points identified in Step 4, take its center as the center of a circle, and within a radius of... Within the cylindrical neighborhood, search for all ground points in the ground point cloud obtained from step 3.

[0051] Step 5-2: Using the local ground points searched in Step 5-1, fit the local plane equation using the least squares method: Ax + By + Cz + D = 0;

[0052] Step 5-3: Calculate each point in the stubble point cloud cluster. = The vertical distance h to the local fitting plane is calculated using the following formula:

[0053] h= ,

[0054] Step 5-4: Take the maximum vertical distance of all points in the cluster. = max( The final measured height of the corn stubble was determined by this measurement.

[0055] Among them, A, B, C, and D are the parameters of the local plane equation obtained by least squares fitting. This represents the vertical distance from the i-th stubble point to the local fitting plane.

[0056] As a preferred method, in step 1, the RGB-D camera is mounted on a portable bracket, maintaining a height of approximately 0.8-1.2m above the ground. The operator moves slowly along the crop's direction of travel in an "S" shaped path to take pictures, ensuring multi-angle coverage of the same area and maintaining an overlap rate of over 60% between adjacent frames.

[0057] As a preferred option, in step 5-1, the search radius... The value ranges from 0.1m to 0.3m.

[0058] As a preferred option, in step 4-1, the cluster radius Eps ranges from 0.01 m to 0.05 m, and the minimum number of points MinPts ranges from 10 to 25.

[0059] The principle of this invention is as follows: First, acquire multi-view RGB-D data of farmland. Second, based on adaptive keyframe and pose graph optimization techniques, achieve high-precision stitching and reconstruction of the entire three-dimensional point cloud through coordinate transformation and point cloud registration formulas. Third, construct a micro-topography adaptive segmentation algorithm for micro-topographic undulations, achieving accurate extraction of the real ground surface through local topographic envelope. Fourth, introduce an anisotropic geometric constraint density clustering algorithm, combining PCA multidimensional features and crop row linear constraints to accurately extract stubble, filtering out weeds and lodged straw interference, achieving accurate segmentation of single-plant corn stubble point clouds. Finally, calculate the precise stubble height using local reference surface fitting equations and percentile statistical methods.

[0060] The present invention has the following advantages:

[0061] 1. High data integrity: Through adaptive keyframe filtering and pose graph optimization, multi-view data is effectively integrated, solving the problems of target occlusion and data loss under single viewpoint, and significantly improving the integrity of 3D reconstruction with small stubs.

[0062] 2. Strong terrain adaptability: The proposed micro-topography adaptive segmentation algorithm (MAS-LSP) uses local grid statistics to replace the global plane assumption, which can closely fit the undulation of field furrows and accurately extract the ground reference, effectively avoiding the undersegmentation or oversegmentation problems of traditional methods on unstructured terrain.

[0063] 3. High recognition accuracy and robustness: Combining DBSCAN clustering and PCA analysis, and supplemented by crop row linear constraints, it can accurately distinguish corn stubble from similarly shaped weeds and clods of soil, achieving high-precision individual recognition in complex backgrounds, with an F1 score of over 89%.

[0064] The F1 score is a commonly used metric in classification problems to comprehensively evaluate precision and recall. It is defined as the harmonic mean of the two, i.e., F1. Precision = Number of correctly identified stubble / Total number of "stubble" detected by the algorithm – measures precision (avoiding false positives); Recall = Number of correctly identified stubble / Total number of actual stubble – measures recall (avoiding false negatives).

[0065] 4. Lightweight and easy to deploy: The entire method is based on geometric constraints, does not rely on deep learning models and massive labeled data, has low requirements for computing resources, and can run efficiently on portable devices or edge computing terminals without GPU support, providing a feasible technical path for developing low-cost field measurement tools. Attached Figure Description

[0066] Figure 1 This is the overall flowchart of the method of the present invention.

[0067] Figure 2 This is a flowchart of the adaptive keyframe extraction, registration, and stitching process of the present invention.

[0068] Figure 3 This is a schematic diagram of the adaptive keyframe registration and splicing principle of the present invention.

[0069] Figure 4 This is a schematic diagram of the envelope fitting of the micro-terrain adaptive segmentation algorithm (MAS-LSP) of this invention.

[0070] Figure 5 This is a schematic diagram of the stubble segmentation and height detection of the present invention.

[0071] Figure 6 This is an experimental diagram of stubble-retaining farmland reconstruction in this invention.

[0072] Figure 7 This is an experimental diagram of stubble-retaining farmland division in this invention. Detailed Implementation

[0073] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] like Figure 1 A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion is described below. The implementation process includes steps 1 to 5.

[0075] Step 1: Acquire a continuous sequence of color and depth images of unstructured farmland using a handheld or vehicle-mounted RGB-D camera.

[0076] The image sequence acquisition process is as follows: In this embodiment, a handheld RGB-D camera is used, with an Intel RealSense D455 depth camera selected as the core sensing device. The camera is fixed on a portable carbon fiber bracket, maintaining a height of approximately 1.0m off the ground, with the lens optical axis slightly tilted downwards (approximately 15°). The operator walks slowly along the direction of the corn planting rows in an "S" shaped path, holding the camera to acquire multi-angle, full-coverage image sequences of the target area, ensuring that the overlap rate between adjacent frames exceeds 60%. During the acquisition process, violent shaking should be avoided, and samples should be collected under different lighting conditions (such as cloudy and sunny days) as much as possible.

[0077] Step 2: Based on the image sequence acquired in Step 1, a high-precision, high-completeness 3D point cloud of the field is reconstructed through adaptive keyframe selection and optimization, such as... Figure 2and Figure 3 As shown.

[0078] Step 2-1: For the acquired RGB image sequence, calculate the Laplacian variance of each frame as a sharpness measure. Frames with a score lower than a preset threshold (e.g., 100) are judged as blurry frames and are discarded. Calculate the optical flow change between adjacent frames. If the optical flow change is less than a threshold (e.g., 5 pixels), it indicates that the pose change between the two frames is too small, and the frames are redundant and are discarded. Finally, high-quality keyframes are selected.

[0079] Step 2-2: Based on the sharpness, inter-frame differences and thresholds obtained in Step 2-1, a set of key frames with high texture quality and significant pose changes is intelligently selected according to the thresholds, and blurry and redundant frames are removed.

[0080] Steps 2-3: Perform bilateral filtering noise reduction on the selected keyframe depth maps, and use camera intrinsic parameters to backproject the depth maps into 3D space to generate local point clouds; convert the pixel coordinates (u,v) of the k-th frame and the corresponding depth value into 3D points in the camera coordinate system. = The conversion formula is:

[0081] X = Y = , =d,

[0082] in, The focal length of the camera in the horizontal and vertical directions. Let d be the coordinates of the principal point of the camera, and d be the depth value corresponding to pixel (u, v) in the depth map.

[0083] Steps 2-4: Extract Fast Feature Histogram (FPFH) descriptors from the local point clouds of adjacent keyframes, perform coarse registration using the RANSAC algorithm, and obtain the initial transformation matrix; calculate the pose transformation matrix between the current frame and the previous keyframe, including the translation vector. and rotation matrix Set translation threshold and rotation threshold When satisfied or When the current frame is identified as a key frame, it is used to participate in the global stitching.

[0084] As a preferred option, in steps 2-4, the translation threshold... The value ranges from 0.05 m to 0.10 m, and the rotation threshold is... The value ranges from 5° to 10°.

[0085] Steps 2-5: Based on the initial transformation in Step 2-4, a Point-to-Plane (ICP) algorithm is used for fine registration to optimize the relative pose of adjacent frames. The Iterative Nearest Point (IPP) algorithm is then used to register adjacent keyframes. The optimal rotation matrix R and translation vector t are solved by minimizing the objective function E, achieving a unified stitching of the point cloud to the world coordinate system.

[0086] E = ,

[0087] in, For points in the current frame source point cloud, N represents the nearest point in the target frame point cloud, and N is the number of matching points.

[0088] Steps 2-6: Construct a pose graph containing the poses of all keyframes. By minimizing the global pose residual, the Levenberg-Marquardt algorithm is used for global optimization to eliminate cumulative drift and finally generate a complete dense point cloud of the field scene.

[0089] Step 3: Based on the 3D point cloud obtained in Step 2, the Micro-Topography Adaptive Segmentation Algorithm (MAS-LSP) is used to construct a local terrain envelope, accurately segmenting the point cloud into ground point cloud and non-ground point cloud, such as... Figure 4 As shown.

[0090] Step 3-1: Project the stitched and reconstructed point cloud onto the XY 2D plane, and divide it into sections with dimensions of... (In this embodiment, a regular grid of 0.05m is used.)

[0091] Step 3-2, for each grid Calculate the p-th percentile (p=10) of the Z-axis height of the internal point cloud, and use it as the local ground height of the grid. The principle formula is as follows:

[0092] .

[0093] Step 3-3: For empty grids caused by occlusion, use 3 Interpolation fills in the median of the 3-grid neighborhood.

[0094] Steps 3-4: Height Normalization: Calculate the difference in height between each point and the ground level of its corresponding grid. The principle formula is shown below:

[0095] ,

[0096] in, This represents the set of Z-axis height values ​​for all point clouds within the current grid, where z is the original Z-axis height value of a single point.

[0097] Step 4: Based on the non-ground point cloud obtained in Step 3, the anisotropic geometrically constrained density clustering algorithm (AG-DBSCAN) is used, combined with PCA multidimensional features and crop row linear constraints, to accurately extract individual point cloud clusters of maize stubble. The specific steps are as follows:

[0098] Step 4-1, Density Clustering: Perform DBSCAN density clustering on the non-ground point cloud obtained in Step 3 to divide the spatially discrete points into independent initial object clusters. The clustering radius Eps and the minimum number of points MinPts are set according to the point cloud density.

[0099] The cluster radius Eps ranges from 0.01 m to 0.05 m, and the minimum number of points MinPts ranges from 10 to 25.

[0100] Step 4-2, PCA Feature Extraction: For each initial object cluster Construct its covariance matrix and perform singular value decomposition to obtain eigenvalues. and the corresponding feature vector Calculate the following geometric features:

[0101] ① Linearity: Describe whether the object is long and thin;

[0102] ② Verticality: This describes whether the main direction is perpendicular to the ground surface.

[0103] ③ Aspect Ratio: .

[0104] in, Represents the unit vector in the vertical direction. This represents the length of the point cloud cluster along the Z-axis. It refers to the maximum span in the horizontal direction (XY plane).

[0105] Step 4-3, Weighted scoring and discrimination: Constructing a multidimensional feature scoring function S( When the total score At that time, it was determined that it might be corn stubble. It is a scoring threshold used to determine whether a candidate cluster is a corn stubble.

[0106] Multidimensional feature scoring function S( Specifically:

[0107] ,

[0108] in, This is an indicator function; its value is 1 if the condition is true, and 0 otherwise. , , , The weights are for the corresponding features; , , These are the threshold values ​​for linearity, verticality, and height ratio, respectively. This refers to a reasonable range of values ​​for the stubble height.

[0109] Step 4-4: Crop Row Linear Constraints: Extract the centroid coordinates of all candidate stubble clusters and fit them to a straight line using the RANSAC algorithm to simulate the crop row lines in mechanized planting; calculate the distance from the centroid of each candidate stubble cluster to the nearest fitted crop row line. If this distance is greater than the inter-row distance threshold, the line will be considered closed. If the height is set to 10 cm, it is identified as inter-row weed and removed. The remaining clumps are the accurately identified corn stubble individuals.

[0110] Step 5: Based on the ground point cloud obtained in Step 3 and the stubble point cloud cluster extracted in Step 4, search for ground points within a preset range at the base of each stubble, fit a local reference plane, and calculate the vertical distance from the stubble point cloud cluster to this plane, which is used as the height measurement value of the stubble. Details are as follows.

[0111] Step 5-1: For each cluster of corn stubble points identified in Step 4, calculate its directed bounding box. For example... Figure 5 As shown, unlike the traditional Axially Bounding Box (AABB), the OBB determines the principal orientation of the point cloud through Principal Component Analysis (PCA) and constructs a minimum-volume cuboid that tightly encloses the point cloud along this orientation and follows its shape. With its center as the center, within a radius of... Within the cylindrical neighborhood, search for all ground points in the ground point cloud obtained from step 3; search radius The value ranges from 0.1m to 0.3m.

[0112] Step 5-2: Using the local ground points searched in Step 5-1, fit the local plane equation using the least squares method: Ax + By + Cz + D = 0;

[0113] Step 5-3: Calculate each point in the stubble point cloud cluster. = The vertical distance h to the local fitting plane is calculated using the following formula:

[0114] h= ,

[0115] Step 5-4: Take the maximum vertical distance of all points in the cluster. = max( The final height of the corn stubble is determined by the measured value. The system iterates through all identified stubble clusters and outputs the measurement results.

[0116] Among them, A, B, C, and D are the parameters of the local plane equation obtained by least squares fitting. This represents the vertical distance from the i-th stubble point to the local fitting plane.

[0117] To verify the effectiveness of the method proposed in this invention, field data collection and algorithm testing were conducted in a typical corn field after harvest. The experimental field had obvious ridge and furrow undulations and was covered with a large amount of lodged straw, weeds, and soil clods, representing a typical unstructured and complex scenario.

[0118] Quantitative analysis showed that the single-view depth map had obvious blind spots on the stubble back side and root area, with a point cloud integrity of only 42.6%. Through multi-frame fusion, the point cloud density of the reconstructed field scene was increased to an average of 300,000 points / m², as shown in the reconstruction results. Figure 6 As shown, the integrity of the stubble structure reached 91.5%. In particular, for small stubble with a diameter of less than 3 cm, multi-view reconstruction effectively filled the deep fracture area, providing a continuous geometric surface for feature extraction.

[0119] The segmentation effect of this invention is as follows: Figure 7 As shown, although there are some falsely detected stubble, the number is small, and almost all stubble was found and divided into red parts.

[0120] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion, characterized in that, Includes the following steps: Step 1: Acquire a continuous sequence of color and depth images of unstructured farmland using a handheld or vehicle-mounted RGB-D camera; Step 2: Based on the image sequence acquired in Step 1, a high-precision and high-completeness three-dimensional point cloud of the field is reconstructed through adaptive keyframe filtering and optimization. Step 3: Based on the three-dimensional point cloud of the field obtained in Step 2, a micro-topography adaptive segmentation algorithm is used to construct a local terrain envelope surface, which accurately segments the point cloud into ground point cloud and non-ground point cloud. Step 4: Based on the non-ground point cloud obtained in Step 3, an anisotropic geometric constraint density clustering algorithm is used, combined with PCA multidimensional features and crop row linear constraints, to accurately extract individual point cloud clusters of corn stubble. Step 5: Based on the ground point cloud obtained in Step 3 and the stubble point cloud cluster extracted in Step 4, search for ground points within a preset range at the base of each stubble, fit a local reference plane, and calculate the vertical distance from the stubble point cloud cluster to the plane, which is used as the height measurement value of the stubble.

2. The method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion as described in claim 1, characterized in that, In step 1, the RGB-D camera slowly moves along an "S"-shaped path along the direction of the corn planting rows to acquire multi-angle, full-coverage image sequences of the target area.

3. The method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion according to claim 1, characterized in that, In step 2, the adaptive keyframe selection and multi-view point cloud reconstruction are performed as follows: Step 2-1: For the acquired image sequence, calculate the Laplacian variance of each frame as a measure of sharpness, and calculate the optical flow variation between adjacent frames; Step 2-2: Based on the sharpness, inter-frame differences and thresholds obtained in Step 2-1, a set of key frames with high texture quality and significant pose changes is intelligently selected according to the thresholds, and blurry and redundant frames are removed. Steps 2-3: Perform bilateral filtering noise reduction on the selected keyframe depth maps, and use camera intrinsic parameters to backproject the depth maps into 3D space to generate local point clouds; convert the pixel coordinates (u,v) of the k-th frame and the corresponding depth value into 3D points in the camera coordinate system. = The conversion formula is: X = , Y = , =d, in, The focal length of the camera in the horizontal and vertical directions. Here are the coordinates of the camera's principal point, and d represents the depth value corresponding to pixel (u, v) in the depth map. Steps 2-4: Extract Fast Feature Histogram (FPFH) descriptors from the local point clouds of adjacent keyframes, perform coarse registration using the RANSAC algorithm, and obtain the initial transformation matrix; calculate the pose transformation matrix between the current frame and the previous keyframe, including the translation vector. and rotation matrix Set translation threshold and rotation threshold When satisfied or When this happens, the current frame is identified as a keyframe and included in the global stitching. Steps 2-5: Based on the initial transformation in Step 2-4, a point-to-surface ICP algorithm is used for fine registration to optimize the relative pose of adjacent frames. The iterative nearest-point ICP algorithm is then used to register adjacent keyframes. The optimal rotation matrix R and translation vector t are solved by minimizing the objective function E, achieving a unified stitching of the point cloud to the world coordinate system. E = ， in, For the points in the current frame source point cloud, N represents the nearest point in the target frame's point cloud, where N is the number of matching points. Steps 2-6: Construct a pose graph containing the poses of all keyframes. By minimizing the global pose residual, the Levenberg-Marquardt algorithm is used for global optimization to eliminate cumulative drift and finally generate a complete dense point cloud of the field scene.

4. The method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion as described in claim 1, characterized in that, Step 3, the micro-topography adaptive segmentation algorithm, has the following specific steps: Step 3-1: Project the stitched and reconstructed point cloud onto the XY 2D plane, and divide it into sections with dimensions of... A regular grid; Step 3-2, for each grid Calculate the p-th percentile of the Z-axis height of the internal point cloud, and use it as the local ground height of the mesh. The principle formula is as follows: ； Step 3-3: For empty grids caused by occlusion, use 3 Interpolate and fill in the median of the 3-grid neighborhood; Steps 3-4: Height Normalization: Calculate the difference in height between each point and the ground level of its corresponding grid. The principle formula is shown below: ， in, This represents the set of Z-axis height values ​​for all point clouds within the current grid, where z is the original Z-axis height value of a single point.

5. The method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion according to claim 1, characterized in that, In step 4, the anisotropic geometrically constrained density clustering algorithm, combined with PCA multidimensional features and crop row linear constraints, accurately extracts stubble retention. The specific steps are as follows: Step 4-1, Density Clustering: Perform DBSCAN density clustering on the non-ground point cloud obtained in Step 3 to divide the spatially discrete points into independent initial object clusters. The clustering radius Eps and the minimum number of points MinPts are set according to the point cloud density; Step 4-2, PCA Feature Extraction: For each initial object cluster Construct its covariance matrix and perform singular value decomposition to obtain eigenvalues. and the corresponding feature vector Calculate the following geometric features: ① Linearity: Describe whether the object is long and thin; ② Verticality: This describes whether the main direction is perpendicular to the ground surface. ③ Height-to-diameter ratio: ; in, Represents the unit vector in the vertical direction. This represents the length of the point cloud cluster along the Z-axis. It refers to the maximum span in the horizontal direction, i.e., the XY plane. Step 4-3, Weighted scoring and discrimination: Constructing a multidimensional feature scoring function S( When the total score At that time, it was determined that it might be corn stubble. It is a scoring threshold used to determine whether a candidate cluster is a corn stubble; Step 4-4: Crop Row Linear Constraints: Extract the centroid coordinates of all candidate stubble clusters and fit them to a straight line using the RANSAC algorithm to simulate the crop row lines in mechanized planting; calculate the distance from the centroid of each candidate stubble cluster to the nearest fitted crop row line. If this distance is greater than the inter-row distance threshold, the line will be considered closed. If any weeds are found between rows, they are identified as weeds and removed. The remaining clusters are the precisely identified corn stubble individuals.

6. The method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion according to claim 5, characterized in that, Step 4-3 Multidimensional Feature Scoring Function S( Specifically: ， in, This is an indicator function; its value is 1 if the condition is true, and 0 otherwise. , , , , where is the weight of the corresponding feature; , , These are the threshold values ​​for linearity, verticality, and height ratio, respectively. This refers to a reasonable range of values ​​for the stubble height.

7. The method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion according to claim 1, characterized in that, Step 5 involves calculating the stubble height based on a local benchmark. The specific steps are as follows: Step 5-1: For each cluster of corn stubble points identified in Step 4, take its center as the center of a circle, and within a radius of... Within the cylindrical neighborhood, search for all ground points in the ground point cloud obtained from step 3. Step 5-2: Using the local ground points searched in Step 5-1, fit the local plane equation using the least squares method: Ax + By + Cz + D = 0; Step 5-3: Calculate each point in the stubble point cloud cluster. = The vertical distance h to the local fitting plane is calculated using the following formula: h= ， Step 5-4: Take the maximum vertical distance of all points in the cluster. = max( The final measured height of the corn stubble was determined by this measurement. Among them, A, B, C, and D are the parameters of the local plane equation obtained by least squares fitting. This represents the vertical distance from the i-th stubble point to the local fitting plane.

8. A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion as described in claim 2, characterized in that, In step 1, the RGB-D camera is mounted on a portable bracket, maintaining a height of approximately 0.8-1.2m above the ground. The operator moves slowly along the crop's direction of travel in an "S" shaped path to take pictures, ensuring multi-angle coverage of the same area and maintaining an overlap rate of over 60% between adjacent frames.

9. A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion as described in claim 7, characterized in that, In step 5-1, the search radius The value ranges from 0.1m to 0.3m.

10. A method for measuring corn stubble height based on depth camera point cloud reconstruction and feature fusion as described in claim 5, characterized in that, In step 4-1, the cluster radius Eps ranges from 0.01 m to 0.05 m, and the minimum number of points MinPts ranges from 10 to 25.

Images