A Method for Detecting Crop Rows of Soybeans at the Mature Stage Based on 3D LiDAR

Abstract

The present invention proposes a method for detecting soybean crop rows based on 3D LiDAR, belonging to the field of lidar detection. It solves the problems that the existing image methods are greatly affected by light, and there is little research on the detection of soybean crop rows by lidar, especially in the aspects of different planting densities, missing seedlings and determination of missing rows. The method includes: collecting soybean point cloud data and performing point cloud tilt correction; performing segmented clustering according to the corrected point cloud; extracting the point cloud of the canopy of a single soybean plant by using the main stem method; performing spatial division on the canopy point cloud through a dynamic grid division method, dynamically calculating the grid size according to the point cloud density, and establishing a two-dimensional grid index; performing missing seedling point cloud completion and pseudo-crop row determination according to the divided canopy point cloud grid; using the least square method to perform linear fitting on the completed canopy point cloud to generate the center line of the crop row as the recognition result of the crop area. It is mainly used in the field of soybean detection.

Description

Technical Field

[0001] The present invention belongs to the field of lidar detection, and particularly relates to a method for detecting soybean mature crop rows based on 3D LiDAR. Background Technique

[0002] As one of the important food crops in the world, soybeans have extensive economic and nutritional value. Its total global planting area is about 120 million hectares, spreading all over the world, and the annual output is nearly 300 million tons. In the task of harvesting agricultural machinery during the mature period of soybeans, a single agricultural machine often needs to undertake the harvesting requirements of soybeans with different planting densities in multiple farmlands. As a prerequisite for the automatic navigation of agricultural machinery, crop row extraction currently mainly has two methods, namely image cameras and lidar.

[0003] Early work on crop row detection based on image cameras usually uses the Hough transform (HT) algorithm to extract the points in the image space and map them to the parameter space, and determines the parameter representation of the crop row navigation line through the cumulative voting mechanism. With the development of deep learning, the combination of image segmentation has new applications in crop row detection, not limited to the experimental farmland environment without interference. Regarding the research on the influence of outdoor light on green crops, some scholars proposed "Automatic Segmentation of Crop / Background Based on Luminance Partition Correction and Adaptive Threshold" in 2020 (a method for crop detection based on color index with luminance discrimination and color saturation calibration). Through the high-tone classification otsu method, the gray-scale images of weeds, duckweed, and rice are clustered to extract the feature points of green rice, and a two-dimensional adaptive clustering method is used to fit the crop rows. With the concept of precision agriculture proposed, to prevent pesticide waste and environmental pollution, the ResC-UNet network was improved. By performing contour search and minimum rectangle fitting on the converted image, the center line is extracted while obtaining the width, and the actual row line is extracted according to the position relationship between the width and the row line. An image dataset for the seedling stage and the middle growth stage was constructed, replacing the backbone network of YOLOv8 for crop row detection, improving the SuperGreen method, enhancing the distinction between corn and the farmland background, and adopting the newly proposed local-global detection method to identify the corn crop row line. The process of this method first locates the local crop rows and then further identifies the global crop rows. To sum up, the method for crop row detection based on image cameras has been relatively mature. However, due to the complex and changeable lighting environment in farmlands, the adaptability of the model based on image cameras is not good, and there is no systematic algorithm for processing. Lidar is superior to image cameras in terms of detection range, resistance to rain and snow environments, and resistance to changes in light and darkness. Therefore, it is difficult for algorithms based on image cameras to detect crop rows in different complex environments.

[0004] As a measuring device that uses pulsed laser to irradiate a target and measures the return time of the reflected pulse to determine the target distance, lidar has higher accuracy in target detection because it is insensitive to changes in shadow and lighting conditions. With the progress of technology, low-cost lidar has gradually become an agricultural application technology that has attracted much attention. In the early stage, lidar combined with navigation lines was usually applied to fruit trees. From the extracted orchard structure and terrain change features, the nearest neighbor pattern gradient was used to find the row direction, and thus the center line was found. As the device accuracy improved, when the ground crops obscured each other, the corn crops were bisected in horizontal strips, and the Euclidean clustering algorithm was improved to filter and extract feature points according to the candidate point information and corresponding thresholds. Considering the morphological changes of the same crop at different times, some scholars proposed a dynamic horizontal division clustering method to determine the feature points of the crop canopy. For the autonomous navigation of agricultural robots in grape greenhouses, an online regression model of the structure was constructed by applying the Hough transform and parametric and search methods.

[0005] From the above work, it can be seen that the current crop row detection method based on lidar has formed a detection pipeline of segmentation, noise reduction, canopy point extraction and line fitting. However, there is relatively little research on crops such as soybeans. The canopy center points have not been extracted according to the growth characteristics of soybeans, and there is a lack of crop row detection methods applicable to different densities of such crops, as well as the determination of missing seedlings and missing crop rows. Most of the 3-D structure feature extraction methods for crops such as soybeans are based on single density features. Therefore, the lidar-based soybean row detection algorithm still needs to be improved. Summary of the Invention

[0006] In view of this, the present invention aims to propose a method for detecting soybean mature crop rows based on 3D LiDAR to solve the problem that the existing image methods are greatly affected by light, while there is relatively little research on lidar in the detection of soybean crop rows, especially in terms of different planting densities, determination of missing seedlings and missing rows.

[0007] To achieve the above object, the present invention adopts the following technical solutions: A method for detecting soybean mature crop rows based on 3D LiDAR, the method comprising:

[0008] Step S1: Collect soybean point cloud data and perform point cloud tilt correction on the soybean point cloud data;

[0009] Step S2: Perform segmented clustering according to the corrected point cloud, and the segmented clustering includes: using DBSCAN density clustering in the low-density area and Euclidean distance clustering in the high-density area;

[0010] Step S3: Extract the canopy point cloud of a single soybean plant after segmented clustering using the main stem method, including: locating the lowest point of the root system to construct a cylindrical region, and extracting the highest point on the Z-axis within this region as the canopy point cloud;

[0011] Step S4: Conduct spatial partitioning on the canopy point cloud through a dynamic grid partitioning method, dynamically calculate the grid size according to the point cloud density, and establish a two-dimensional grid index;

[0012] Step S5: Complete the missing seedling point cloud and determine the pseudo-crop rows based on the partitioned canopy point cloud grid, including: generating a completed point cloud for the region where the point cloud missing rate in the column direction exceeds the threshold;

[0013] Step S6: Use the least squares method to perform linear fitting on the completed canopy point cloud to generate the center line of the crop row, and the center line of the crop row is the recognition result of the crop area.

[0014] Furthermore, a preferred method is also proposed. The step S1 includes: performing an Euler angle rotation matrix transformation on the tilted point cloud data collected by LiDAR to correct the tilted coordinate system to the horizontal ground coordinate system.

[0015] Furthermore, a preferred method is also proposed. In the step S3, when locating the lowest point of the root system to construct a cylindrical region and extracting the highest point on the Z-axis within this region as the canopy point cloud, it includes:

[0016] Extract the point cloud at the bottom of the main root system of the soybean;

[0017] Taking the root system point cloud as the center to construct a cylindrical region with a radius R = 5 cm, and the height range ;

[0018] Extract the highest point on the Z-axis within the cylindrical region as the canopy point cloud of this crop:

[0019]

[0020] Among them, is the set of point clouds within the cylindrical region, is the canopy point cloud of this soybean plant, is the th Z-axis coordinate of the point.

[0021] Furthermore, a preferred method is also proposed. In the step S4, when dynamically calculating the grid size according to the point cloud density, it includes:

[0022] ,

[0023] ,

[0024] ,

[0025] Among them, is the area of a single grid, is the grid expected value, is the area width, is the area height, is the total number of grids in the X-axis direction, is the total number of grids in the Y-axis direction, is the total size of the grid width, is the total size of the grid height.

[0026] Furthermore, a preferred method is also proposed. The step S5 includes:

[0027]

[0028]

[0029] Among them, is the pseudo-crop row determination, is the number of grids in the X-axis direction for complementing the point cloud, is the number of grids in the Y-axis direction for complementing the point cloud, is to determine that this row skips the navigation line generation for the pseudo-crop row, is to determine that this row generates the navigation line for the soybean crop row, and y represent the coordinate values of the axes of the complemented point cloud, is the total number of canopy point clouds in the column, is the width of a single grid, is the height of a single grid, is the minimum value of the grid width, is the minimum value of the grid height.

[0030] Furthermore, a preferred method is also proposed. The step S6 includes:

[0031]

[0032] Among them, is the slope of the straight line, is the intercept of the straight line, is the Y-axis coordinate of the canopy point cloud, represents the X-axis coordinate of the canopy point cloud.

[0033] Furthermore, a preferred method is also proposed. The method further includes: establishing a point cloud processing time threshold model according to the coupling relationship between the lidar scanning frequency and the agricultural machinery driving speed to achieve dynamic adaptation of the processing delay and the mechanical movement.

[0034] Furthermore, a preferred method is also proposed. The point cloud processing time threshold model includes:

[0035] Basic time constraint based on scanning frequency :

[0036]

[0037] Wherein, is the output frequency;

[0038] Maximum allowable delay frames :

[0039]

[0040] Wherein, is the effective soybean crop row scanning range, is the vehicle speed;

[0041] Maximum allowable processing time :

[0042]

[0043] Wherein, is the safety factor.

[0044] Based on the same inventive concept, the present invention also provides a computer device, including a memory and a processor. A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for detecting soybean mature crop rows based on 3D LiDAR described in any one of the above.

[0045] Based on the same inventive concept, the present invention also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of a method for detecting soybean mature crop rows based on 3D LiDAR described in any one of the above are executed.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] Existing LiDAR methods use the height threshold method to extract canopy points. However, due to the dense branches and leaves of soybean plants, the top leaves are easily misjudged as canopy points, resulting in the deviation of feature points. The present invention first creates the main stem method (MSM). Combining with the biological characteristics of the upright growth of soybeans, root lowest point positioning and Z-axis highest point screening are proposed. The data of the embodiment show that the positioning error of canopy points under LD density is reduced by 62.5% (AP reaches 93%, which is 4.3% higher than the comparative algorithm), and the spatial distribution of feature points conforms to the growth law of the main stem of soybeans.

[0048] Existing algorithms cannot adapt to the differences in point cloud distributions at different planting densities, resulting in high-density occlusion and low-density broken rows. The present invention proposes a collaborative scheme of segmented clustering and dynamic grid division. Through Y-axis segmented clustering, the problem of clustering failure caused by density mutation is solved (in the embodiment, the AP reaches 88.4% under the HD density, an increase of 6% compared with the traditional DBSCAN); based on the point cloud density, the grid size is dynamically calculated to achieve an adaptive match between grid division and planting density.

[0049] Regarding the problem of false detection caused by missing seedlings and weed interference, the present invention designs a dual-threshold determination mechanism to complete the filling of missing seedlings, with the filling rate increased by 37% (in the embodiment, the broken row repair rate reaches 92% under the HD density); and pseudo crop row filtering is performed to eliminate weed interference, reducing the false detection rate to 5.2% (a decrease of 8.3% compared with the traditional method).

[0050] Existing algorithms do not consider the matching problem between agricultural machinery movement and data processing delay, resulting in navigation lag. The present invention first establishes a point cloud processing time threshold model. Through dynamic delay control and multi-frame buffering mechanisms, navigation interruption caused by single-frame processing timeout is avoided.

[0051] Furthermore, the traditional view holds that crop row detection should first ensure the universality of the algorithm. However, the present invention designs the main stem method and dynamic grid specifically for soybeans, achieving unexpected effects. Although the main stem method increases the computational amount by 18%, through segmented clustering preprocessing, the total time consumption only increases by 3%; the dynamic grid division breaks the industry convention of fixed grids, reducing the detection stability in different density scenarios from 0.18 to 0.09. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0053] Figure 1 is a flowchart of a method for detecting crop rows of soybeans at the mature stage based on 3D LiDAR according to the present invention;

[0054] Figure 2 is a visualization comparison diagram of the point cloud before rotation according to the present invention;

[0055] Figure 3 is a visualization comparison diagram of the point cloud after rotation according to the present invention;

[0056] Figure 4 is a segmented clustering diagram of the point cloud of soybeans according to the present invention;

[0057] Figure 5 is a comparison diagram of the canopy point cloud extracted by the height threshold and root system method according to the present invention;

[0058] Figure 6 Schematic diagram of dynamic division of canopy point cloud grid method according to the present invention. In the figure, Xmax, Xmin, Ymin, and Ymax are the boundary values of the point cloud; is the width of the area, is the height of the area;

[0059] Figure 7 Schematic diagram of the soybean canopy grid division result according to the present invention; among them, is the minimum value of the grid width, is the maximum value of the grid width, is the maximum value of the grid height, is the minimum value of the grid height;

[0060] Figure 8 Schematic diagram of pseudo-crop row determination according to the present invention;

[0061] Figure 9 Schematic diagram of grid point cloud completion according to the present invention;

[0062] Figure 10 Effect diagram of fitting of soybean crop rows with low-density planting according to the present invention;

[0063] Figure 11 Effect diagram of fitting of soybean crop rows with medium-density planting according to the present invention;

[0064] Figure 12 Effect diagram of fitting of soybean crop rows with high-density planting according to the present invention;

[0065] Figure 13 Effect diagram of extraction of canopy points with low-density planting according to the present invention;

[0066] Figure 14 Effect diagram of extraction of canopy points with medium-density planting according to the present invention;

[0067] Figure 15 Effect diagram of extraction of canopy points with high-density planting according to the present invention;

[0068] Figure 16 Effect diagram of filling missing seedlings with low-density planting according to the present invention;

[0069] Figure 17 Effect diagram of filling missing seedlings and pseudo-crop determination with medium-density planting according to the present invention;

[0070] Figure 18 Effect diagram of filling missing seedlings and pseudo-crop determination with medium-density planting according to the present invention. Specific implementation method

[0071] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0072] Embodiment 1. Refer to Figure 1 This embodiment will be described. A method for detecting soybean crop rows at the maturity stage based on 3D LiDAR according to this embodiment includes:

[0073] Step S1: Collect soybean point cloud data and perform point cloud tilt correction on the soybean point cloud data;

[0074] Step S2: Perform segmented clustering on the corrected point cloud. The segmented clustering includes: using DBSCAN density clustering in the low-density area and Euclidean distance clustering in the high-density area;

[0075] Step S3: Extract the point cloud of the canopy of a single soybean plant after segmented clustering by the main stem method, including: locating the lowest point of the root system to construct a cylindrical area, and extracting the highest point on the Z-axis in this area as the canopy point cloud;

[0076] Step S4: Perform spatial division on the canopy point cloud through a dynamic grid division method, dynamically calculate the grid size according to the point cloud density, and establish a two-dimensional grid index;

[0077] Step S5: Perform missing seedling point cloud completion and pseudo-crop row determination according to the divided canopy point cloud grid, including: generating completed point cloud for areas where the point cloud missing rate in the column direction exceeds the threshold;

[0078] Step S6: Perform linear fitting on the completed canopy point cloud by the least squares method to generate the crop row center line, and the crop row center line is the recognition result of the crop area.

[0079] Embodiment 2. Refer to Figure 2 , Figure 3 and Figure 4 This embodiment will be described. This embodiment is a further limitation on a method for detecting soybean crop rows at the maturity stage based on 3D LiDAR described in Embodiment 1. Step S1 includes: performing Euler angle rotation matrix transformation on the tilted point cloud data collected by LiDAR to correct the tilted coordinate system to the horizontal ground coordinate system, specifically including:

[0080] Soybean point cloud data is collected using a lidar installed on agricultural machinery. The lidar makes an inclination angle of 15° with the ground, resulting in a certain inclination angle between the lidar and the Y-axis of the plane when generating soybean point cloud data. Therefore, the inclined point cloud needs to be rotated by Euler angles to align with the actual horizontal plane, and a point cloud matrix transformation is performed. As shown in the following formula, the Euler angles are multiplied in the inner rotation mode, where the Euler angles of the coordinate Z, Y, and X axes with the horizontal plane are γ, β, and α respectively. represents the point cloud matrix. The three-dimensional point cloud rotation matrix M is the product of three rotation matrices, where γ and α are 0°, and β is 15°:

[0081]

[0082] M zyx =

[0083] Figure 2 and Figure 3 is the visual comparison of the point cloud before and after rotation. It can be seen from Figure 3 that the lidar coincides with the ground horizontal coordinate axis after rotating the Euler angles.

[0084] According to the working principle of the lidar, the point cloud density in the front area is relatively high, and the point cloud density in the rear area is relatively low. Most of the point clouds in the rear area belong to the crop canopy with a small elevation difference, but the density difference between different areas is large, and the point cloud distribution of the crop rows has an obvious spatial structure. Therefore, the preprocessed point cloud data is segmented and clustered as Figure 4 shown. First, the point cloud is projected onto the XY-axis plane. The highest point and the lowest point are extracted according to the Y-axis coordinates of the points, and the middle value is taken as the threshold of the Y-axis spatial position, and the point cloud is divided into two parts, A and B, above and below the threshold. Since the point cloud in area B is relatively sparse, DBSCAN density clustering is used. Data is read from the point cloud dataset to construct a point cloud object containing point coordinates, and clustering parameters are set, including the clustering tolerance and the minimum cluster size MinPts. The neighborhood formula is as follows:

[0085]

[0086] where, if the number of points (including itself) included in the neighborhood of is at least Minpts, it is called a core point. If a point is not a core point but is within the neighborhood of a certain core point , then is a boundary point. If it is neither a core point nor a boundary point, it is a noise point.

[0087] The point cloud in area A is relatively dense, and Euclidean distance clustering is used. The distance judgment formula is as follows:

[0088]

[0089] Among them, 、 、 and 、 、 are the specific coordinates of points and respectively.

[0090] Embodiment 3. Refer to Figure 5 to illustrate this embodiment. This embodiment further limits the method for detecting soybean mature crop rows based on 3D LiDAR described in Embodiment 1. In step S3, the lowest point of the root system is located to construct a cylindrical region, and the highest point on the Z-axis in this region is extracted as the canopy point cloud, including:

[0091] Extract the point cloud at the bottom of the main root system of soybeans ;

[0092] Construct a cylindrical region with a radius R = 5 cm centered on the root system point cloud , and the height range is ;

[0093] Extract the highest point on the Z-axis in the cylindrical region as the canopy point cloud of the crop:

[0094]

[0095] Among them, is the point cloud set in the cylindrical region, is the crop canopy point, is the th Z-axis coordinate of the point.

[0096] Due to the characteristics of soybeans growing upright with many branches and leaves and having leaves at the top, directly extracting the canopy point cloud with a height threshold is likely to result in the extracted point cloud being concentrated around the leaves. Therefore, in this embodiment, the main stem method (MSM) is proposed to further extract the canopy center point cloud. Its core idea is to use the main stem part of the soybeans above the ground collected by the lidar as the bottom main stem range and extract the canopy points within the main stem range. As Figure 5 shown, first extract the point cloud at the bottom of the main root system of soybeans , and then take a circle with a radius of 5 cm centered on this point (this part covers the entire main stem and part of the branches of the soybean crop), and select the point cloud at all elevations within this circle to form a cylinder. The point cloud set is denoted as Find the point with the highest Z-axis in and mark it as the canopy point of this soybean crop. Use the following formula to extract the root points:

[0097]

[0098] where all the point sets within a cluster in the point cloud dataset are set as , represents the Z-axis coordinate of the th point.

[0099] With the root point as the center, a radius = 5 cm and a height range cm to construct a cylinder. For a certain point , the condition for determining whether it is inside the cylinder is as follows:

[0100]

[0101] The points that meet this condition are extracted into a new point set .

[0102] Find the highest point on the Z-axis in as the canopy point of this crop, and its formula is defined as follows:

[0103] .

[0104] Embodiment 4. Refer to Figure 6 and Figure 7 to illustrate this embodiment. This embodiment further limits a 3D LiDAR-based soybean maturity crop row detection method described in Embodiment 1. In step S4, dynamically calculating the grid size according to the point cloud density includes:

[0105] ,

[0106] ,

[0107] ,

[0108] where, is the area of a single grid, is the grid expectation value, is the area width, is the area height, is the total number of grids in the X-axis direction, is the total number of grids in the Y-axis direction, is the total size of the grid width, is the total size of the grid height.

[0109] Since the spacing of the canopy point clouds extracted at different planting densities of crops varies greatly, directly fitting by rows requires prior measurement parameters of the crop row spacing to be given, and the number of horizontal lines is adjusted according to different spacings. To achieve crop row detection at different planting densities of crops, this embodiment proposes a dynamic division (DD-CPM) of the canopy point cloud grid. As Figure 6 and Figure 7 shown, by processing the initial point cloud data, the crop contour is obtained. After extracting the canopy point cloud, the crop canopy row point cloud is processed to obtain the crop canopy feature points with accurate coordinates, and then the grid is dynamically divided. The specific steps are as follows:

[0110] First, project the extracted canopy point cloud onto the XY plane, remove the Z-axis information, ensure that the subsequent two-dimensional feature extraction focuses on the coordinate changes in the XY plane, and reduce the interference of noise. Traverse all two-dimensional plane point clouds, extract the minimum and maximum values of the X and Y axes, X min 、X max 、Y min 、Y max as the boundary values of the point cloud, and calculate the area of the region and the point density. The formula is as follows:

[0111] ,

[0112] ,

[0113]

[0114]

[0115]

[0116]

[0117] Among them, and are the width and height of the region respectively, is the area of the region, is the total number of point clouds, is the point density (number of points per unit area).

[0118] After determining the point density, a grid expected value is set in this embodiment (to ensure that there is only 1 to 2 points in each grid as much as possible), and the area of a single grid is calculated according to this expected value. Combining the region width and height , the number of grids was further calculated. The formula is as follows:

[0119]

[0120] ,

[0121] ,

[0122] Among them, is the area of a single grid, and are the total numbers of grids in the X-axis direction and Y-axis direction respectively, and boundary protection was carried out on the grids to prevent the division number from being zero.

[0123] After dividing the grids, a two-dimensional array was created to divide the point cloud into the corresponding grids according to the coordinate positions. The grid index of each point ( , ) was calculated through the following formula:

[0124] , .

[0125] Embodiment 5, see Figure 8 and Figure 9 to illustrate this embodiment. This embodiment is a further limitation on the method for detecting soybean mature crop rows based on 3D LiDAR described in Embodiment 4. The step S5 includes:

[0126]

[0127]

[0128] Among them, is the pseudo-crop row determination, is the total number of grids in the X-axis direction for filling the point cloud, is the total number of grids in the Y-axis direction for filling the point cloud, is to skip the navigation line generation for determining that this row is a pseudo-crop row, is to generate a navigation line for determining that this row is a soybean crop row, and y represent the coordinate axis values of the filled point cloud, is the total number of canopy point clouds in the column, is the width of a single grid, is the height of a single grid, is the minimum value of the grid width, is the minimum value of the grid height.

[0129] As Figure 8 shown, during the growth process of some crops, weeds grow simultaneously. In crop row detection, it is easy to classify them as pseudo-crop rows (crop rows in the same column). Moreover, according to the working characteristics of lidar, the point cloud in the distance is relatively sparse, small crops are likely to be missed, and there are phenomena such as missing seedlings and broken rows in some crop rows. To prevent misdetection in this situation, in this embodiment, the missing seedling canopy points are complemented and pseudo-crop rows are determined.

[0130] First, extract the point cloud in the same column of the Y-axis from the divided canopy point cloud grid, and count the total number of canopy point clouds in this column . Secondly, to ensure the accuracy of the missing canopy rows, determine whether is less than half of the maximum number of points in a single column of the original canopy point cloud, denoted as . If so, mark it as . If not, determine whether there is valid canopy point cloud information within a certain range for each column, and complement the missing grid point cloud column by column. Among them, the complemented point cloud is placed at the center point of the grid to ensure that it can become an effective predicted canopy point cloud as much as possible. The formula is as follows: where

[0131]

[0132]

[0133] where represents the determination of pseudo-crop rows, and respectively represent the grid numbers in the X-axis direction and Y-axis direction of the complemented point cloud, and y represent the coordinate axis values of the complemented point cloud.

[0134] Figure 9 In

[0135] the white wireframe represents the missing canopy row, the blue square represents the complemented canopy point cloud, the red square represents the canopy point cloud, and the white square represents the original point cloud. Figure 10 、 Figure 11 and Figure 12 illustrate this embodiment. This embodiment further limits a method for detecting soybean mature crop rows based on 3D LiDAR described in Embodiment 1. The step S6 includes:

[0136]

[0137] where is the slope of the straight line, is the intercept of the straight line, is the Y-axis coordinate of the canopy point cloud, represents the X-axis coordinate of the canopy point cloud.

[0138] Since crops are usually arranged in a straight line during sowing and management, in order to effectively capture the position of their centerlines, a linear model is selected for estimation in this embodiment. The core of the linear model is to optimize and solve the observed data through the Least Squares Method (LSM) to find the best linear fit, thereby detecting the centerlines of the canopy rows. When establishing this linear model, the Ceres Solver algorithm is used to solve the nonlinear least squares optimization problem with boundary constraints. The model is expressed as follows:

[0139]

[0140] where, and represent the slope and intercept of the straight line respectively, and represent the Y-axis coordinate and X-axis coordinate of the canopy points respectively.

[0141] The x and y coordinate values of the original canopy and the complemented canopy point clouds are used as the input of the least squares method to determine the optimal linear parameters. Through iterative solution, the actual ([[]]END]] and ) parameter values are finally obtained, and the centerlines of the canopy rows are generated. The obtained fitted centerlines are visualized and superimposed on the original crop row images to visually present the fitting accuracy and effect, facilitating further verification and analysis. Figure 10 、 Figure 11 and Figure 12 respectively show the fitting effects of soybean crop rows with low, medium, and high planting densities.

[0142] Embodiment Seven: This embodiment further limits the method for detecting soybean mature crop rows based on 3D LiDAR described in Embodiment One. The method further includes: establishing a point cloud processing time threshold model according to the coupling relationship between the lidar scanning frequency and the agricultural machinery driving speed to achieve dynamic adaptation of the processing delay and mechanical movement.

[0143] Embodiment Eight: This embodiment further limits the method for detecting soybean mature crop rows based on 3D LiDAR described in Embodiment Seven. The point cloud processing time threshold model includes:

[0144] Basic time constraint based on the scanning frequency :

[0145]

[0146] Among them, is the output frequency;

[0147] The maximum allowable number of delayed frames :

[0148]

[0149] Among them, is the effective soybean crop row scanning range, is the vehicle speed;

[0150] The maximum allowable processing time :

[0151]

[0152] Among them, is the safety factor.

[0153] Combined with Embodiment 7, this embodiment is described. During the operation of the soybean harvester, in order to avoid the control lag caused by the point cloud processing delay, it is necessary to ensure that within the processing time of each frame of point cloud, the vehicle does not travel more than the maximum distance. Taking the common operating speed of the soybean harvester in the farmland (4 km / h - 8 km / h) as the standard, in this embodiment, the Robosense-Helios32 lidar is configured, and the effective soybean crop row scanning range is 5 m, and the output frequency is 5 Hz. The basic time constraint based on the scanning frequency is:

[0154]

[0155] Using the delayed processing mechanism, the system can cache N frames of data before processing is completed. The maximum allowable number of delayed frames calculated based on the scanning range and the vehicle speed is:

[0156]

[0157] Considering the delayed processing mechanism, the maximum allowable processing time can be extended to:

[0158]

[0159] Among them, the safety factor is 0.8, which reserves time considering system delay, data transmission time, processing overhead, buffer management, etc.

[0160] Embodiment Nine. A computer device described in this embodiment includes a memory and a processor. A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for detecting soybean mature crop rows based on 3D LiDAR described in any one of Embodiments One to Eight.

[0161] Embodiment Ten. A computer-readable storage medium described in this embodiment has a computer program stored thereon. When the computer program is run by a processor, it executes the steps of a method for detecting soybean mature crop rows based on 3D LiDAR described in any one of Embodiments One to Eight.

[0162] Embodiment Eleven. Refer to Figure 13 , Figure 14 , Figure 15 , Figure 16 , Figure 17 and Figure 18 to illustrate this embodiment. This embodiment provides a specific example for a method for detecting soybean mature crop rows based on 3D LiDAR described in Embodiment One, and is also used to explain Embodiments Two to Eight. Specifically:

[0163] In this embodiment, the data processing platform uses an Intel Core i7-7700HQ core processor with a frequency of 2.80 GHz, a 64-bit operating system, and 8.00 GB of random access memory (RAM). The lidar communicates with the laptop through a network port. The algorithm is implemented in C++ language based on Ubuntu (16.04) under the Robot Operating System (ROS).

[0164] In this embodiment, a configured Robosense-Helios32 lidar is used and installed in the front of the agricultural machinery. The height of the agricultural machinery is 2.1 m, and the average driving speed can reach 2.3 m / s. The horizontal field of view angle of the lidar is 360°, the vertical field of view angle is 70°, the maximum detection distance is 200 m, the ranging random error is less than 2 cm, the angle random error is less than 0.05°, the number of beams is 32, the horizontal angle resolution is 0.1°, the vertical angle resolution is 1°, and the maximum point cloud output is 1,200,000 points / s. The installation position of the lidar should meet three requirements: (1) The soybean crops are in front of the lidar; (2) The lidar can acquire at least six rows of crops; (3) The height of the lidar can be adjusted according to the height of different planting densities of soybeans.

[0165] In this embodiment, the accuracy of detecting the center line of soybean crops is verified by using the artificial calibration soybean center line crop evaluation standard verification algorithm. The error angles (θ1, θ2, θ3, θ4, θ5, θ6) are defined as the absolute deviation angles between the center line extracted from left to right by the method proposed in the present invention and the artificial calibration. When θ is less than 5°, it is considered that the extraction of the corn row center line is successful; otherwise, it is considered that the extraction of the corn row center line fails. The average extraction accuracy and processing time of different corn rows are calculated by the following formula to test the accuracy and real-time performance of the method proposed in the present invention.

[0166]

[0167] In the formula, is the total number of soybean rows in the k-th frame, is the angle of the i-th fitted soybean row in the k-th frame, is the angle of the i-th manually marked soybean row in the K-th frame, is the total number of frames of the point cloud, is the processing time of the i-th frame, is the average extraction accuracy, is the average processing time.

[0168] In order to verify the effectiveness of the method proposed in the present invention in detecting the center line of soybean crop rows under different planting density conditions, soybean point clouds of three different planting treatments, LD (low density), MD (medium density), and HD (high density), are selected for processing. Among them, 55 frames of point cloud data are randomly selected from each treatment, a total of 165 frames, and the center lines of soybean rows are manually marked as the standard for the extraction algorithm. And the method proposed in the present invention is compared with the crop row detection models of two different clustering methods, Algorithm1 (crop row recognition based on horizontal strip dynamic clustering algorithm) and Algorithm2 (crop row recognition based on DBSCAN pure density clustering algorithm). The results are shown in Table 1. Among them, θ1AP is the average accuracy of the canopy navigation line of the first column of crop rows, θ2AP is the average accuracy of the canopy navigation line of the second column of crop rows, θ3AP is the average accuracy of the canopy navigation line of the third column of crop rows, θ4AP is the average accuracy of the canopy navigation line of the fourth column of crop rows, θ5AP is the average accuracy of the canopy navigation line of the fifth column of crop rows, θ6AP is the average accuracy of the canopy navigation line of the sixth column of crop rows, and θAP is the average accuracy of the overall crop row canopy navigation line.

[0169] Table 1 Comparison results of different algorithms

[0170]

[0171] As can be seen from Table 1, with the increase in the soybean planting density, the average accuracy of all three methods is continuously decreasing. This is because the greater the density, the more the soybeans obscure each other, affecting the extraction of the center line. However, with the increase in density, the average accuracy reduction of the soybean crop row detection of the present invention is 2.5% and 2.1% respectively. Compared with the average accuracy reduction of 4.4%, 6.2% and 3.2%, 3.1% of Algorithm1 and Algorithm2, it is reduced by 1.9%, 4.1% and 0.7%, 1%. And under the three different soybean planting density conditions of LD, MD, and HD, the average accuracy rates of the method proposed in the present invention are 93%, 90.5% and 88.4 respectively, which are 2.5%, 6.0%, 8.5% higher than Algorithm1 and 4.3%, 5.0%, 6.0% higher than Algorithm2. It can be seen that the soybean crop row detection algorithm of the present invention is more effective than the two comparative crop row detection methods. It should be noted that Algorithm1 uses the horizontal strip dynamic clustering algorithm and uses the least squares method to determine the center line of the crop row. The average accuracy in LD and MD is 1.8% and 0.6% higher than that of the DBSCAN pure density clustering algorithm of Algorithm2, but it is 2.5% lower in the HD planting area. This is because the pure density algorithm is composed of density point cloud data, resulting in a large deviation in the detection of crop row feature points under medium and low planting density conditions. The data in the center of the point cloud is denser, so the detection effect on the canopy center is lower than that of Algorithm1. Therefore, the invention adopts different clustering methods according to the characteristics of the point cloud data in segments to ensure that more confident feature points are obtained.

[0172] The processing time of a single frame of the method proposed in the present invention under the three soybean planting densities is 113ms, 121ms and 134ms respectively, which is higher than the other two comparative algorithms. This is because Algorithm1 and Algorithm2 use the elevation method to extract the canopy point cloud, reducing the processing time. However, in the present invention, in order to further improve the point cloud accuracy of the soybean canopy extraction, the main stem method is adopted to extract the canopy point cloud according to the growth characteristics of soybean crops. The initial segmentation and clustering of all soybean point cloud data lead to an increase in processing time. Although the inference speed of the proposed method does not reach the highest, when processing a single frame of point cloud, it is still less than the estimated maximum processing time of 1.6s. Therefore, the method proposed in the present invention is sufficient to meet the requirements of real-time applications.

[0173] For the main stem method (MSM) for extracting the canopy center proposed in the present invention, in order to explore its function and verify its effectiveness, the average angular difference between the crop rows fitted by 55 groups of single-frame six-row soybean crops and the real crop rows was analyzed, and the effect of the elevation method for extracting the canopy center point was compared. The obtained results are as Figure 13 、 Figure 14 and Figure 15 shown. As can be seen from the figure Figure 13The difference in the average canopy angle extracted by the main stem method and the elevation method in the low-density soybean area is within 2.8 (±0.7) and 3.2 (±0.5) respectively. It can be seen that when the soybean planting spacing is large and the plants are relatively scattered, the effect of extracting canopy points by the main stem method is slightly better than that of the elevation method. However, as the planting density increases, from Figure 14 and Figure 15 it can be seen that the differences in the average canopy angle extracted by the main stem method and the elevation method are within 3.2 (±0.6), 3.4 (±0.6) and 3.7 (±0.4), 4.6 (±0.5) respectively. The main stem method is affected by the increase in soybean planting density, and the leaf spacing of soybeans gradually decreases, resulting in a gradual decrease in the clustering effect, and multiple soybean plants are identified as the same plant. Its algorithm for finding the bottom point cloud of the main stem cannot accurately judge, resulting in a gradual increase in the average angle difference. However, compared with the elevation method, it can be seen that the average angle difference of the method proposed in the present invention is smaller and closer to the real crop row, while the latter has a larger range of angular deviation affected by the lateral width of the crop row.

[0174] From Figure 16 it can be seen that there are a total of 54 soybean crops determined by the current frame grid method, and the actual extracted canopy points are 25. Among them, 29 missing seedlings and pseudo-soybean canopy points are generated. And according to the grid method, when there are only missing seedlings in the current soybean crop row, only the missing seedling canopy point cloud will be complemented. The blue point cloud is the complemented canopy point cloud, and the white column is the pseudo-crop row. When there is a large gap in the planting interval of some soybean rows or inter-row planting, as shown in Figure 17 the second column and the fifth column, it can be seen that grid division is carried out on it, but no canopy point complementation is carried out and it is not considered in the subsequent crop row determination. However, when there are weeds in the determined area of the soybean crop row, there are a small number of original canopy point clouds in some columns on the grid. From Figure 18 it can be seen that the maximum effective number of rows of the original canopy point cloud in this frame is 18 plants in the fifth column, and the original canopy point cloud of the grid in the sixth column is 6 plants, which is less than half of the maximum effective number of rows in the current frame. It is determined that the current column is a pseudo-crop row.

[0175] The specific embodiments of the present invention disclosed above are only used to help explain the present invention. The specific embodiments do not describe all the details in detail, nor limit the invention to the specific embodiments described. According to the content of this specification, many modifications and changes can be made. These embodiments are selected and specifically described in this specification to better explain the principle and practical application of the present invention, so that those skilled in the art can understand and utilize the present invention well.

Claims

1. A method for detecting soybean mature crop rows based on 3D LiDAR, characterized in that, The method includes: Step S1: Collect soybean point cloud data and perform point cloud tilt correction on the soybean point cloud data; Step S2: Perform segmented clustering based on the corrected point cloud. The segmented clustering includes: using DBSCAN density clustering in the low-density area and Euclidean distance clustering in the high-density area; Step S3: Extract the canopy point cloud of individual soybean plants after segmented clustering by using the main stem method, including: locating the lowest point of the root system to construct a cylindrical region and extracting the highest point on the Z-axis within this region as the canopy point cloud; Step S4: Perform spatial partitioning on the canopy point cloud by using the dynamic grid partitioning method, dynamically calculate the grid size according to the point cloud density, and establish a two-dimensional grid index; Step S5: Complete the missing seedling point cloud and determine the pseudo crop rows based on the partitioned canopy point cloud grid, including: generating the completed point cloud for the area where the point cloud missing rate in the column direction exceeds the threshold; Step S6: Perform linear fitting on the completed canopy point cloud by using the least squares method to generate the crop row center line, and the crop row center line is the recognition result of the crop area.

2. The method for detecting soybean mature crop rows based on 3D LiDAR according to claim 1, wherein The said Step S1 includes: performing Euler angle rotation matrix transformation on the tilted point cloud data collected by LiDAR to correct the tilted coordinate system to the horizontal ground coordinate system.

3. A method for detecting soybean mature crop rows based on 3D LiDAR according to claim 1, characterized in that, In the said Step S3, locating the lowest point of the root system to construct a cylindrical region and extracting the highest point on the Z-axis within this region as the canopy point cloud includes: Extract the point cloud at the bottommost part of the main root system of soybeans ; Take the root point cloud as the center to construct a cylindrical region with a radius R = 5 cm, and the height range is ; Extracting the highest point on the Z-axis within the cylindrical region as the canopy point cloud of this crop: Among them, is the point cloud set within the cylindrical region, is the point cloud of the soybean canopy of this plant, is the Z-axis coordinate of the 4. A method for detecting soybean mature crop rows based on 3D LiDAR according to claim 1, characterized in that, In the said Step S4, dynamically calculating the grid size according to the point cloud density includes: ， ， ， Among them, is the area of a single grid, is the grid expected value, is the region width, is the region height, is the total number of grids in the X-axis direction, is the total number of grids in the Y-axis direction, is the total size of the grid width, is the total size of the grid height, is the point density.

5. The method for detecting soybean mature crop rows based on 3D LiDAR according to claim 4, characterized in that The said Step S5 includes: Among them, is the determination of pseudo-crop rows, is the number of grids in the X-axis direction for complementing the point cloud, is the number of grids in the Y-axis direction for complementing the point cloud, is to generate a navigation line by skipping the navigation line for the determined pseudo-crop row, is to generate a navigation line for the determined soybean crop row, x and y represent the coordinate axis values of the complemented point cloud, is the total number of canopy point clouds in the column, is the width of a single grid, is the height of a single grid, is the minimum value of the grid width, is the minimum value of the grid height.

6. A method for detecting soybean mature crop rows based on 3D LiDAR according to claim 1, characterized in that, The said Step S6 includes: Among them, is the slope of the straight line, is the intercept of the straight line, is the Y-axis coordinate of the canopy point cloud, represents the X-axis coordinate of the canopy point cloud.

7. A method for detecting soybean mature crop rows based on 3D LiDAR according to claim 1, characterized in that, The method further includes: establishing a point cloud processing time threshold model according to the coupling relationship between the LiDAR scanning frequency and the agricultural machinery driving speed to achieve dynamic adaptation of the processing delay and the mechanical movement.

8. A method for detecting soybean mature crop rows based on 3D LiDAR according to claim 7, characterized in that, The said point cloud processing time threshold model includes: Basic time constraint based on scanning frequency : Among them, is the output frequency; Maximum allowable number of delayed frames : Among them, is the effective soybean crop row scanning range, is the vehicle speed; Maximum allowable processing time : Among them, is the safety factor.

9. A computer device, characterized in that: Including a memory and a processor, where a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for detecting crop rows of soybean at the mature stage based on 3D LiDAR according to any one of claims 1 - 8.

10. A computer-readable storage medium, characterized in that, A computer program is stored on this computer-readable storage medium. When the computer program is run by the processor, it executes the steps of a method for detecting crop rows of soybean at the mature stage based on 3D LiDAR according to any one of claims 1 - 8.

Images