Soybean mature period crop row detection method based on 3D LiDAR

Abstract

The invention provides a soybean mature period crop row detection method based on 3D LiDAR, and belongs to the field of laser radar detection. The problems that an existing image method is greatly influenced by illumination, research on detection of soybean crop rows by a laser radar is less, and particularly, defects exist in the aspects of judgment of different planting densities, seedling leakage and missing rows are solved. The method comprises the following steps: collecting soybean point cloud data, and performing point cloud tilt correction; performing segmented clustering according to the corrected point cloud; single-plant soybean canopy point cloud is extracted through a stem method; performing space division on the canopy point cloud through a dynamic grid division method, dynamically calculating the grid size according to the point cloud density, and establishing a two-dimensional grid index; according to the divided canopy point cloud grids, carrying out seedling missing point cloud completion and pseudo crop row determination; and performing linear fitting on the complemented canopy point cloud by adopting a least square method, and generating a crop row center line as a crop region identification result. The method is mainly used in soybean detection field.

Description

Technical Field

[0001] The invention belongs to the field of laser radar detection, and in particular relates to a soybean crop row detection method at maturity based on 3D LiDAR. Background Art

[0002] As one of the world's most important food crops, soybeans have extensive economic and nutritional value. The total global planting area is about 120 million hectares all over the world, with an annual output of nearly 300 million tons. In the task of harvesting soybeans by agricultural machinery during the maturity period, one agricultural machine often needs to undertake the harvesting of soybeans with different planting densities in multiple fields. As a prerequisite for automatic navigation of agricultural machinery, crop row extraction currently has two main methods, namely image cameras and lidar.

[0003] Early work on crop row detection based on image cameras usually used the Hough transform (HT) algorithm to extract points in the image space and map them to the parameter space, and determined the parameter representation of the crop row navigation line through a cumulative voting mechanism. With the development of deep learning, it has been combined with image segmentation to have new applications in crop row detection, and is no longer limited to experimental farmland environments without interference. In 2020, some scholars proposed "Automatic Segmentation of Crop / Background Based on Luminance Partition Correction and Adaptive Threshold" (based on color index crop detection with brightness distinction and color saturation calibration method) to study the influence of outdoor lighting on green crops. Through the high-pitched classification otsu method, the grayscale images of weeds, duckweed, and rice were clustered to extract green rice feature points, and the crop rows were fitted using a two-dimensional adaptive clustering method. With the introduction of the concept of precision agriculture, in order to prevent pesticide waste and environmental pollution, the ResC-UNet network was improved. By performing contour search and minimum rectangle fitting on the converted image, the width was obtained while extracting the center line, and the actual row line was extracted based on the positional relationship between the width and the row line. A seedling and mid-growth image dataset was constructed, the YOLOv8 backbone network was replaced for crop row detection, the SuperGreen method was improved, the distinction between corn and farmland background was enhanced, and the newly proposed local-global detection method was used to identify corn crop rows. The process of this method first locates the local crop row, and then further identifies the global crop row. In summary, the crop row detection method based on image cameras is relatively mature, but due to the complex and changeable farmland lighting environment, the model adaptability based on image cameras is poor, and there is no systematic algorithm processing. LiDAR is superior to image cameras in detection range, resistance to rain and snow environment, and resistance to light and dark changes. Therefore, it is difficult for algorithms based on image cameras to detect crop rows in different complex environments.

[0004] As a measuring device that uses pulsed laser to illuminate the target and measure the return time of the reflected pulse to determine the target distance, LiDAR has higher accuracy in target detection because it is insensitive to changes in shadows and lighting conditions. With the advancement of technology, low-cost LiDAR has gradually become a highly-regarded agricultural application technology. In the early days, LiDAR combined with navigation lines was usually used in fruit trees. From the extracted orchard structure and terrain change features, the nearest neighbor pattern gradient was used to find the row direction and thus the center line. With the improvement of equipment accuracy, when the ground crops cover each other, the corn crops are divided into horizontal strips, and the Euclidean clustering algorithm is improved to use the candidate point information to filter and extract feature points according to the corresponding threshold. Considering the morphological changes of the same crop at different times, some scholars have proposed a dynamic horizontal partition clustering method to determine the crop canopy feature points. For the autonomous navigation of agricultural robots in greenhouse crop vineyards, an online regression model of the structure is constructed by applying Hough transform and parameterization and search methods.

[0005] From the above work, it can be seen that the current crop row detection method based on LiDAR has formed a detection pipeline of segmentation, noise reduction, canopy point extraction and line fitting. However, there are few corresponding studies on crops such as soybeans, and the canopy center point has not been extracted based on the growth characteristics of soybean crops. There is a lack of crop row detection methods suitable for different densities of such crops, as well as the determination of missed and missing crop rows. Most of the 3-D structural feature extraction methods for crops such as soybeans are based on single density features, so the soybean row detection algorithm based on LiDAR needs to be improved. Summary of the invention

[0006] In view of this, the present invention aims to propose a soybean crop row detection method at maturity based on 3D LiDAR to solve the problem that the existing image method is greatly affected by light, while there is little research on lidar in the detection of soybean crop rows, especially in the judgment of different planting densities, missing seedlings and missing rows.

[0007] To achieve the above object, the present invention adopts the following technical solution: a method for detecting soybean crop rows at maturity based on 3D LiDAR, the method comprising: Step S1: collecting soybean point cloud data, and performing point cloud tilt correction on the soybean point cloud data; Step S2: performing segmented clustering according to the corrected point cloud, wherein the segmented clustering includes: using DBSCAN density clustering in low-density areas and using Euclidean distance clustering in high-density areas; Step S3: extracting the canopy point cloud of a single soybean plant after segmented clustering using the main stem method, including: locating the lowest point of the root system to construct a cylindrical area, and extracting the highest point on the Z axis in the area as the canopy point cloud; Step S4: spatially divide the canopy point cloud by a dynamic grid division method, dynamically calculate the grid size according to the point cloud density, and establish a two-dimensional grid index; Step S5: completing missing seedling point clouds and determining pseudo crop rows according to the divided canopy point cloud grids, including: generating a completed point cloud for an area where the missing point cloud rate in the column direction exceeds a threshold; Step S6: Use the least square method to perform linear fitting on the completed canopy point cloud to generate the crop row center line, which is the crop area recognition result.

[0008] Furthermore, a preferred method is proposed, wherein step S1 comprises: performing Euler angle rotation matrix transformation on the inclined point cloud data collected by LiDAR, and correcting the inclined coordinate system to a horizontal ground coordinate system.

[0009] Furthermore, a preferred method is proposed, in step S3, the lowest point of the root system is located to construct a cylindrical area, and the highest point of the Z axis in the area is extracted as the canopy point cloud, including: Extract the bottom point cloud of soybean trunk root system; Root point cloud Construct a cylindrical area with a radius of R = 5cm as the center, and the height range ; Extract the highest point on the Z axis in the cylindrical area as the canopy point cloud of the crop:

[0010] in, is the point cloud set in the cylindrical area, is the soybean canopy point cloud. For the The Z coordinate of a point.

[0011] Furthermore, a preferred method is proposed, in which the grid size is dynamically calculated according to the point cloud density in step S4, including: , ,

[0012] ,

[0013] in, is the area of ​​a single grid, is the grid expected value, is the area width, is the area height, is the total number of grids in the X-axis direction, is the total number of grids in the Y-axis direction, is the total size of the grid width, The total size of the grid height.

[0014] Furthermore, a preferred embodiment is proposed, wherein step S5 comprises:

[0015]

[0016] in, For pseudo crop line determination, To complete the number of grids in the X-axis direction of the point cloud, To complete the number of grids in the Y-axis direction of the point cloud, To determine that the row is a pseudo crop row, skip the navigation line generation. Generate a guide line to determine if this row is a soybean crop row. and y represent the coordinate axis values ​​of the completed point cloud. for The total number of canopy point clouds in the column, is the width of a single grid, is the height of a single grid, is the minimum grid width, The minimum grid height.

[0017] Furthermore, a preferred embodiment is proposed, wherein step S6 comprises:

[0018] in, is the slope of the straight line, is the intercept of the straight line, is the Y-axis coordinate of the canopy point cloud, Represents the X-axis coordinate of the canopy point cloud.

[0019] Furthermore, a preferred embodiment is proposed, wherein the method further comprises: establishing a point cloud processing time threshold model according to the coupling relationship between the laser radar scanning frequency and the agricultural machinery driving speed, so as to realize dynamic adaptation of processing delay and mechanical movement.

[0020] Furthermore, a preferred method is proposed, wherein the point cloud processing time threshold model comprises: Basic timing constraints based on scan frequency :

[0021] in, is the output frequency; Maximum allowed delay frames :

[0022] in, For effective soybean crop row scanning range, is the vehicle speed; Maximum allowed processing time :

[0023] in, is the safety factor.

[0024] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a soybean mature crop row detection method based on 3D LiDAR as described in any one of the above items.

[0025] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a soybean mature crop row detection method based on 3D LiDAR as described in any one of the above are executed.

[0026] Compared with the prior art, the present invention has the following beneficial effects: The existing LiDAR method uses the height threshold method to extract canopy points, but soybean plants have dense branches and leaves, and the top leaves are easily misjudged as canopy points, resulting in feature point offset. The present invention pioneered the main stem method (MSM), combined with the biological characteristics of soybean upright growth, and proposed root lowest point positioning and Z-axis highest point screening. The data in the embodiment show that the canopy point positioning error is reduced by 62.5% under LD density (AP reaches 93%, an increase of 4.3% over the comparison algorithm), and the spatial distribution of feature points conforms to the growth law of soybean main stem.

[0027] The existing algorithm cannot adapt to the difference in point cloud distribution of different planting densities, resulting in high-density occlusion and low-density line breaks. The present invention proposes a collaborative solution of segmented clustering and dynamic mesh division. Through Y-axis segmented clustering, the clustering failure problem caused by density mutation is solved (AP reaches 88.4% at HD density in the embodiment, which is 6% higher than the traditional DBSCAN); the grid size is dynamically calculated based on the point cloud density to achieve adaptive matching of mesh division and planting density.

[0028] In order to solve the problem of false detection caused by missing seedlings and weed interference, the present invention designs a dual-threshold judgment mechanism to fill in the missing seedlings, and the filling rate is improved by 37% (the broken row repair rate under HD density in the embodiment is 92%); and pseudo crop rows are filtered to eliminate weed interference, and the false detection rate is reduced to 5.2% (a decrease of 8.3% compared with the traditional method).

[0029] Existing algorithms do not consider the matching problem between agricultural machinery movement and data processing delay, resulting in navigation lag. This invention establishes a point cloud processing time threshold model for the first time, and avoids navigation interruption caused by single frame processing timeout through dynamic delay control and multi-frame buffer mechanism.

[0030] Furthermore, the traditional view is that crop row detection should prioritize the universality of the algorithm, but the present invention specifically designs the main stem method and dynamic grid for soybeans, achieving unexpected results. Although the main stem method increases the amount of calculation by 18%, the total time consumption is only increased by 3% through segmented clustering preprocessing; the dynamic grid division breaks the industry practice of fixed grids, reducing the detection stability of different density scenes from 0.18 to 0.09. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings: Figure 1 This is a flow chart of a method for detecting soybean crop rows at maturity based on 3D LiDAR according to the present invention; Figure 2 This is a visualization comparison diagram of the point cloud before rotation described in the present invention; Figure 3 This is a visualization comparison diagram of the rotated point cloud described in the present invention; Figure 4 The soybean crop point cloud segmentation clustering diagram of the present invention; Figure 5 A comparison diagram of canopy point cloud extracted by the height threshold and root system method described in the present invention; Figure 6 Schematic diagram of dynamic division of canopy point cloud grid method according to the present invention, in which Xmax, Xmin, Ymin, and Ymax are boundary values ​​of point cloud; is the width of the region, is the height of the region; Figure 7 This is a schematic diagram of the soybean canopy grid division result of the present invention; wherein, is the minimum grid width, is the maximum grid width, is the maximum grid height, is the minimum grid height; Figure 8 This is a schematic diagram of pseudo crop row determination according to the present invention; Fig. 9 This is a schematic diagram of grid point cloud completion according to the present invention; Fig.10This is a fitting effect diagram of a soybean crop row planted at a low density according to the present invention; Fig.11 This is a fitting effect diagram of a soybean crop row planted at a medium density according to the present invention; Fig.12 This is a fitting effect diagram of a soybean crop row planted at a high density according to the present invention; Fig.13 This is the effect diagram of the low-density planting canopy point extraction described in the present invention; Fig.14 This is the effect diagram of the medium-density planting canopy point extraction described in the present invention; Fig.15 This is the high-density planting canopy point extraction effect diagram of the present invention; Fig.16 This is a diagram showing the effect of supplementing missing seedlings in low-density planting according to the present invention; Fig.17 This is a diagram showing the effect of filling in missing seedlings and determining pseudo crops in medium-density planting according to the present invention; Fig.18 This is a diagram showing the effect of filling missing seedlings and determining pseudo crops in medium-density planting according to the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely explain the technical solutions in the embodiments of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict, and the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0033] Implementation method 1, see Figure 1 This embodiment describes a method for detecting soybean crop rows at maturity based on 3D LiDAR, the method comprising: Step S1: collecting soybean point cloud data, and performing point cloud tilt correction on the soybean point cloud data; Step S2: performing segmented clustering according to the corrected point cloud, wherein the segmented clustering includes: using DBSCAN density clustering in low-density areas and using Euclidean distance clustering in high-density areas; Step S3: extracting the canopy point cloud of a single soybean plant after segmented clustering using the main stem method, including: locating the lowest point of the root system to construct a cylindrical area, and extracting the highest point on the Z axis in the area as the canopy point cloud; Step S4: spatially divide the canopy point cloud by a dynamic grid division method, dynamically calculate the grid size according to the point cloud density, and establish a two-dimensional grid index; Step S5: completing missing seedling point clouds and determining pseudo crop rows according to the divided canopy point cloud grids, including: generating a completed point cloud for an area where the missing point cloud rate in the column direction exceeds a threshold; Step S6: Use the least square method to perform linear fitting on the completed canopy point cloud to generate the crop row center line, which is the crop area recognition result.

[0034] Implementation Method 2: See Figure 2 , Figure 3 and Figure 4 This embodiment is a further limitation of the soybean mature crop row detection method based on 3D LiDAR described in Embodiment 1, wherein step S1 comprises: performing Euler angle rotation matrix transformation on the inclined point cloud data collected by LiDAR to correct the inclined coordinate system to the horizontal ground coordinate system, specifically comprising: The soybean point cloud data was collected using a laser radar installed on an agricultural machine. The laser radar was tilted at an angle of 15° to the ground, which resulted in a certain tilt angle with the Y axis of the plane when the laser radar generated the soybean point cloud data. Therefore, the tilted point cloud needs to be rotated by Euler angles to align with the actual horizontal plane and perform point cloud matrix transformation. As shown in the following formula, the Euler angles are multiplied inwardly, where the coordinates Z, Y, and X axes are γ, β, and α with the horizontal plane Euler angles respectively. Represents the point cloud matrix. The three-dimensional point cloud rotation matrix M is the product of three rotation matrices, where γ and α are 0° and β is 15°:

[0035] M zyx =

[0036] Figure 2 and Figure 3 For visual comparison of point clouds before and after rotation, Figure 3 It can be seen that the laser radar coincides with the horizontal coordinate axis of the ground after rotating the Euler angle.

[0037] According to the working principle of LiDAR, the point cloud density in the front area is higher, and the point cloud density in the rear area is lower. The point clouds in the rear area mostly belong to the crop canopy with a small elevation difference, but the density difference between different areas is large, and the point cloud distribution of crop rows has an obvious spatial structure. Therefore, the pre-processed point cloud data is segmented and clustered as follows: Figure 4 As shown in the figure, first project the point cloud onto the XY axis plane, extract the highest and lowest points according to the Y axis coordinates of the points, take the middle value as the threshold of the Y axis spatial position, and divide the point cloud into two parts, A and B, above and below the threshold. Since the point cloud in area B is relatively sparse, DBSCAN density clustering is used to read data from the point cloud dataset and construct a point cloud object containing the point coordinates. , set clustering parameters, including clustering tolerance and the minimum cluster size MinPts, field The formula is as follows:

[0038] Among them, if of The number of points included in the field (including itself) is at least Minpts, it is called a core point. Not a core point, but in a core point of In the field, It is a boundary point. If it is neither a core point nor a boundary point, it is a noise point.

[0039] The point cloud in area A is relatively dense and uses Euclidean distance clustering. The distance determination formula is as follows:

[0040] in, , , and , , Points and The specific coordinates of .

[0041] Implementation method three, see Figure 5 This embodiment further defines the soybean mature crop row detection method based on 3D LiDAR described in Embodiment 1, wherein the lowest point of the root system is located in step S3 to construct a cylindrical area, and the highest point of the Z axis in the area is extracted as the canopy point cloud, including: Extract the bottom point cloud of soybean trunk root system ; Root point cloud Construct a cylindrical area with a radius of R = 5cm as the center, and the height range ; Extract the highest point on the Z axis in the cylindrical area as the canopy point cloud of the crop:

[0042] in, is the point cloud set in the cylindrical area, is the crop canopy point, For the The Z coordinate of a point.

[0043] Since soybeans have many upright branches and leaves and have leaves on the top, it is easy for the extracted point clouds to be concentrated around the leaves when directly extracting the canopy point cloud using the height threshold. Therefore, this implementation proposes to further extract the canopy center point cloud using the main stem method (MSM). The core idea is to use the main stem part of the soybean root system above the ground collected by the lidar as the main stem range at the bottom, and extract the canopy points within the main stem range. Figure 5 As shown, first extract the bottom point cloud of the soybean trunk root system , and then use this point as the center radius Take a 5 cm circle (this part covers the entire trunk of the soybean crop and some branches), select all the point clouds of the elevation within the circle to form a cylinder, and the point cloud collection is recorded as .exist Find the highest point on the Z axis and mark it as the canopy point of the soybean crop . Use the following formula to extract the root points:

[0044] Among them, all the point sets in a cluster in the point cloud dataset are set as , Indicates The Z coordinate of a point.

[0045] Root point Center, radius =5cm and height range cm to construct a cylinder. For a point , the conditions for judging whether it is inside the cylinder are as follows:

[0046] Points that meet this condition are extracted to a new point set middle.

[0047] exist Find the highest point on the Z axis as the canopy point of the crop, and its formula is defined as follows: .

[0048] Implementation method 4: See Figure 6 and Figure 7 This embodiment further defines the soybean mature crop row detection method based on 3D LiDAR described in Embodiment 1, and the step S4 dynamically calculates the grid size according to the point cloud density, including: , ,

[0049] ,

[0050] in, is the area of ​​a single grid, is the grid expected value, is the area width, is the area height, is the total number of grids in the X-axis direction, is the total number of grids in the Y-axis direction, is the total size of the grid width, The total size of the grid height.

[0051] Since the spacing of canopy point clouds extracted from different crop planting densities varies greatly, direct row fitting requires the actual measurement parameters of crop row spacing to be given in advance, and the number of horizontal lines to be adjusted according to different spacings. In order to achieve crop row detection at different crop planting densities, this implementation method proposes a dynamic division of canopy point cloud grids (DD-CPM). Figure 6 and Figure 7 As shown in the figure, the crop outline is obtained by processing the initial point cloud data. After extracting the canopy point cloud, the crop canopy line point cloud is processed to obtain the crop canopy feature points with precise coordinates, and then dynamically divide the grid. The specific steps are as follows: First, the extracted canopy point cloud is projected onto the XY plane, and the Z-axis information is removed to ensure that the subsequent 2D feature extraction focuses on the coordinate changes of the XY plane and reduces noise interference. min , X max , Y min , Y max As the boundary value of the point cloud, the area and point density are calculated. The formula is as follows: ,

[0052] ,

[0053]

[0054]

[0055]

[0056]

[0057] in, and are the width and height of the region, is the area of ​​the region, is the total number of point clouds, is the point density (number of points per unit area).

[0058] After determining the point density, the grid expected value is set in this embodiment (Try to ensure that there are only 1 to 2 points in each grid) and calculate the area of ​​a single grid based on this expected value. Combined with the area width and height , and further calculated the number of grids. The formula is as follows:

[0059] ,

[0060] ,

[0061] in, is the area of ​​a single grid, and They are the total number of grids in the X-axis direction and the Y-axis direction respectively, and in order to prevent the number of divisions from being zero, the grid boundaries are protected.

[0062] After dividing the grid, create a two-dimensional array to divide the point cloud into corresponding grids according to the coordinate position. The grid index ( , ), calculated by the following formula: , .

[0063] Implementation method 5: See Figure 8 and Fig. 9 This embodiment further defines the soybean mature crop row detection method based on 3D LiDAR described in Embodiment 4, and the step S5 includes:

[0064]

[0065] in, For pseudo crop line determination, To complete the number of grids in the X-axis direction of the point cloud, To complete the number of grids in the Y-axis direction of the point cloud, To determine that the row is a pseudo crop row, skip the navigation line generation. Generate a guide line to determine if this row is a soybean crop row. and y represent the coordinate axis values ​​of the completed point cloud. for The total number of canopy point clouds in the column, is the width of a single grid, is the height of a single grid, is the minimum grid width, The minimum grid height.

[0066] like Figure 8 As shown, some crops grow with weeds during their growth process, and are easily classified as pseudo crop rows (crop rows in the same column) during crop row detection. In addition, due to the working characteristics of the laser radar, small crops are easily missed due to the sparse point cloud in the distance, and some crop rows are prone to missing seedlings, broken rows, etc. To prevent such misdetection, the present embodiment determines the missing seedling canopy point completion and pseudo crop rows.

[0067] First, extract the point cloud in the same column of the Y axis from the divided canopy point cloud grid and count the number of points in the column. Total number of canopy point clouds , and secondly, to ensure the accuracy of missing canopy rows, determine Is it less than half of the maximum number of columns in the original canopy point cloud? If yes, mark it as If not, then judge the current Every other paragraph Whether the range contains valid canopy point cloud information, the missing grid point cloud is completed by column, and the completed point cloud is placed at the center of the grid to ensure that it is as effective as possible to predict the canopy point cloud. The formula is as follows:

[0068]

[0069] in, represents pseudo crop row determination, and Respectively represent the number of grids in the X-axis and Y-axis directions of the completed point cloud, and y represent the coordinate axis values ​​of the completed point cloud.

[0070] Fig. 9 The white wireframe in the middle represents the missing canopy row, the blue square represents the canopy point cloud completion, the red square represents the canopy point cloud, and the white square represents the original point cloud.

[0071] Implementation Method 6: See Fig.10 , Fig.11 and Fig.12 This embodiment further defines the soybean mature crop row detection method based on 3D LiDAR described in Embodiment 1, and the step S6 includes:

[0072] in, is the slope of the straight line, is the intercept of the straight line, is the Y-axis coordinate of the canopy point cloud, Represents the X-axis coordinate of the canopy point cloud.

[0073] Since crops are usually arranged in straight lines during sowing and management, in order to effectively capture their centerline positions, a linear model is chosen in this implementation to estimate. The core of the linear model is to optimize and solve the observed data through the least squares method (LSM) to find the best straight line fit, thereby detecting the centerline of the canopy row. When establishing this linear model, the Ceres Solver algorithm is used to solve the nonlinear least squares optimization problem with boundary constraints. The model is expressed as follows:

[0074] in, and represent the slope and intercept of the straight line, respectively. and Represent the Y-axis coordinate and X-axis coordinate of the canopy point respectively.

[0075] The x and y coordinates of the original canopy and the completed canopy point cloud are used as the input of the least squares method to determine the optimal linear parameters. Through iterative solution, the actual ( and ) parameter values ​​and generated the centerline of the canopy row. The obtained fitted centerline was visualized and superimposed on the original crop row image to intuitively present the fitting accuracy and effect, which is convenient for further verification and analysis. Fig.10 , Fig.11 and Fig.12 The fitting results of soybean crop rows at three different planting densities, low, medium and high, are shown respectively.

[0076] Implementation method seven. This implementation method is a further limitation of the soybean crop row detection method based on 3D LiDAR in the mature stage described in implementation method one. The method also includes: establishing a point cloud processing time threshold model based on the coupling relationship between the laser radar scanning frequency and the agricultural machinery driving speed, so as to realize dynamic adaptation of processing delay and mechanical movement.

[0077] Embodiment 8: This embodiment further limits the soybean crop row detection method based on 3D LiDAR in the mature stage described in Embodiment 7. The point cloud processing time threshold model includes: Basic timing constraints based on scan frequency :

[0078] in, is the output frequency; Maximum allowed delay frames :

[0079] in, For effective soybean crop row scanning range, is the vehicle speed; Maximum allowed processing time :

[0080] in, is the safety factor.

[0081] In combination with Implementation Example 7, this implementation example is explained. During the driving process of the soybean harvester, in order to avoid control lag caused by point cloud processing delay, it is necessary to ensure that the maximum distance will not be exceeded within the processing time of each frame of point cloud. Taking the common driving speed of soybean harvesters in farmland (4km / h-8km / h) as the standard, this implementation example is equipped with Robosense-Helios32 laser radar, and the effective soybean crop row scanning range is 5m, output frequency The basic time constraint based on the scanning frequency is:

[0082] Using the delayed processing mechanism, the system can cache N frames of data before processing is completed. The maximum allowed delayed frames are calculated based on the scanning range and vehicle speed. for:

[0083] Considering the delayed processing mechanism, the maximum allowed processing time Can be extended to:

[0084] in, The safety factor is 0.8, which takes into account the time reserved for system delays, data transmission time, processing overhead, buffer management, etc.

[0085] Embodiment 9. A computer device described in this embodiment includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a soybean mature crop row detection method based on 3D LiDAR described in any one of embodiments 1 to 8.

[0086] Embodiment 10. A computer-readable storage medium described in this embodiment stores a computer program, and when the computer program is executed by a processor, the steps of a method for detecting soybean crop rows at maturity based on 3D LiDAR are executed as described in any one of embodiments 1 to 8.

[0087] Implementation Method 11: See Fig.13 , Fig.14 , Fig.15 , Fig.16 , Fig.17 and Fig.18 This embodiment provides a specific example of the soybean mature crop row detection method based on 3D LiDAR described in the first embodiment, and is also used to explain the second to eighth embodiments, specifically: The data processing platform in this implementation adopts an Intel Core i7-7700HQ core processor with a frequency of 2.80GHz, a 64-bit operating system, and 8.00GB random access memory (RAM). The laser radar communicates with the laptop through a network port. The implementation of the algorithm is developed in C++ language based on Ubuntu (16.04) under the Robot Operating System (ROS).

[0088] In this embodiment, a Robosense-Helios32 laser radar is used and installed in front of the agricultural machinery, where the height of the agricultural machinery is 2.1m and the average driving speed can reach 2.3m / s. The horizontal field of view of the laser radar is 360°, the vertical field of view is 70°, the maximum detection distance is 200m, the ranging random error is less than 2cm, the angle random error is less than 0.05°, the beam is 32, the horizontal angular resolution is 0.1°, the vertical angular resolution is 1°, and the maximum point cloud output is 1200,000 points / s. The installation position of the laser radar should meet three requirements: (1) the soybean crop is in front of the laser radar; (2) the laser radar can obtain at least six rows of crops; (3) the height of the laser radar can be adjusted according to the height of different soybean planting densities.

[0089] In this embodiment, the soybean centerline crop evaluation standard verification algorithm is used to detect the accuracy of the soybean crop row centerline. The error angle (θ1, θ2, θ3, θ4, θ5, θ6) is defined as the absolute deviation angle between the centerline extracted from left to right by the method proposed in the present invention and the manual calibration. When θ is less than 5°, the corn row centerline extraction is considered successful; otherwise, the corn row centerline extraction is considered to have failed. The average extraction accuracy and processing time of different corn rows are calculated using the following formula to verify the accuracy and real-time performance of the method proposed in the present invention.

[0090]

[0091] In the formula, is the total number of soybean rows in the kth frame, is the angle of the i-th fitted soybean row in the k-th frame, The angle of the soybean row for the i-th manually labeled soybean row in the K-th frame, is the total number of point cloud frames, is the processing time of the i-th frame, is the average extraction accuracy, is the average processing time.

[0092] In order to verify the effectiveness of the method proposed in the present invention in detecting the center line of soybean crop rows under different planting density conditions, soybean point clouds with three different planting treatments, LD (low density), MD (medium density), and HD (high density), were selected for processing. 55 frames of point cloud data were randomly selected from each treatment, totaling 165 frames, and the center lines of soybean rows were manually annotated as the standard of the extraction algorithm. The method proposed in the present invention is compared with the crop row detection models of two different clustering methods, Algorithm 1 (crop row identification based on horizontal strip dynamic clustering algorithm) and Algorithm 2 (crop row identification based on DBSCAN pure density clustering algorithm). The results are shown in Table 1, where θ1AP is the average accuracy of the canopy navigation line of the first column of crop rows, θ2AP is the average accuracy of the canopy navigation line of the second column of crop rows, θ3AP is the average accuracy of the canopy navigation line of the third column of crop rows, θ4AP is the average accuracy of the canopy navigation line of the fourth column of crop rows, θ5AP is the average accuracy of the canopy navigation line of the fifth column of crop rows, θ6AP is the average accuracy of the canopy navigation line of the sixth column of crop rows, and θAP is the average accuracy of the canopy navigation line of the overall crop rows.

[0093] Table 1 Comparison results of different algorithms

[0094] As can be seen from Table 1, with the increase of soybean planting density, the average accuracy of the three methods is constantly decreasing. This is because the greater the density, the mutual shielding between soybeans affects the centerline extraction. However, with the increase of density, the average accuracy of the soybean crop row detection of the present invention is reduced by 2.5% and 2.1%, respectively, compared with the average accuracy of Algorithm1 and Algorithm2, which are reduced by 4.4%, 6.2% and 3.2%, 3.1%, respectively, which is reduced by 1.9%, 4.1% and 0.7%, 1%. And under the three different soybean planting density conditions of LD, MD, and HD, the average accuracy of the method proposed by the present invention is 93%, 90.5%, and 88.4, respectively, which is 2.5%, 6.0%, and 8.5% higher than Algorithm1, and 4.3%, 5.0%, and 6.0% higher than Algorithm2. It can be seen that the soybean crop row detection algorithm of the present invention is more effective than the two contrasting crop row detection methods. It is worth noting that Algorithm 1 uses a horizontal strip dynamic clustering algorithm and uses the least squares method to determine the average accuracy of the center line of the crop row in LD and MD, which is 1.8% and 0.6% higher than the DBSCAN pure density clustering algorithm of Algorithm 2, but 2.5% lower in the HD planting area. This is because the pure density algorithm is composed of density point cloud data, which leads to a large deviation in the detection of crop row feature points under medium and low planting density conditions. The data in the center of the point cloud is denser, so the detection effect of the canopy center is lower than that of Algorithm 1. Therefore, the invention adopts different clustering methods according to the feature segmentation of the point cloud data to ensure that feature points with higher confidence are obtained.

[0095] The single-frame processing time of the method proposed in the present invention at three soybean planting densities is 113ms, 121ms and 134ms respectively, which is higher than the other two comparison algorithms. This is because Algorithm1 and Algorithm2 use the elevation method to extract the canopy point cloud, reducing the processing time. However, in order to further improve the point cloud accuracy of soybean canopy extraction in the present invention, the main stem method is used to extract the canopy point cloud according to the growth characteristics of soybean crops. The initial segmentation and clustering of all soybean point cloud data leads to an increase in processing time. Although the reasoning speed of the proposed method does not reach the highest, it is still less than the estimated maximum processing time of 1.6s when processing a single-frame point cloud. Therefore, the method proposed in the present invention is sufficient to meet the requirements of real-time applications.

[0096] In order to explore the function and verify the effectiveness of the main stem method for extracting the canopy center (MSM) proposed in this paper, the average angle difference between the crop rows fitted by 55 groups of six-row soybean crops in a single frame and the real crop rows was analyzed, and the effect of extracting the canopy center point by the elevation method was compared. The results are shown in the figure. Fig.13 , Fig.14 and Fig.15 As shown in the figure, Fig.13The average canopy angle differences between the main stem method and the elevation method in the soybean low-density area are within the range of 2.8 (±0.7) and 3.2 (±0.5), respectively. It can be seen that when the soybean planting spacing is large and the plants are more scattered, the main stem method is slightly better than the elevation method in extracting canopy points. However, as the planting density increases, Fig.14 and Fig.15 It can be seen that the average angle difference of the canopy extracted by the main stem method and the elevation method is within the range of 3.2 (±0.6), 3.4 (±0.6), 3.7 (±0.4), 4.6 (±0.5). The main stem method is affected by the increase in soybean planting density, and the distance between soybean leaves gradually decreases, and the clustering effect gradually decreases, resulting in multiple soybean plants being identified as the same plant. The algorithm for finding the point cloud at the bottom of the main stem cannot accurately judge, resulting in the average angle difference gradually increasing. However, compared with the elevation method, it can be seen that the method proposed in the present invention has a smaller average angle difference and is closer to the actual crop row, while the latter is affected by the lateral width of the crop row. The angle deviation range is larger.

[0097] from Fig.16 It can be seen that the current frame grid method determines that there are 54 soybean crops in total, and the actual canopy points extracted are 25, of which 29 are missing seedlings and pseudo soybean canopy points. In addition, according to the grid method, when the current soybean crop row is determined to have only missing seedlings, only the missing seedling canopy point cloud will be completed. The blue point cloud is the completed canopy point cloud, and the white column is the pseudo crop row. When the soybean row has a large gap in the planting interval or is planted in alternate rows, as shown in the figure below, the missing seedlings and pseudo soybean canopy points are generated. Fig.17 As shown in the second and fifth columns of the , it can be seen that the grid is divided but the canopy point completion is not performed and is not considered in the subsequent crop row determination. However, when weeds appear in the soybean crop row determination area, a small amount of original canopy point cloud appears in some columns of the grid. Fig.18 It can be seen that the maximum number of valid rows of the original canopy point cloud in this frame is 18 plants in the 5th column, and the original canopy point cloud in the 6th column is 6 plants, which is less than half of the maximum number of valid rows in the current frame. The column is determined to be a pseudo crop row.

[0098] The specific embodiments of the present invention disclosed above are only used to help explain the present invention. The specific embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. According to the content of this specification, many modifications and changes can be made. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.

Claims

1. A method for detecting soybean crop rows at maturity based on 3D LiDAR, characterized in that: The method comprises: Step S1: collecting soybean point cloud data, and performing point cloud tilt correction on the soybean point cloud data; Step S2: performing segmented clustering according to the corrected point cloud, wherein the segmented clustering includes: using DBSCAN density clustering in low-density areas and using Euclidean distance clustering in high-density areas; Step S3: extracting the canopy point cloud of a single soybean plant after segmented clustering using the main stem method, including: locating the lowest point of the root system to construct a cylindrical area, and extracting the highest point on the Z axis in the area as the canopy point cloud; Step S4: spatially divide the canopy point cloud by a dynamic grid division method, dynamically calculate the grid size according to the point cloud density, and establish a two-dimensional grid index; Step S5: completing missing seedling point clouds and determining pseudo crop rows according to the divided canopy point cloud grids, including: generating a completed point cloud for an area where the missing point cloud rate in the column direction exceeds a threshold; Step S6: Use the least square method to perform linear fitting on the completed canopy point cloud to generate the crop row center line, which is the crop area recognition result.

2. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 1, characterized in that: The step S1 includes: performing Euler angle rotation matrix transformation on the inclined point cloud data collected by LiDAR, and correcting the inclined coordinate system to a horizontal ground coordinate system.

3. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 1, characterized in that: The step S3 in which the lowest point of the root system is located to construct a cylindrical region and the highest point of the Z axis in the region is extracted as the canopy point cloud includes: Extract the bottom point cloud of soybean trunk root system ; Root point cloud Construct a cylindrical area with a radius of R = 5cm as the center, and the height range ; Extract the highest point on the Z axis in the cylindrical area as the canopy point cloud of the crop: in, is the point cloud set in the cylindrical area, is the soybean canopy point cloud. For the The Z coordinate of a point.

4. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 1, characterized in that: The step S4 dynamically calculates the grid size according to the point cloud density, including: ， ， ， in, is the area of ​​a single grid, is the grid expected value, is the area width, is the area height, is the total number of grids in the X-axis direction, is the total number of grids in the Y-axis direction, is the total size of the grid width, The total size of the grid height.

5. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 4, characterized in that: The step S5 comprises: in, For pseudo crop line determination, To complete the number of grids in the X-axis direction of the point cloud, To complete the number of grids in the Y-axis direction of the point cloud, To determine that the row is a pseudo crop row, skip the navigation line generation. Generate a guide line to determine if this row is a soybean crop row. and y represent the coordinate axis values ​​of the completed point cloud. for The total number of canopy point clouds in the column, is the width of a single grid, is the height of a single grid, is the minimum grid width, The minimum grid height.

6. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 1, characterized in that: The step S6 comprises: in, is the slope of the straight line, is the intercept of the straight line, is the Y-axis coordinate of the canopy point cloud, Represents the X-axis coordinate of the canopy point cloud.

7. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 1, characterized in that: The method also includes: establishing a point cloud processing time threshold model according to the coupling relationship between the laser radar scanning frequency and the agricultural machinery driving speed, so as to achieve dynamic adaptation of processing delay and mechanical movement.

8. The method for detecting soybean crop rows at maturity based on 3D LiDAR according to claim 7, characterized in that: The point cloud processing time threshold model includes: Basic timing constraints based on scan frequency : in, is the output frequency; Maximum allowed delay frames : in, For effective soybean crop row scanning range, is the vehicle speed; Maximum allowed processing time : in, is the safety factor.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes a soybean mature crop row detection method based on 3D LiDAR according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a method for detecting soybean crop rows at maturity based on 3D LiDAR as described in any one of claims 1 to 8.

Images