AI-based pallet edge detection method and system

Abstract

The application relates to the technical field of edge detection, and discloses an AI-based cargo pile edge detection method and system. The method comprises the following steps: a radar device is carried on a drone to scan a target cargo pile area to obtain three-dimensional point cloud data, and the data is subjected to coordinate system conversion, denoising and down-sampling processing; ground detection is performed on the three-dimensional point cloud data, a ground plane where the ground is located is identified, and points above the ground plane in the three-dimensional point cloud data are divided into a cargo pile candidate area; a certain point in the cargo pile candidate area is selected as a seed three-dimensional point, a belonging condition is set, three-dimensional points meeting the belonging condition are classified into a current growing cargo pile area until there is no three-dimensional point meeting a growing rule, and finally, a growing cargo pile area obtained is taken as a final cargo pile area; cargo pile contour boundary points are detected and classified, and an accurate contour boundary line is fitted after optimization processing. The application can improve the precision and efficiency of cargo pile edge detection and is suitable for various complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge detection, in particular to an AI-based cargo pile edge detection method and system. BACKGROUND

[0002] In logistics storage, cargo yard and other scenarios, accurate recognition and management of cargo piles are crucial for improving work efficiency, optimizing space utilization and ensuring cargo safety. Traditional manual detection methods are not only time-consuming and labor-intensive, but also susceptible to subjective factors, resulting in low detection accuracy. With the development of automation technology, although some detection methods based on sensors have emerged, these methods often rely on specific environmental conditions or cargo pile shapes, and still have limitations in detecting cargo pile edges in complex scenarios.

[0003] A similar prior art is Chinese patent application No. CN119379923A, which provides a membrane type cargo containment system reference line marking method based on a UAV, including: obtaining the theoretical coordinate values of 16 vertices, the theoretical lengths of 8 longitudinal edges, the theoretical coordinates of the reference axis ring line and the theoretical length of the X direction of the liquid dome according to the design drawing, obtaining the actual coordinate values of 16 vertices, the actual lengths of 8 longitudinal edges, the actual coordinates of the reference axis ring line and the actual length of the X direction of the liquid dome using point cloud processing software, the difference between the actual measurement value and the cargo hold theoretical value does not exceed the check length, and the optimal ring line is selected for spraying.

[0004] Another similar prior art is Chinese patent application No. CN113610916A, which provides a volume calculation algorithm and system for irregular objects such as solid waste based on point cloud data, including: projecting the information of the point cloud data of the measured irregular object to the Z-axis of the space system, determining the probability density function on the z-axis, and then determining the curvature function on the z-axis projection, and calculating the curvature of each point on the z-axis projection using the curvature function; determining the z-axis adjacent slice interval optimal length list according to the curvature of each point, the set reference interval, the reference curvature and the relative interval coefficient; slicing the point cloud according to the list to obtain the z-axis direction point cloud body in turn; detecting the boundary of each slice cross section and calculating the cross section area of each point cloud slice; determining the volume of each body according to the cross section area of each point cloud slice and the height of each body, and adding all the bodies to obtain the final volume of the measured irregular object.

[0005] However, the above two technical solutions do not consider the problem of improving the accuracy of cargo pile edge detection in complex scenarios. Therefore, the present application provides an AI-based cargo pile edge detection method and system. SUMMARY

[0006] The present application provides an AI-based cargo pile edge detection method and system to improve the accuracy and efficiency of cargo pile edge detection.

[0007] In a first aspect, the present application provides an AI-based cargo pile edge detection method, which comprises:

[0008] Step S1, using a UAV to carry a radar device, scanning a target cargo pile area according to a preset flight path, obtaining three-dimensional point cloud data of the target cargo pile area, the three-dimensional point cloud data comprising three-dimensional coordinate information and reflection intensity information, performing coordinate system conversion on the three-dimensional point cloud data, and then performing denoising processing on the converted three-dimensional point cloud data, and performing down-sampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data;

[0009] Step S2, performing ground detection on the representative three-dimensional point cloud data, identifying a plane where the ground is located, and dividing points in the representative three-dimensional point cloud data located above the ground plane into a cargo pile candidate region;

[0010] Step S3, selecting an arbitrary point in the cargo pile candidate region as a seed three-dimensional point, setting a belonging condition, and belonging three-dimensional points meeting the belonging condition to a current growing cargo pile region until there is no three-dimensional point meeting the belonging condition, and finally obtaining a growing cargo pile region as a final cargo pile region;

[0011] Step S4, detecting the cargo pile region to obtain contour boundary points of the cargo pile, classifying and optimizing the contour boundary points, fitting the cargo pile contour based on the optimized contour boundary points, and obtaining an accurate contour boundary line of the cargo pile.

[0012] In combination with the first aspect, in a first implementation manner of the first aspect of the present application, the coordinate system conversion on the three-dimensional point cloud data comprises:

[0013] The position and attitude information of the UAV in the world coordinate system are determined, and the relative position and attitude relationship between the radar device and the UAV is obtained, a coordinate conversion matrix is established according to the position and attitude information of the UAV and the relative position and attitude relationship between the radar device and the UAV, and the coordinates of each three-dimensional point cloud data are converted from the radar coordinate system to the world coordinate system through matrix operation.

[0014] In combination with the first aspect, in a second implementation manner of the first aspect of the present application, the denoising processing on the converted three-dimensional point cloud data comprises:

[0015] A local neighborhood is constructed around each three-dimensional point in the three-dimensional point cloud data, the local neighborhood is a three-dimensional sphere, the number of three-dimensional points in the local neighborhood is counted, the volume of the local neighborhood is calculated, a result value obtained by dividing the number of three-dimensional points by the volume of the local neighborhood is taken as a first characteristic value, a standard deviation of reflection intensity of all three-dimensional points in the local neighborhood is calculated, and the standard deviation is taken as a second characteristic value; a three-dimensional point is determined as a noise point and removed if the first characteristic value is not in a first range or the second characteristic value is not in a second range.

[0016] With reference to the first aspect, in a third implementation manner of the first aspect, the ground plane where the ground surface is located is identified, and the identification includes:

[0017] The three-dimensional point cloud data is divided according to different scales to form a plurality of small cubic grids of different scales, in each scale of the cubic grid, neighborhood three-dimensional points in a first neighborhood of each small cubic are collected, a mean value of coordinates of the neighborhood three-dimensional points is calculated, the coordinates of each three-dimensional point are subtracted from the mean value of the coordinates, a covariance matrix of the three-dimensional point cloud data after the subtraction is calculated, eigenvalues and corresponding eigenvectors are obtained by performing eigenvalue decomposition on the covariance matrix, an eigenvector corresponding to a smallest eigenvalue is selected as a normal vector, a local plane of the corresponding small cubic is determined based on the normal vector and an arbitrary point in the small cubic, and the ground plane is obtained based on the local plane.

[0018] With reference to the first aspect, in a fourth implementation manner of the first aspect, the ground plane is obtained based on the local plane, and the obtaining includes:

[0019] Starting from the largest scale, for the local plane of each small cubic, an angle between normal vectors and a plane distance between the local plane and a neighboring local plane are calculated, it is determined whether any two local planes meet a preset condition, if the preset condition is met, the preset condition is that the angle between the normal vectors is less than a first threshold value and the plane distance is less than a second threshold value, the two local planes are fitted into one local plane, and the step is repeated to combine all local planes meeting the preset condition to obtain a plurality of candidate planes;

[0020] The candidate planes on the largest scale and the local planes meeting the preset condition on a next scale are combined until all local planes on the smallest scale are combined to obtain a plurality of final candidate global planes, and a global plane with the largest number of three-dimensional points in the plurality of candidate global planes is taken as a final ground plane.

[0021] With reference to the first aspect, in a fifth implementation manner of the first aspect, the process of generating the pallet point cloud region includes:

[0022] In each time of classifying the three-dimensional points into the current growing pallet area, a plurality of three-dimensional points most recently classified into the pallet area are selected, a covariance matrix of the selected plurality of three-dimensional points is calculated, the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and eigenvectors, an eigenvector corresponding to the largest eigenvalue is selected as a main direction of the current growing pallet area, and an angle between a plurality of adjacent three-dimensional points and the main direction is calculated. A distance threshold value, a reflection intensity threshold value and an angle threshold value are preset, and the adjacent three-dimensional points meeting the classification condition are classified into the current growing pallet area, the classification condition being that the distance between the adjacent three-dimensional points and the three-dimensional points in the current growing area is less than the distance threshold value, the difference between the reflection intensity of the adjacent three-dimensional points and the reflection intensity in the current growing area is less than the reflection intensity threshold value, and the angle between the adjacent three-dimensional points and the main direction is less than the angle threshold value.

[0023] In combination with the first aspect, in a sixth implementation manner of the first aspect of the present application, the contour boundary points of the pallet are detected, including:

[0024] The normal vector of each three-dimensional point in the pallet area is calculated, the angle between the normal vector of the three-dimensional point and the three-dimensional points in the second neighborhood of the three-dimensional point is calculated, and if the angle is greater than a preset second threshold value, the corresponding three-dimensional point is determined as a boundary point. For each boundary point, the local curvature and the normal vector change rate are also calculated. A first curvature threshold value C1, a second curvature threshold value C2, a first normal vector change threshold value N1 and a second normal vector change threshold value N2 are preset. If the local curvature is greater than C1 and the normal vector change rate is greater than N1, the corresponding boundary point is classified as a convex boundary point. If the local curvature is less than C2 and the normal vector change rate is less than N2, the corresponding boundary point is classified as a concave boundary point. Otherwise, the corresponding boundary point is classified as a smooth boundary point.

[0025] In combination with the first aspect, in a seventh implementation manner of the first aspect of the present application, the contour boundary points are optimized, including:

[0026] The contour boundary points are locally fitted using a moving least squares method, and the positions of the contour boundary points are adjusted.

[0027] Secondly, the present application provides an AI-based pallet edge detection system, which comprises:

[0028] A preprocessing module is configured to use a drone carrying a radar device to scan a target pallet area according to a preset flight path, obtain three-dimensional point cloud data of the target pallet area, the three-dimensional point cloud data comprising three-dimensional coordinate information and reflection intensity information, perform coordinate system conversion on the three-dimensional point cloud data, and then perform denoising processing on the converted three-dimensional point cloud data, and perform downsampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data.

[0029] The first detection module is configured to perform ground detection on the three-dimensional point cloud data, identify a plane where the ground is located, and divide points above the ground plane in the three-dimensional point cloud data into a pallet candidate region.

[0030] The second detection module is configured to select an arbitrary point in the pallet candidate region as a seed three-dimensional point, set a merging condition, and merge three-dimensional points meeting the merging condition into a current growing pallet region until there is no three-dimensional point meeting the merging condition, and finally obtain a growing pallet region as a final pallet region.

[0031] The boundary determination module is configured to detect the pallet region to obtain contour boundary points of the pallet, classify and optimize the contour boundary points, fit a contour of the pallet based on the optimized contour boundary points, and obtain an accurate contour boundary line of the pallet.

[0032] Compared with the prior art, the application has at least the following advantages:

[0033] In the technical solution provided in the application, a UAV is used to carry a radar device to collect data, and preprocessing steps such as downsampling and ground detection are combined, so that the application can quickly process large-scale point cloud data and improve the efficiency of pallet contour recognition; through denoising processing, multi-scale analysis and dynamic growth direction adjustment, the application has high robustness to noise and outliers, and can stably identify the pallet contour in a complex scene; through multi-scale analysis, region growing segmentation and a smoothing algorithm based on a moving least square method, the application can accurately identify the contour of the pallet and has high precision. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0035] Figure 1 is an embodiment schematic diagram of the AI-based pallet edge detection method in the embodiment of the application;

[0036] Figure 2 is a flowchart of identifying a ground plane where the ground is located in the embodiment of the application;

[0037] Figure 3 is a flowchart of obtaining a ground plane based on a local plane in the embodiment of the application;

[0038] Figure 4 is an embodiment schematic diagram of the AI-based pallet edge detection system in the embodiment of the application. DETAILED DESCRIPTION

[0039] The embodiments of the present application provide an AI-based cargo pile edge detection method and system. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" or "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0040] For ease of understanding, the specific flow of the embodiments of the present application is described below. Please refer to Figure 1 One embodiment of the AI-based cargo pile edge detection method in the embodiments of the present application includes:

[0041] Step S1, using a drone to carry a radar device, scanning a target cargo pile area according to a preset flight path, obtaining three-dimensional point cloud data of the target cargo pile area, the three-dimensional point cloud data including three-dimensional coordinate information and reflection intensity information, performing coordinate system conversion on the three-dimensional point cloud data, and then performing denoising processing on the converted three-dimensional point cloud data, and performing downsampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data.

[0042] Specifically, in order to obtain accurate three-dimensional point cloud data of the target pile area, a radar device is carried by a UAV, and the target pile area is scanned according to a preset flight path. The radar device transmits radar waves at a certain frequency and receives radar wave signals reflected from the pile surface. According to the propagation time and intensity of the signals, the spatial position coordinates and reflection intensity values of each point on the pile surface are determined, thereby generating three-dimensional point cloud data containing a large number of discrete points, each point containing three-dimensional coordinate information and reflection intensity information. In order to facilitate subsequent processing, the obtained three-dimensional point cloud data is converted from the original acquisition coordinate system to a unified world coordinate system. In order to improve data quality, the converted three-dimensional point cloud data is denoised. A statistical analysis-based denoising method is used to calculate the distribution characteristics of points in the local neighborhood of each point, such as mean, variance, etc. Points that differ greatly from the distribution characteristics of most points in the neighborhood are judged to be noise points and are removed to improve the accuracy and efficiency of subsequent processing. In order to reduce the amount of data, the denoised three-dimensional point cloud data is down-sampled. For example, a voxel grid down-sampling method can be used to divide the three-dimensional space into small voxel cubes, and then a representative point is retained in each voxel cube, usually the centroid point or the point with the maximum reflection intensity in the voxel cube. Down-sampling can reduce the size of the point cloud data while retaining the main shape characteristics of the pile, improving the efficiency of subsequent processing. The final down-sampled point cloud data is referred to as representative three-dimensional point cloud data.

[0043] Step S2, ground detection is performed on the representative three-dimensional point cloud data, the plane where the ground is located is identified, and the points in the representative three-dimensional point cloud data above the ground plane are divided into pile candidate regions.

[0044] Specifically, in order to identify the plane where the ground is located, ground detection is performed on the down-sampled point cloud data. The specific method of identifying the plane where the ground is located will be explained in detail later. By effectively identifying the plane where the ground is located, accurate reference is provided for subsequent pile region division. In order to preliminarily determine the pile region, first, the points in the representative three-dimensional point cloud data above the ground plane are divided into pile candidate regions. For example, for each point corresponding to the representative three-dimensional point cloud data, the vertical distance from the point to the ground plane is calculated. If the distance is greater than a preset threshold, the point is considered to belong to the pile candidate region. The threshold can be set according to the minimum height of the pile and the actual application scenario, and is usually set to the minimum height value of the pile. Through this step, the point cloud data that may belong to the pile is preliminarily screened, providing an initial region for subsequent region growing segmentation.

[0045] Step S3, select an arbitrary point in the pallet candidate region as a seed three-dimensional point, set the inclusion condition, and include the three-dimensional points meeting the inclusion condition in the current growing pallet region until there is no three-dimensional point meeting the inclusion condition, and take the finally obtained growing pallet region as the final pallet region.

[0046] Specifically, in order to obtain an accurate pallet region, an arbitrary point in the pallet candidate region is selected as a seed three-dimensional point, the selection of the seed three-dimensional point can also be based on the density, reflection intensity or other characteristics of the three-dimensional point cloud data, a representative point is selected as the starting point of growth, and the selection of a suitable seed point can improve the accuracy and efficiency of region growing. In order to guide region growing, the inclusion condition is set, and the three-dimensional points meeting the inclusion condition are included in the current growing pallet region. By setting the inclusion condition, it can be ensured that the growth process is along the boundary of the pallet, and growth to the non-pallet region is avoided. For each unclassified three-dimensional point, it is judged whether the inclusion condition is met. If it is met, it is included in the current growing pallet region. Until there is no three-dimensional point meeting the growth rule, the final growing pallet region is taken as the final pallet region. This step ensures that the growth process is along the boundary of the pallet by dynamically adjusting the growth direction, and improves the accuracy of pallet region segmentation.

[0047] Step S4, detect the contour boundary points of the pallet region, classify and optimize the contour boundary points, fit the pallet contour based on the optimized contour boundary points, and obtain the accurate contour boundary line of the pallet.

[0048] Specifically, in order to extract the contour boundary points of the cargo pile, the final cargo pile region is detected. A normal vector based boundary detection algorithm is used to calculate the normal vector of each point. For those points with large differences in normal vector with the normal vector of surrounding points, they are considered as boundary points. The specific boundary point identification method will be explained in detail later. In order to use different fitting strategies in the subsequent contour fitting process, the identified contour boundary points are classified. According to the local curvature and normal vector change rate of the boundary points, the boundary points are divided into convex boundary points, concave boundary points and smooth boundary points. Through classification marking, different fitting strategies can be used for different types of boundary points to improve the quality and adaptability of contour fitting. In order to smooth the boundary and remove possible burrs and discontinuous points, the extracted boundary points are optimized. Through optimization, the position of the boundary points is adjusted to make them more smooth and continuous, while maintaining the overall shape characteristics of the cargo pile contour. According to the optimized boundary points, a parametric curve fitting method is used to fit the cargo pile contour. Select an appropriate curve model such as Bezier curve, spline curve, etc. Determine the parameters of the curve model so that the fitted curve can be as close as possible to the optimized boundary points, thereby obtaining the accurate contour curve of the cargo pile. This step can realize accurate identification and description of irregular cargo pile contour, and improve the automation level and efficiency of cargo pile management.

[0049] In a specific embodiment, coordinate system conversion is performed on the three-dimensional point cloud data, specifically including the following steps:

[0050] The position and attitude information of the unmanned aerial vehicle in the world coordinate system is determined, and the relative position and attitude relationship between the radar device and the unmanned aerial vehicle is obtained. A coordinate conversion matrix is established according to the position and attitude information of the unmanned aerial vehicle and the relative position and attitude relationship between the radar device and the unmanned aerial vehicle. Through matrix operation, the coordinates of each three-dimensional point cloud data are converted from the radar coordinate system to the world coordinate system.

[0051] Specifically, in order to facilitate subsequent processing, the three-dimensional point cloud data obtained by the above method is converted from the original acquisition coordinate system to the unified world coordinate system, ensuring the compatibility and consistency between different data, and providing a unified coordinate system for subsequent processing.

[0052] In a specific embodiment, the converted three-dimensional point cloud data is denoised, specifically including the following steps:

[0053] A local neighborhood is constructed with each three-dimensional point in the three-dimensional point cloud data as the center, the local neighborhood is a three-dimensional sphere, the number of three-dimensional points in the local neighborhood is counted, the volume of the local neighborhood is also calculated, the result value obtained by dividing the number of three-dimensional points by the volume of the local neighborhood is taken as a first characteristic value, the standard deviation of the reflection intensity of all three-dimensional points in the local neighborhood is also calculated, and the standard deviation is taken as a second characteristic value, a first range and a second range are preset, and the three-dimensional point whose first characteristic value is not in the first range or whose second characteristic value is not in the second range is determined as a noise point and removed.

[0054] Specifically, in order to improve the accuracy and reliability of the cargo pile contour recognition, improve the data quality, the converted three-dimensional point cloud data is denoised, and the noise points are effectively identified and removed, thereby providing a higher quality data basis for subsequent cargo pile region segmentation and contour extraction, in order to analyze the local features around each point, a local neighborhood is constructed with each three-dimensional point in the three-dimensional point cloud data as the center, the local neighborhood is defined as a three-dimensional spherical region, and the radius thereof is dynamically adjusted according to the density of the point cloud data, for example, the average distance between all points in the point cloud data is first counted, and then the radius r of the local neighborhood is set to k x average distance, wherein k is a constant greater than 1, used to control the size of the local neighborhood, in this way, it can be ensured that the local neighborhood can cover enough adjacent points, while avoiding interference in a too large range, the first characteristic value reflects the density of the point cloud in the local neighborhood, a higher density usually means that the local neighborhood belongs to the cargo pile region, and a lower density may indicate a noise point or a background region, the second characteristic value reflects the variation degree of the reflection intensity in the local neighborhood, a smaller standard deviation indicates that the reflection intensity is more consistent, while a larger standard deviation may mean that there is noise, in order to distinguish noise points and effective points, a first range and a second range are preset, the two ranges can be adjusted according to the characteristics of the point cloud data and the actual application scenario, if the first characteristic value of a certain point is not in the first range, or the second characteristic value is not in the second range, the point is determined as a noise point and removed, this method can comprehensively consider the density and reflection intensity variation of the point cloud, and effectively identify the noise points that do not meet the cargo pile characteristics.

[0055] In a specific embodiment, a plane where the ground is located is identified, specifically including the following steps:

[0056] The three-dimensional point cloud data is divided according to different scales to form a plurality of small cubic grids of different scales. In each scale cubic grid, the neighbor three-dimensional points in the first neighborhood of each small cubic are collected, the coordinate mean value of the neighbor three-dimensional points is calculated, the coordinate of each three-dimensional point is subtracted from the coordinate mean value, the covariance matrix of the three-dimensional point cloud data after the coordinate mean value is subtracted is calculated, the covariance matrix is eigenvalue decomposed to obtain eigenvalues and corresponding eigenvectors, the eigenvector corresponding to the smallest eigenvalue is selected as a normal vector, and a local plane of the corresponding small cubic is determined based on the normal vector and an arbitrary point in the small cubic. The ground plane is obtained based on the local plane.

[0057] Specifically, in order to accurately identify the ground plane of the region where the goods pile is located, thereby providing a reliable basis for the segmentation of the goods pile region, as shown in Figure 2 The flowchart for identifying the ground plane is shown. The local plane fitting is performed in the cubic grid of different scales, and the local planes meeting the preset conditions are gradually merged to finally determine the ground plane, so as to improve the accuracy and robustness of ground detection and ensure the accuracy of the segmentation of the goods pile region.

[0058] In order to capture the features of the ground plane under different scales, the point cloud data is divided according to different scales to form a plurality of small cubic grids of different scales. For example, three scales can be selected, and the sizes of the cubic are 0.25m, 0.15m and 0.1m. This multi-scale division can adapt to the ground features in different scenes and improve the robustness of ground detection. In each scale cubic grid, for each small cubic, the neighbor three-dimensional points in the first neighborhood of the small cubic are collected, the first neighborhood can be defined as the points in the cubic adjacent to the current cubic, the coordinate mean value of the neighbor three-dimensional points is calculated, the coordinate of each three-dimensional point is subtracted from the coordinate mean value, the translation of the point cloud data is realized, and the mean value of the translated point cloud data is the origin. Through the translation operation, the subsequent covariance matrix calculation and eigenvalue decomposition process can be simplified. The covariance matrix of the three-dimensional point cloud data after the coordinate mean value is subtracted is calculated. Assuming that the translated point cloud data is P={p1,p2,…,pN}, where each point The covariance matrix is The covariance matrix C is decomposed to obtain three eigenvalues e1, e2, e3 and corresponding eigenvectors v1, v2, v3, the eigenvalue represents the change degree of data in the corresponding eigenvector direction, and the eigenvector corresponding to the smallest eigenvalue is selected as the normal vector, because the eigenvector corresponding to the smallest eigenvalue represents the direction with the smallest data change, that is, the normal vector of the local plane, and the local plane of the corresponding small cube is determined based on the normal vector and an arbitrary point in the small cube. The plane equation is calculated, and the plane determined by the plane equation is determined as the local plane of each small cube. The ground plane is obtained based on the local plane subsequently, and the method for obtaining the ground plane based on the local plane will be explained in detail subsequently.

[0059] In a specific embodiment, the ground plane is obtained based on the local plane, specifically including the following steps:

[0060] Starting from the largest scale, for each local plane of each small cube, the normal vector included angle and the plane distance between the local plane and the local plane of the adjacent small cube are calculated, and it is judged whether any two local planes meet the preset condition. If the preset condition is met, the two local planes are fitted into one local plane, and the step is repeated, and all local planes meeting the preset condition are merged to obtain a plurality of candidate planes.

[0061] The candidate planes at the largest scale and the local planes meeting the preset condition at the next scale are merged until all local planes at the smallest scale are merged to obtain a plurality of final candidate global planes. The global plane with the largest number of three-dimensional points in the plurality of candidate global planes is taken as the final ground plane.

[0062] Specifically, as Figure 3 shown is a flowchart for obtaining the ground plane based on the local plane. In order to gradually refine the identification of the ground plane, the merging of the local planes starts from the largest scale, and at each scale, the local planes of each small cube are analyzed. For each local plane of each small cube, the normal vector included angle and the plane distance between the local plane and the local plane of the adjacent small cube are calculated. The calculation formula of the normal vector included angle is , wherein n1 and n2 are the normal vectors of the two local planes, and the distance d between the two local planes is calculated. Assuming that the equations of the two local planes are and , the plane distance is The geometric relationship between two local planes can be quantified by the above calculation, which provides a basis for subsequent merging. The preset conditions are that the angle between the normal vectors is less than a preset first threshold and the plane distance is less than a preset second threshold. If the two local planes meet these conditions, they are considered to belong to the same global plane and can be merged. The two local planes that meet the preset conditions are fitted into one local plane, for example, a new normal vector and distance parameter can be calculated by a weighted average method to obtain a new local plane. The above steps are repeated to merge all local planes that meet the preset conditions to obtain a plurality of candidate planes. The identification of the ground plane can be gradually refined through the step-by-step merging process, improving the detection accuracy.

[0063] Starting from the largest-scale cubic grid, a series of candidate planes are obtained after the local plane fitting and merging steps. These candidate planes are based on the merging results of the largest-scale local planes and can capture the approximate shape and position of the ground. These candidate planes are selected as the initial global plane set to provide a basis for subsequent cross-scale merging, and the local planes in the next smaller-scale cubic grid are gradually considered. For each smaller-scale local plane, the angle between its normal vector and the normal vector of each global plane in the current global plane set and the plane distance are calculated. If the angle between the normal vector of the local plane and the normal vector of a global plane is less than a preset first threshold and the plane distance is less than a preset second threshold, the local plane and the global plane are considered to belong to the same ground plane and can be merged. The above cross-scale merging process is repeated to gradually compare and merge all smaller-scale local planes with the current global plane set. This step-by-step refinement process can make full use of information at different scales to improve the accuracy of ground plane detection.

[0064] Assuming there are three scales in total, the candidate planes on the largest scale and the local planes on the second largest scale that meet the preset conditions are merged to determine a plurality of candidate planes. Then, the plurality of candidate planes determined again and all local planes on the smallest scale are merged to obtain a plurality of final candidate global planes.

[0065] For each candidate global plane, the number of three-dimensional points it contains is counted. The global plane with the most three-dimensional points can usually better fit the ground point cloud data because it contains more ground point information. Therefore, from the plurality of candidate global planes, the global plane containing the most three-dimensional points is selected as the final ground plane. This selection method based on the number of point clouds can effectively avoid misjudgment and ensure the reliability and accuracy of the final determined ground plane.

[0066] In a specific embodiment, the process of generating a pallet point cloud region specifically includes the following steps:

[0067] At each time of attributing a three-dimensional point to the current growing pallet area, a plurality of three-dimensional points most recently attributed to the pallet area are selected, a covariance matrix of the selected plurality of three-dimensional points is calculated, the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and eigenvectors, an eigenvector corresponding to the largest eigenvalue is selected as a main direction of the current growing pallet area, and an angle between a plurality of adjacent three-dimensional points and the main direction is calculated. A distance threshold value, a reflection intensity threshold value, and an angle threshold value are preset, and the adjacent three-dimensional points meeting the attribution condition are attributed to the current growing pallet area. The attribution condition is that the distance between the adjacent three-dimensional point and the three-dimensional point in the current growing area is less than the distance threshold value, the difference between the reflection intensity of the adjacent three-dimensional point and the reflection intensity in the current growing area is less than the reflection intensity threshold value, and the angle between the adjacent three-dimensional point and the main direction is less than the angle threshold value.

[0068] Specifically, in order to accurately identify and extract the pallet area from complex point cloud data, at each time of attributing a three-dimensional point to the current growing pallet area, a plurality of three-dimensional points most recently attributed to the pallet area are selected, these points are used to calculate the main direction of the current growing area, to ensure that the growing direction can be dynamically adjusted to adapt to the actual shape of the pallet. By selecting the most recently attributed points, it can be ensured that the calculated main direction can reflect the latest state of the current growing area, improving the accuracy and adaptability of the growing direction. The covariance matrix of the selected plurality of three-dimensional points is calculated, the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and corresponding eigenvectors. The eigenvector corresponding to the largest eigenvalue represents the direction with the largest data change, i.e. the main distribution direction of the point cloud data, which can be used as a reference for the growing direction. Therefore, the eigenvector corresponding to the largest eigenvalue is selected as the main direction of the current growing pallet area. The angle between a plurality of adjacent three-dimensional points and the main direction is calculated, which is used to select points with a small angle with the main direction, to ensure that the growing process can be carried out along the actual boundary of the pallet, avoiding growing into a non-pallet area. The distance threshold value, the reflection intensity threshold value, and the angle threshold value are preset, which are used to control the selection of points during the growing process, to ensure that only points meeting the conditions can be attributed to the growing area. Reasonable setting of these threshold values can effectively filter out points that do not meet the characteristics of the pallet, improving the accuracy of the growing process. For each adjacent three-dimensional point, it is checked whether it meets the attribution condition. If a point meets the above three conditions at the same time, the point is attributed to the current growing pallet area. Through strict attribution condition screening, it can be ensured that the growing process can accurately proceed along the actual boundary of the pallet, avoiding misattributing points in a non-pallet area to the growing area.

[0069] The above steps are repeated until there is no adjacent three-dimensional point meeting the attribution condition. The finally obtained growing pallet area is taken as the final pallet area. This step-by-step growing process can gradually refine the extraction of the pallet area, improving the extraction accuracy.

[0070] In a specific embodiment, the pallet point cloud region is detected to obtain the contour boundary points of the pallet, specifically including the following steps:

[0071] The normal vector of each three-dimensional point in the pallet region is calculated, the angle between the normal vector of the three-dimensional point and the three-dimensional points in its second neighborhood is calculated, and if the angle is greater than a preset second threshold, the corresponding three-dimensional point is determined as a boundary point. For each boundary point, the corresponding local curvature and normal vector change rate are also calculated. A first curvature threshold C1, a second curvature threshold C2, a first normal vector change threshold N1 and a second normal vector change threshold N2 are preset. If the local curvature is greater than C1 and the normal vector change rate is greater than N1, the corresponding boundary point is classified as a convex boundary point. If the local curvature is less than C2 and the normal vector change rate is less than N2, the corresponding boundary point is classified as a concave boundary point. Otherwise, the corresponding boundary point is classified as a smooth boundary point.

[0072] Specifically, in order to accurately extract the contour boundary points of the pallet from the pallet point cloud region and classify these boundary points, first, the normal vector of each three-dimensional point in the pallet region is calculated, the angle between the normal vector of each three-dimensional point and the three-dimensional points in its second neighborhood is calculated, and a second threshold is preset. If the angle between the normal vector of a certain point and any point in its second neighborhood is greater than the second threshold, the point is determined as a boundary point. For each boundary point, the corresponding local curvature and normal vector change rate are also calculated. Based on the local curvature and the normal vector change rate, the above method is used to classify the boundary points into convex boundary points, concave boundary points and smooth boundary points. By classifying the boundary points, different fitting strategies can be used in the subsequent contour fitting process according to the type of the boundary points. Special fitting algorithms can be used for convex boundary points and concave boundary points to ensure the accuracy and smoothness of the contour, while general fitting methods can be used for smooth boundary points, thereby improving the quality and adaptability of the entire pallet contour fitting.

[0073] Further, the normal vector of the three-dimensional point is calculated, including:

[0074] Selecting points in its neighborhood, selecting points in its neighborhood, the definition of neighborhood can be based on fixed radius or fixed number of nearest neighbors, such as selecting points with a distance less than a fixed radius from the three-dimensional point, or selecting the nearest points from the three-dimensional point, calculating the mean of the coordinates of all points in the field, translating all points in the field based on the mean, constructing a covariance matrix of all points in the field based on the translated points, performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and eigenvectors. The eigenvalue represents the degree of change of the data in the corresponding eigenvector direction. The eigenvector corresponding to the smallest eigenvalue usually represents the direction with the smallest data change, i.e. the normal vector. The eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the corresponding three-dimensional point.

[0075] Further, the local curvature is calculated, including:

[0076] For each boundary point Q, select the two points Q1 and Q2 closest to this boundary point, respectively called the first point and the second point, calculate the first vector V1 = Q1 - Q and the second vector V2 = Q - Q1, calculate the angle θ1 between the first vector and the second vector, which is approximated by the product of the angle and the lengths of the two vectors, the two vectors being the first vector V1 and the second vector V2, the formula for calculating the local curvature b is: , denotes the modulus of a vector.

[0077] Further, the normal vector change rate r is calculated, including:

[0078] For each boundary point Q, obtain the normal vectors M1 and M2 of the two points Q1 and Q2 closest to this boundary point, then the formula for calculating the normal vector change rate r is: .

[0079] In a specific embodiment, the contour boundary points are optimized, specifically including the following steps:

[0080] The mobile least squares method is used to locally fit the boundary points and adjust the positions of the contour boundary points.

[0081] Specifically, in order to improve the smoothness and accuracy of the cargo pile contour boundary points, the present application selects to use the mobile least squares method to locally fit and adjust the positions of the boundary points, which aims to reduce noise and irregularities through local fitting while maintaining the overall shape features of the cargo pile contour, providing a higher quality data basis for subsequent contour fitting and analysis.

[0082] The AI-based cargo pile edge detection method in the embodiments of the present application is described above, and the AI-based cargo pile edge detection system in the embodiments of the present application is described below, please refer to Figure 4 , an embodiment of the AI-based cargo pile edge detection system in the embodiments of the present application includes:

[0083] A preprocessing module is configured to use a drone carrying a radar device to scan a target cargo pile area according to a preset flight path, obtain three-dimensional point cloud data of the target cargo pile area, the three-dimensional point cloud data including three-dimensional coordinate information and reflection intensity information, perform coordinate system conversion on the three-dimensional point cloud data, and then perform denoising processing on the converted three-dimensional point cloud data, and perform downsampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data;

[0084] A first detection module is configured to perform ground detection on the representative three-dimensional point cloud data, identify a ground plane where the ground is located, and divide points in the representative three-dimensional point cloud data located above the ground plane into a cargo pile candidate area;

[0085] The second detection module is configured to select a certain point as a seed three-dimensional point from the pallet candidate region, set a belonging condition, and belong three-dimensional points meeting the belonging condition to a current growing pallet region until there is no three-dimensional point meeting the growing rule, and finally obtain a growing pallet region as a final pallet region.

[0086] The boundary determination module is configured to detect the pallet point cloud region to obtain contour boundary points of the pallet, classify the contour boundary points, optimize the contour boundary points, fit the pallet contour based on the optimized boundary points, and obtain an accurate contour boundary line of the pallet.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0088] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0089] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An AI-based pallet edge detection method, characterized by, The method comprises: Step S1, using a UAV carrying a radar device to scan a target pile area according to a preset flight path, to obtain three-dimensional point cloud data of the target pile area, the three-dimensional point cloud data comprising three-dimensional coordinate information and reflection intensity information, performing coordinate system conversion on the three-dimensional point cloud data, and then performing denoising processing on the converted three-dimensional point cloud data, and performing downsampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data; Step S2, performing ground detection on the representative three-dimensional point cloud data, identifying a ground plane where the ground is located, and dividing points above the ground plane in the representative three-dimensional point cloud data into a pile candidate area; Step S3, selecting an arbitrary point in the pile candidate area as a seed three-dimensional point, setting a merging condition, merging three-dimensional points meeting the merging condition into a current growing pile area until there is no three-dimensional point meeting the merging condition, and taking a final growing pile area as a final pile area; Step S4, detecting the pile area to obtain contour boundary points of the pile, classifying and optimizing the contour boundary points, fitting a pile contour based on the optimized contour boundary points, and obtaining an accurate contour boundary line of the pile; The ground plane where the ground is located is identified, comprising: The three-dimensional point cloud data is divided according to different scales to form a plurality of small cubic grid cells of different scales, in each scale cubic grid cell, the neighborhood three-dimensional points in the first neighborhood of each small cubic grid cell are collected, the coordinate mean value of the neighborhood three-dimensional points is calculated, the coordinate of each three-dimensional point is subtracted from the coordinate mean value, the covariance matrix of the three-dimensional point cloud data after the coordinate mean value is subtracted is calculated, the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and corresponding eigenvectors, the eigenvector corresponding to the smallest eigenvalue is selected as a normal vector, and the local plane of the corresponding small cubic grid cell is determined based on the normal vector and an arbitrary point in the small cubic grid cell; and the ground plane is obtained based on the local plane; The ground plane is obtained based on the local plane, comprising: Starting from the largest scale, the normal vector included angle and the plane distance between the local plane of each small cubic grid cell and the local plane of an adjacent small cubic grid cell are calculated at each scale, it is determined whether any two local planes meet a preset condition, if the preset condition is met, the preset condition is that the normal vector included angle is less than a preset first threshold value and the plane distance is less than a preset second threshold value, the two local planes are fitted into one local plane, and the step is repeated to combine all local planes meeting the preset condition to obtain a plurality of candidate planes; The candidate planes on the largest scale and the local planes on the next scale meeting the preset condition are combined until all local planes on the smallest scale are combined to obtain a plurality of final candidate global planes, and the global plane with the largest number of three-dimensional points in the plurality of candidate global planes is taken as the final ground plane.

2. The method of claim 1, wherein, The three-dimensional point cloud data is subjected to coordinate system conversion, comprising: The position and attitude information of the unmanned aerial vehicle in the world coordinate system is determined, and the relative position and attitude relationship between the radar device and the unmanned aerial vehicle is obtained. A coordinate conversion matrix is established according to the position and attitude information of the unmanned aerial vehicle and the relative position and attitude relationship between the radar device and the unmanned aerial vehicle. The coordinates of each three-dimensional point cloud data are converted from the radar coordinate system to the world coordinate system through matrix operation.

3. The method of claim 1, wherein, The converted three-dimensional point cloud data is subjected to denoising processing, including: A corresponding local neighborhood is constructed with each three-dimensional point in the three-dimensional point cloud data as the center. The local neighborhood is a three-dimensional sphere. The number of three-dimensional points in the local neighborhood is counted. The volume of the local neighborhood is calculated. The result value obtained by dividing the number of three-dimensional points by the volume of the local neighborhood is taken as a first characteristic value. The standard deviation of the reflection intensity of all three-dimensional points in the local neighborhood is calculated. The standard deviation is taken as a second characteristic value. A first range and a second range are preset. The three-dimensional points whose first characteristic value is not in the first range or whose second characteristic value is not in the second range are determined as noise points and are removed.

4. The method of claim 1, wherein, The process of generating the pallet point cloud region includes: When each three-dimensional point is classified into the current growing pallet region, a plurality of three-dimensional points most recently classified into the pallet region are selected. The covariance matrix of the selected plurality of three-dimensional points is calculated. The covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and eigenvectors. The eigenvector corresponding to the largest eigenvalue is selected as the main direction of the current growing pallet region. The included angles between a plurality of adjacent three-dimensional points and the main direction are calculated. A distance threshold value, a reflection intensity threshold value, and an angle threshold value are preset. The adjacent three-dimensional points that meet the classification condition are classified into the current growing pallet region. The classification condition is that the distance between the adjacent three-dimensional point and the three-dimensional point in the current growing region is less than the distance threshold value, the difference between the reflection intensity of the adjacent three-dimensional point and the reflection intensity in the current growing region is less than the reflection intensity threshold value, and the included angle between the adjacent three-dimensional point and the main direction is less than the angle threshold value.

5. The method of claim 1, wherein, The pallet point cloud region is detected to obtain the contour boundary points of the pallet, including: The normal vector of each three-dimensional point in the pallet region is calculated. The included angle between the normal vector of the three-dimensional point and the three-dimensional points in the second neighborhood of the three-dimensional point is calculated. If the included angle is greater than a preset second threshold value, the corresponding three-dimensional point is determined as a boundary point. For each boundary point, the local curvature and the normal vector change rate are also calculated. A first curvature threshold value C1, a second curvature threshold value C2, a first normal vector change threshold value N1, and a second normal vector change threshold value N2 are preset. If the local curvature is greater than C1 and the normal vector change rate is greater than N1, the corresponding boundary point is classified as a convex boundary point. If the local curvature is less than C2 and the normal vector change rate is less than N2, the corresponding boundary point is classified as a concave boundary point. Otherwise, the corresponding boundary point is classified as a smooth boundary point.

6. The method of claim 1, wherein, The contour boundary points are subjected to optimization processing, including: using a moving least squares method to locally fit the contour boundary points, and adjusting the positions of the contour boundary points.

7. An AI-based pallet edge detection system for implementing the AI-based pallet edge detection method according to any one of claims 1-6, characterized by, The system includes: The preprocessing module is configured to use a UAV carrying a radar device to scan a target pile area according to a preset flight path, to obtain three-dimensional point cloud data of the target pile area, the three-dimensional point cloud data including three-dimensional coordinate information and reflection intensity information, to perform coordinate system conversion on the three-dimensional point cloud data, and to perform denoising processing on the converted three-dimensional point cloud data, and to perform downsampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data; The first detection module is configured to perform ground detection on the representative three-dimensional point cloud data, to identify a ground plane, and to divide points above the ground plane in the representative three-dimensional point cloud data into a pile candidate area; The second detection module is configured to select an arbitrary point in the pile candidate area as a seed three-dimensional point, to set a merging condition, to merge three-dimensional points meeting the merging condition into a current growing pile area, and to stop until there is no three-dimensional point meeting the merging condition, and to obtain a final growing pile area as a final pile area; The boundary determination module is configured to detect the pile area to obtain contour boundary points of the pile, to classify and optimize the contour boundary points, to fit a pile contour based on the optimized contour boundary points, and to obtain an accurate contour boundary line of the pile; wherein the ground plane is identified by: dividing the three-dimensional point cloud data into a plurality of small cubic grids of different scales, collecting, in each scale of cubic grid, neighborhood three-dimensional points in a first neighborhood of each small cubic, calculating a coordinate mean value of the neighborhood three-dimensional points, subtracting the coordinate mean value from a coordinate of each three-dimensional point, calculating a covariance matrix of the three-dimensional point cloud data after the coordinate mean value is subtracted, performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and corresponding eigenvectors, selecting an eigenvector corresponding to the smallest eigenvalue as a normal vector, determining a local plane of the corresponding small cubic based on the normal vector and an arbitrary point in the small cubic, and obtaining the ground plane based on the local plane; obtaining the ground plane based on the local plane includes: starting from the largest scale, calculating, at each scale, a normal vector included angle and a plane distance between the local plane of each small cubic and the local planes of adjacent small cubics, determining whether any two local planes meet a preset condition, fitting the two local planes into one local plane if the preset condition is met, the preset condition being that the normal vector included angle is less than a preset first threshold value and the plane distance is less than a preset second threshold value, and repeating the step to combine all local planes meeting the preset condition to obtain a plurality of candidate planes; merging the candidate planes at the largest scale and the local planes meeting the preset condition at the next scale until all local planes at the smallest scale are merged to obtain a plurality of final candidate global planes, and selecting a global plane with the most three-dimensional points in the plurality of candidate global planes as the final ground plane.

Images