AI-based goods stack edge detection method and system

Abstract

The invention relates to the technical field of edge detection, and discloses an AI-based goods stack edge detection method and system. The method comprises the following steps: scanning a target cargo stack area by using radar equipment carried by an unmanned aerial vehicle to obtain three-dimensional point cloud data, and performing coordinate system conversion, denoising and down-sampling processing on the data; ground detection is carried out on the representative three-dimensional point cloud data, a ground plane where the ground is located is identified, and points, located above the ground plane, in the representative three-dimensional point cloud data are divided into cargo stack candidate areas; selecting a certain point from the cargo stack candidate area as a seed three-dimensional point, setting a classification condition, classifying the three-dimensional points meeting the classification condition into the current growth cargo stack area until the three-dimensional points meeting the growth rule do not exist, and taking the finally obtained growth cargo stack area as a final cargo stack area; and detecting and classifying the contour boundary points of the goods stack, and performing fitting after optimization processing to obtain an accurate contour boundary line. The method can improve the accuracy and efficiency of goods stack edge detection, and is suitable for various complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge detection, in particular to an AI-based cargo pile edge detection method and system. BACKGROUND

[0002] In logistics storage, cargo yard and other scenarios, accurate recognition and management of cargo piles are crucial for improving work efficiency, optimizing space utilization and ensuring cargo safety. Traditional manual detection methods are not only time-consuming and labor-intensive, but also susceptible to subjective factors, resulting in low detection accuracy. With the development of automation technology, although some detection methods based on sensors have emerged, these methods often rely on specific environmental conditions or cargo pile shapes, and still have limitations in detecting cargo pile edges in complex scenarios.

[0003] A similar prior art is Chinese patent application No. CN119379923A, which provides a membrane type cargo containment system reference line marking method based on a UAV, including: obtaining the theoretical coordinate values of 16 vertices, the theoretical lengths of 8 longitudinal edges, the theoretical coordinates of the reference axis ring line and the theoretical length of the X direction of the liquid dome according to the design drawing, obtaining the actual coordinate values of 16 vertices, the actual lengths of 8 longitudinal edges, the actual coordinates of the reference axis ring line and the actual length of the X direction of the liquid dome using point cloud processing software, the difference between the actual measurement value and the cargo hold theoretical value does not exceed the check length, and the optimal ring line is selected for spraying.

[0004] Another similar prior art is Chinese patent application No. CN113610916A, which provides a volume calculation algorithm and system for irregular objects such as solid waste based on point cloud data, including: projecting the information of the point cloud data of the measured irregular object to the Z-axis of the space system, determining the probability density function on the z-axis, and then determining the curvature function on the z-axis projection, and calculating the curvature of each point on the z-axis projection using the curvature function; determining the z-axis adjacent slice interval optimal length list according to the curvature of each point, the set reference interval, the reference curvature and the relative interval coefficient; slicing the point cloud according to the list to obtain the z-axis direction point cloud body in turn; detecting the boundary of each slice cross section and calculating the cross section area of each point cloud slice; determining the volume of each body according to the cross section area of each point cloud slice and the height of each body, and adding all the bodies to obtain the final volume of the measured irregular object.

[0005] However, the above two technical solutions do not consider the problem of improving the accuracy of cargo pile edge detection in complex scenarios. Therefore, the present application provides an AI-based cargo pile edge detection method and system. SUMMARY

[0006] The present application provides an AI-based cargo pile edge detection method and system to improve the accuracy and efficiency of cargo pile edge detection.

[0007] In a first aspect, the application provides an AI-based pallet edge detection method, which comprises: Step S1, using a UAV to carry a radar device, scanning a target pallet area according to a preset flight path, obtaining three-dimensional point cloud data of the target pallet area, the three-dimensional point cloud data comprising three-dimensional coordinate information and reflection intensity information, performing coordinate system conversion on the three-dimensional point cloud data, and then performing denoising processing on the converted three-dimensional point cloud data, and performing down-sampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data; Step S2, performing ground detection on the representative three-dimensional point cloud data, identifying a plane where the ground is located, and dividing points in the representative three-dimensional point cloud data located above the ground plane into a pallet candidate area; Step S3, selecting an arbitrary point in the pallet candidate area as a seed three-dimensional point, setting a belonging condition, and belonging three-dimensional points meeting the belonging condition to a current growing pallet area until there is no three-dimensional point meeting the belonging condition, and finally obtaining a growing pallet area as a final pallet area; Step S4, detecting the pallet area to obtain contour boundary points of the pallet, classifying and optimizing the contour boundary points, fitting the pallet contour based on the optimized contour boundary points, and obtaining an accurate contour boundary line of the pallet.

[0008] In combination with the first aspect, in a first implementation manner of the first aspect of the application, the coordinate system conversion on the three-dimensional point cloud data comprises: The position and attitude information of the UAV in the world coordinate system are determined, the relative position and attitude relationship between the radar device and the UAV are obtained, the coordinate conversion matrix is established according to the position and attitude information of the UAV and the relative position and attitude relationship between the radar device and the UAV, and the coordinates of each three-dimensional point cloud data are converted from the radar coordinate system to the world coordinate system through matrix operation.

[0009] In combination with the first aspect, in a second implementation manner of the first aspect of the application, the denoising processing on the converted three-dimensional point cloud data comprises: A corresponding local neighborhood is constructed with each three-dimensional point in the three-dimensional point cloud data as the center, the local neighborhood is a three-dimensional sphere, the number of three-dimensional points in the local neighborhood is counted, the volume of the local neighborhood is calculated, the number of three-dimensional points is divided by the volume of the local neighborhood to obtain a first feature value, the standard deviation of the reflection intensity of all three-dimensional points in the local neighborhood is calculated, the standard deviation is taken as a second feature value, a first range and a second range are preset, and the three-dimensional points whose first feature value is not in the first range or second feature value is not in the second range are determined as noise points and removed.

[0010] In a third implementation form of the first aspect, in the method, the ground plane is identified, including: The three-dimensional point cloud data is divided according to different scales to form a plurality of small cubic grids of different scales. In each cubic grid of a scale, neighbor three-dimensional points in a first neighborhood of each small cube are collected, a coordinate mean value of the neighbor three-dimensional points is calculated, each three-dimensional point is subtracted by the coordinate mean value, a covariance matrix of the three-dimensional point cloud data after the subtraction is calculated, eigenvalues and eigenvectors corresponding to the covariance matrix are obtained by performing eigenvalue decomposition on the covariance matrix, an eigenvector corresponding to a smallest eigenvalue is selected as a normal vector, a local plane of the corresponding small cube is determined based on the normal vector and an arbitrary point in the small cube, and the ground plane is obtained based on the local plane.

[0011] In a fourth implementation form of the first aspect, in the method, the ground plane is obtained based on the local plane, including: Starting from a largest scale, for each local plane of each small cube, a normal vector included angle and a plane distance between the local plane and a local plane of a neighboring small cube are calculated, it is determined whether any two local planes meet a preset condition, if the preset condition is met, the preset condition is that the normal vector included angle is less than a preset first threshold value and the plane distance is less than a preset second threshold value, the two local planes are fitted into one local plane, and the step is repeated to combine all local planes meeting the preset condition to obtain a plurality of candidate planes. The candidate planes on the largest scale and the local planes meeting the preset condition on a next scale are combined until all the local planes on the smallest scale are combined to obtain a plurality of final candidate global planes, and a global plane with a largest number of three-dimensional points in the plurality of candidate global planes is taken as a final ground plane.

[0012] In a fifth implementation form of the first aspect, in the method, the process of generating the pallet point cloud region includes: In each time when a three-dimensional point is classified into the current growing pallet region, a plurality of three-dimensional points most recently classified into the pallet region are selected, a covariance matrix of the selected plurality of three-dimensional points is calculated, eigenvalues and eigenvectors are obtained by performing eigenvalue decomposition on the covariance matrix, an eigenvector corresponding to a largest eigenvalue is selected as a main direction of the current growing pallet region, an included angle between a plurality of neighbor three-dimensional points and the main direction is calculated, a distance threshold value, a reflection intensity threshold value and an angle threshold value are preset, and the neighbor three-dimensional points meeting a classification condition are classified into the current growing pallet region, the classification condition is that a distance between the neighbor three-dimensional point and a three-dimensional point in the current growing region is less than the distance threshold value, a difference between a reflection intensity of the neighbor three-dimensional point and a reflection intensity in the current growing region is less than the reflection intensity threshold value, and the included angle between the neighbor three-dimensional point and the main direction is less than the angle threshold value.

[0013] In a sixth implementation form of the first aspect, the detecting the point cloud region of the cargo pile to obtain the contour boundary points of the cargo pile comprises: The normal vector of each three-dimensional point in the cargo pile region is calculated, the angle between the normal vector of the three-dimensional point and the normal vector of the three-dimensional point in the second neighborhood of the three-dimensional point is calculated, if the angle is greater than a preset second threshold, the corresponding three-dimensional point is determined as a boundary point, for each boundary point, the local curvature and the normal vector change rate of the corresponding boundary point are calculated, a preset first curvature threshold C1, a second curvature threshold C2, a first normal vector change threshold N1 and a second normal vector change threshold N2 are set, if the local curvature is greater than C1 and the normal vector change rate is greater than N1, the corresponding boundary point is classified as a convex boundary point, if the local curvature is less than C2 and the normal vector change rate is less than N2, the corresponding boundary point is classified as a concave boundary point, otherwise, the corresponding boundary point is classified as a smooth boundary point.

[0014] In a seventh implementation form of the first aspect, the contour boundary points are optimized, comprising: The contour boundary points are locally fitted by using a moving least square method, and the positions of the contour boundary points are adjusted.

[0015] In a second aspect, the application provides an AI-based cargo pile edge detection system, comprising: A preprocessing module is configured to use a radar device carried by a UAV to scan a target cargo pile region according to a preset flight path, to obtain three-dimensional point cloud data of the target cargo pile region, the three-dimensional point cloud data comprising three-dimensional coordinate information and reflection intensity information, to perform coordinate system conversion on the three-dimensional point cloud data, and to perform denoising processing on the converted three-dimensional point cloud data, and to perform down-sampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data. A first detection module is configured to perform ground detection on the representative three-dimensional point cloud data, to identify a plane where the ground is located, and to divide points in the representative three-dimensional point cloud data located above the ground plane into a cargo pile candidate region. A second detection module is configured to select an arbitrary point in the cargo pile candidate region as a seed three-dimensional point, to set a belonging condition, to classify three-dimensional points meeting the belonging condition into a current growing cargo pile region, and to stop until there is no three-dimensional point meeting the belonging condition, and to obtain a final growing cargo pile region as a final cargo pile region. A boundary determination module is configured to detect the cargo pile region to obtain contour boundary points of the cargo pile, to classify the contour boundary points, and to optimize the contour boundary points, to fit the contour of the cargo pile based on the optimized contour boundary points, and to obtain an accurate contour boundary line of the cargo pile.

[0016] Compared with the prior art, the application has at least the following advantages: In the technical scheme provided in the application, the unmanned aerial vehicle carries a radar device to collect data, and the application can quickly process large-scale point cloud data and improve the efficiency of cargo pile contour recognition by combining with preprocessing steps such as downsampling and ground detection. Through denoising processing, multi-scale analysis and dynamic growth direction adjustment, the application has high robustness to noise and outliers, and can stably identify the contour of the cargo pile in a complex scene. Through multi-scale analysis, region growing segmentation and smoothing algorithm based on the moving least square method, the contour of the cargo pile can be accurately identified, and the application has high precision. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical scheme of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 An embodiment of the AI-based cargo pile edge detection method in the embodiment of the application is shown in the figure. Figure 2 A flowchart for identifying the ground plane in the ground in the embodiment of the application is shown in the figure. Figure 3 A flowchart for obtaining the ground plane based on the local plane in the embodiment of the application is shown in the figure. Figure 4 An embodiment of the AI-based cargo pile edge detection system in the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0019] The embodiments of the application provide an AI-based cargo pile edge detection method and system. In the specification and claims of the application and the above-mentioned drawings, the terms "first", "second", "third", "fourth" and the like (if any) are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "includes" or "has" and any variation thereof is intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0020] For the sake of understanding, the specific process of the embodiments of the application will be described below. Please refer to Figure 1 An embodiment of the AI-based cargo pile edge detection method in the embodiment of the application includes: Step S1, a radar device is carried by a UAV, a target pile area is scanned according to a preset flight path, three-dimensional point cloud data of the target pile area is obtained, the three-dimensional point cloud data includes three-dimensional coordinate information and reflection intensity information, coordinate system conversion is performed on the three-dimensional point cloud data, and then the converted three-dimensional point cloud data is denoised, and the denoised three-dimensional point cloud data is down-sampled to obtain representative three-dimensional point cloud data.

[0021] Specifically, in order to obtain accurate three-dimensional point cloud data of the target pile area, a radar device is carried by a UAV, and the target pile area is scanned according to a preset flight path. The radar device transmits radar waves at a certain frequency and receives radar wave signals reflected from the pile surface. According to the propagation time and intensity of the signals, the spatial position coordinates and reflection intensity values of each point on the pile surface are determined, so as to generate three-dimensional point cloud data containing a large number of discrete points, each point containing three-dimensional coordinate information and reflection intensity information. In order to facilitate subsequent processing, the obtained three-dimensional point cloud data is converted from the original acquisition coordinate system to a unified world coordinate system. In order to improve data quality, the converted three-dimensional point cloud data is denoised. A denoising method based on statistical analysis is used to calculate the distribution characteristics of points in the local neighborhood of each point, such as mean, variance, etc. For points that are significantly different from the distribution characteristics of most points in the neighborhood, they are judged as noise points and are removed to improve the accuracy and efficiency of subsequent processing. In order to reduce the data volume, the denoised three-dimensional point cloud data is down-sampled. For example, a voxel grid down-sampling method can be used to divide the three-dimensional space into small voxel cubes, and then a representative point is retained in each voxel cube, which is usually the centroid point or the point with the maximum reflection intensity in the voxel cube. Through down-sampling processing, the size of the point cloud data can be reduced while retaining the main shape features of the pile, improving the efficiency of subsequent processing. The finally obtained down-sampled point cloud data is referred to as representative three-dimensional point cloud data.

[0022] Step S2, ground detection is performed on the representative three-dimensional point cloud data, a plane where the ground is located is identified, and points above the ground plane in the representative three-dimensional point cloud data are divided into pile candidate regions.

[0023] Specifically, in order to identify the plane where the ground is located, ground detection is performed on the down-sampled point cloud data, and the specific method for identifying the plane where the ground is located will be explained in detail later. By effectively identifying the plane where the ground is located, accurate reference is provided for subsequent cargo pile region division. In order to preliminarily determine the cargo pile region, first, points representing points in the three-dimensional point cloud data above the ground plane are divided into a cargo pile candidate region. For example, for each point representing the three-dimensional point cloud data, the vertical distance from the point to the ground plane is calculated. If the distance is greater than a predetermined threshold, the point is considered to belong to the cargo pile candidate region. The threshold can be set according to the minimum height of the cargo pile and the actual application scenario. Usually, the minimum height value of the cargo pile is taken. Through this step, the point cloud data that may belong to the cargo pile is preliminarily screened, providing an initial region for subsequent region growing segmentation.

[0024] Step S3, selecting an arbitrary point in the cargo pile candidate region as a seed three-dimensional point, setting a belonging condition, and belonging the three-dimensional points meeting the belonging condition to the current growing cargo pile region until there is no three-dimensional point meeting the belonging condition, and finally obtaining the growing cargo pile region as the final cargo pile region.

[0025] Specifically, in order to obtain an accurate cargo pile region, an arbitrary point in the cargo pile candidate region is selected as a seed three-dimensional point. The selection of the seed three-dimensional point can also be based on the density, reflectivity, or other characteristics of the three-dimensional point cloud data. A representative point is selected as the starting point of growth. Selecting an appropriate seed point can improve the accuracy and efficiency of region growing. In order to guide region growing, a belonging condition is set, and the three-dimensional points meeting the belonging condition are belonged to the current growing cargo pile region. By setting the belonging condition, it can be ensured that the growth process is along the boundary of the cargo pile, avoiding growth to non-cargo pile regions. For each un-divided three-dimensional point, it is determined whether it meets the belonging condition. If it meets the condition, it is belonged to the current growing cargo pile region. Until there is no three-dimensional point meeting the growth rule, the final growing cargo pile region is taken as the final cargo pile region. This step dynamically adjusts the growth direction to ensure that the growth process is along the boundary of the cargo pile, improving the accuracy of cargo pile region segmentation.

[0026] Step S4, detecting the cargo pile region to obtain the contour boundary points of the cargo pile, classifying and optimizing the contour boundary points, fitting the cargo pile contour based on the optimized contour boundary points, and obtaining the accurate contour boundary line of the cargo pile.

[0027] Specifically, in order to extract the contour boundary points of the cargo pile, the final cargo pile region is detected. A normal vector based boundary detection algorithm is used to calculate the normal vector of each point. Those points with a large difference in normal vector from the surrounding points are considered as boundary points. The specific identification method of the boundary points will be explained in detail later. In order to adopt different fitting strategies in the subsequent contour fitting process, the identified contour boundary points are classified. According to the local curvature and normal vector change rate of the boundary points, the boundary points are divided into convex boundary points, concave boundary points and smooth boundary points. Through classification marking, different fitting strategies can be used for different types of boundary points to improve the quality and adaptability of contour fitting. In order to smooth the boundary and remove possible burrs and discontinuous points, the extracted boundary points are optimized. Through optimization, the position of the boundary points is adjusted to make them more smooth and continuous, while maintaining the overall shape characteristics of the cargo pile contour. According to the optimized boundary points, a parametric curve fitting method is used to fit the cargo pile contour. Select an appropriate curve model, such as a Bezier curve or a spline curve, determine the parameters of the curve model, and make the fitted curve as close as possible to the optimized boundary points, so as to obtain the accurate contour curve of the cargo pile. This step can realize accurate identification and description of irregular cargo pile contour, and improve the automation level and efficiency of cargo pile management.

[0028] In a specific embodiment, the three-dimensional point cloud data is converted in the coordinate system, specifically including the following steps: The position and attitude information of the unmanned aerial vehicle in the world coordinate system is determined, and the relative position and attitude relationship between the radar device and the unmanned aerial vehicle is obtained. A coordinate conversion matrix is established according to the position and attitude information of the unmanned aerial vehicle and the relative position and attitude relationship between the radar device and the unmanned aerial vehicle. The coordinates of each three-dimensional point cloud data are converted from the radar coordinate system to the world coordinate system through matrix operation.

[0029] Specifically, in order to facilitate subsequent processing, the three-dimensional point cloud data obtained by the above method is converted from the original acquisition coordinate system to the unified world coordinate system, ensuring the compatibility and consistency between different data, and providing a unified coordinate system for subsequent processing.

[0030] In a specific embodiment, the converted three-dimensional point cloud data is denoised, specifically including the following steps: A local neighborhood corresponding to each three-dimensional point in the three-dimensional point cloud data is constructed, the local neighborhood is a three-dimensional sphere, the number of three-dimensional points in the local neighborhood is counted, and the volume of the local neighborhood is calculated. The number of three-dimensional points divided by the volume of the local neighborhood is taken as a first characteristic value. The standard deviation of the reflection intensity of all three-dimensional points in the local neighborhood is calculated, and the standard deviation is taken as a second characteristic value. A first range and a second range are preset. The three-dimensional points whose first characteristic value is not in the first range or second characteristic value is not in the second range are determined as noise points and removed.

[0031] Specifically, in order to improve the accuracy and reliability of the cargo pile contour recognition, improve the data quality, and effectively identify and eliminate noise points, the converted three-dimensional point cloud data is subjected to denoising processing, thereby providing a higher quality data basis for subsequent cargo pile region segmentation and contour extraction, in order to analyze the local features around each point, a corresponding local neighborhood is constructed with each three-dimensional point in the three-dimensional point cloud data as the center, the local neighborhood is defined as a three-dimensional spherical region, and the radius thereof is dynamically adjusted according to the density of the point cloud data, such as first counting the average distance between all points in the point cloud data, and then setting the radius r of the local neighborhood as k x average distance, wherein k is a constant greater than 1, used to control the size of the local neighborhood, in this way, it can be ensured that the local neighborhood can cover enough adjacent points, while avoiding interference in a too large range, the first feature value reflects the density of the point cloud in the local neighborhood, and a higher density usually means that the local neighborhood belongs to the cargo pile region, and a lower density may indicate a noise point or a background region, the second feature value reflects the variation degree of the reflection intensity in the local neighborhood, a smaller standard deviation indicates that the reflection intensity is more consistent, and a larger standard deviation may mean that there is noise, in order to distinguish noise points and effective points, a first range and a second range are preset, the two ranges can be adjusted according to the characteristics of the point cloud data and the actual application scene, if the first feature value of a certain point is not within the first range, or the second feature value is not within the second range, the point is determined as a noise point and is eliminated, this method can comprehensively consider the density and reflection intensity variation of the point cloud, and effectively identify noise points that do not meet the cargo pile features.

[0032] In a specific embodiment, the plane where the ground is located is identified, specifically including the following steps: The three-dimensional point cloud data is divided according to different scales to form a plurality of small cubic grids of different scales, in each scale of the cubic grid, the neighborhood three-dimensional points in the first neighborhood of each small cubic are collected, the coordinate mean value of the neighborhood three-dimensional points is calculated, the coordinate of each three-dimensional point is subtracted from the coordinate mean value, the covariance matrix of the three-dimensional point cloud data after the coordinate mean value is subtracted is calculated, the eigenvalues and corresponding eigenvectors are obtained by eigenvalue decomposition of the covariance matrix, the eigenvector corresponding to the smallest eigenvalue is selected as the normal vector, and the local plane of the corresponding small cubic is determined based on the normal vector and any point in the small cubic, and the ground plane is obtained based on the local plane.

[0033] Specifically, in order to accurately identify the ground plane of the region where the cargo pile is located, thereby providing a reliable basis for the segmentation of the cargo pile region, such as Figure 2The flowchart shown is a ground plane for identifying the ground plane, by local plane fitting in different scale cubic grids, and gradually merging the local planes meeting the preset conditions, to finally determine the ground plane, aiming to improve the accuracy and robustness of ground detection, and ensure the accuracy of the pallet area segmentation.

[0034] To capture the characteristics of the ground plane at different scales, the point cloud data is divided into multiple small cubic grids of different scales, for example, three scales can be selected, with cubic sizes of 0.25m, 0.15m and 0.1m. This multi-scale division can adapt to the ground features in different scenes and improve the robustness of ground detection. In each scale cubic grid, for each small cubic, the neighbor 3D points in its first neighborhood are collected, and the first neighborhood can be defined as the points in the cubic adjacent to the current cubic. The coordinate mean of the neighbor 3D points is calculated, and the coordinates of each 3D point are subtracted from the coordinate mean to realize the translation of the point cloud data, so that the mean of the translated point cloud data is the origin. Through the translation operation, the subsequent covariance matrix calculation and feature decomposition process can be simplified. The covariance matrix of the 3D point cloud data after subtracting the coordinate mean is calculated. Assuming that the translated point cloud data is P={p1,p2,…,pN}, where each point , the covariance matrix is The covariance matrix C is decomposed to obtain three eigenvalues e1, e2, e3 and corresponding eigenvectors v1, v2, v3. The eigenvalue represents the degree of change of the data in the corresponding eigenvector direction, and the eigenvector corresponding to the smallest eigenvalue is selected as the normal vector, because the eigenvector corresponding to the smallest eigenvalue represents the direction with the smallest data change, i.e. the normal vector of the local plane. Based on the normal vector and any point in the small cubic, the local plane of the corresponding small cubic is determined. The plane equation is calculated, and the plane determined by this plane equation is taken as the local plane of each small cubic. The ground plane is obtained based on the local plane in the subsequent process. The specific method of obtaining the ground plane based on the local plane will be explained in detail later.

[0035] In an embodiment, the ground plane is obtained based on the local plane, specifically including the following steps: Starting from the largest scale, for each local plane of each small cubic, the normal vector angle and the plane distance between the local plane and the adjacent small cubic are calculated, and it is determined whether any two local planes meet the preset condition. If the preset condition is met, the two local planes are fitted into one local plane, and the step is repeated to merge all local planes meeting the preset condition to obtain multiple candidate planes. The candidate planes in the maximum scale and the local planes in the next scale that meet the preset condition are merged until all the local planes in the minimum scale are merged to obtain a plurality of candidate global planes, and the global plane with the largest number of three-dimensional points in the plurality of candidate global planes is taken as the final ground plane.

[0036] Specifically, as shown in Figure 3 is a flowchart for obtaining the ground plane based on the local plane, in order to gradually refine the identification of the ground plane, the merging of the local plane starts from the largest scale, at each scale, the local plane of each small cube is analyzed, for the local plane of each small cube, the normal vector angle and the plane distance between the local plane and the adjacent small cube are calculated, the calculation formula of the normal vector angle is , wherein n1 and n2 are the normal vectors of the two local planes, and the distance d between the two local planes is calculated. Assuming that the equations of the two local planes are and , the plane distance is , through the above calculation, the geometric relationship between the two local planes can be quantified, which provides a basis for subsequent merging, the preset condition is that the normal vector angle is less than a preset first threshold value and the plane distance is less than a preset second threshold value, if the two local planes meet these conditions, they are considered to belong to the same global plane and can be merged, the two local planes that meet the preset condition are fitted into a local plane, such as a new normal vector and distance parameter can be calculated by weighted average method, thereby obtaining a new local plane, the above steps are repeated, all the local planes that meet the preset condition are merged to obtain a plurality of candidate planes, through the gradual merging process, the identification of the ground plane can be gradually refined, and the detection accuracy is improved.

[0037] Starting from the cube grid in the largest scale, after the local plane fitting and merging steps, a series of candidate planes are obtained, these candidate planes are based on the merging results of the local planes in the largest scale, and can capture the approximate shape and position of the ground, these candidate planes are selected as the initial global plane set, which provides a basis for subsequent cross-scale merging, and the local planes in the cube grid in the next smaller scale are gradually considered. For each local plane in the smaller scale, the normal vector angle and the plane distance between the local plane and each global plane in the current global plane set are calculated, if the normal vector angle between the local plane and a global plane is less than a preset first threshold value and the plane distance is less than a preset second threshold value, the local plane and the global plane are considered to belong to the same ground plane and can be merged, the above cross-scale merging process is repeated, and all the local planes in the smaller scale are gradually compared and merged with the current global plane set. This gradual refinement process can make full use of the information in different scales and improve the accuracy of ground plane detection.

[0038] Assuming there are three scales in total, the candidate planes on the largest scale and the local planes on the second largest scale that meet the preset conditions are merged to determine a plurality of candidate planes, and then the plurality of candidate planes determined again and all the local planes on the smallest scale are merged to obtain a plurality of final candidate global planes.

[0039] For each candidate global plane, the number of three-dimensional points contained therein is counted, and the global plane with the largest number of three-dimensional points can generally better fit the ground point cloud data, because it contains more ground point information, and therefore, from the plurality of candidate global planes, the global plane containing the largest number of three-dimensional points is selected as the final ground plane. This selection method based on the number of point clouds can effectively avoid misjudgment and ensure the reliability and accuracy of the finally determined ground plane.

[0040] In a specific embodiment, the process of generating the pallet point cloud region specifically includes the following steps: When each three-dimensional point is classified into the current growing pallet region, a plurality of three-dimensional points most recently classified into the pallet region are selected, a covariance matrix of the selected plurality of three-dimensional points is calculated, the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and eigenvectors, the eigenvector corresponding to the largest eigenvalue is selected as the main direction of the current growing pallet region, the angles between a plurality of adjacent three-dimensional points and the main direction are also calculated, a distance threshold, a reflection intensity threshold and an angle threshold are pre-set, and the adjacent three-dimensional points that meet the classification condition are classified into the current growing pallet region, the classification condition being that the distance between the adjacent three-dimensional point and the three-dimensional point in the current growing region is less than the distance threshold, the difference between the reflection intensity and the reflection intensity in the current growing region is less than the reflection intensity threshold, and the angle between the adjacent three-dimensional point and the main direction is less than the angle threshold.

[0041] Specifically, in order to accurately identify and extract the pallet area from complex point cloud data, a plurality of three-dimensional points recently attributed to the pallet area are selected to calculate the main direction of the current growing area when attributing a three-dimensional point to the current growing pallet area each time, so as to ensure that the growing direction can be dynamically adjusted to adapt to the actual shape of the pallet. By selecting the recently attributed points, it can be ensured that the calculated main direction can reflect the latest state of the current growing area, thereby improving the accuracy and adaptability of the growing direction. A covariance matrix of the selected plurality of three-dimensional points is calculated, the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and corresponding eigenvectors. The eigenvector corresponding to the largest eigenvalue represents the direction of the largest data change, i.e. the main distribution direction of the point cloud data, which can be used as a reference for the growing direction. Therefore, the eigenvector corresponding to the largest eigenvalue is selected as the main direction of the current growing pallet area. The angles between a plurality of adjacent three-dimensional points and the main direction are calculated to filter out points with small angles with the main direction, so as to ensure that the growing process can be carried out along the actual boundary of the pallet, and to avoid growing into a non-pallet area. A distance threshold, a reflection intensity threshold and an angle threshold are preset, which are used to control the point filtering in the growing process, so as to ensure that only points meeting the conditions can be attributed to the growing area. Reasonable setting of these thresholds can effectively filter out points not meeting the pallet characteristics, thereby improving the accuracy of the growing process. For each adjacent three-dimensional point, it is checked whether it meets the attribution condition. If a point meets the above three conditions at the same time, the point is attributed to the current growing pallet area. Through the strict attribution condition filtering, it can be ensured that the growing process can accurately proceed along the actual boundary of the pallet, and the points in the non-pallet area are avoided to be attributed to the growing area.

[0042] The above steps are repeated until there is no adjacent three-dimensional point meeting the attribution condition. The finally obtained growing pallet area is taken as the final pallet area. This step-by-step growing process can gradually refine the extraction of the pallet area and improve the extraction accuracy.

[0043] In a specific embodiment, the pallet point cloud area is detected to obtain the contour boundary points of the pallet, specifically including the following steps: The normal vector of each three-dimensional point in the pallet area is calculated. The angle between the normal vector of a three-dimensional point and the three-dimensional points in its second neighborhood is calculated. If the angle is greater than a preset second threshold, the corresponding three-dimensional point is determined as a boundary point. For each boundary point, the corresponding local curvature and normal vector change rate are also calculated. A first curvature threshold C1, a second curvature threshold C2, a first normal vector change threshold N1 and a second normal vector change threshold N2 are preset. If the local curvature is greater than C1 and the normal vector change rate is greater than N1, the corresponding boundary point is classified as a convex boundary point. If the local curvature is less than C2 and the normal vector change rate is less than N2, the corresponding boundary point is classified as a concave boundary point. Otherwise, the corresponding boundary point is classified as a smooth boundary point.

[0044] Specifically, in order to accurately extract the contour boundary points of the goods stack from the goods stack point cloud region and classify the boundary points, first, the normal vector of each three-dimensional point in the goods stack region is calculated, the angle between the normal vector of each three-dimensional point and the three-dimensional points in its second neighborhood is calculated, a second threshold is preset, if the angle between the normal vector of a point and the normal vector of any point in its second neighborhood is greater than the second threshold, the point is determined as a boundary point, for each boundary point, the corresponding local curvature and normal vector change rate are also calculated, based on the local curvature and the normal vector change rate, the boundary points are classified using the above method, into convex boundary points, concave boundary points and smooth boundary points, by classifying the boundary points, in the subsequent contour fitting process, different fitting strategies are adopted according to the type of the boundary points, for convex boundary points and concave boundary points, special fitting algorithms can be used to ensure the accuracy and smoothness of the contour, and for smooth boundary points, general fitting methods can be used, thereby improving the quality and adaptability of the entire goods stack contour fitting.

[0045] Further, the normal vector of the three-dimensional point is calculated, comprising: Selecting points in its neighborhood, the definition of the neighborhood can be based on a fixed radius or a fixed number of nearest neighbors, such as selecting points with a distance less than a fixed radius from the three-dimensional point, or selecting the nearest several points from the three-dimensional point, calculating the mean of the coordinates of all points in the field, translating all points in the field based on the mean, constructing a covariance matrix of all points in the field based on the translated points, performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and eigenvectors, the eigenvalues represent the degree of change of the data in the corresponding eigenvectors, and the eigenvector corresponding to the smallest eigenvalue usually represents the direction with the smallest data change, i.e. the normal vector, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the corresponding three-dimensional point.

[0046] Further, the local curvature is calculated, comprising: For each boundary point Q, selecting the two points Q1 and Q2 closest to the boundary point, referred to as the first point and the second point, respectively, calculating the first vector V1 = Q1-Q and the second vector V2 = Q-Q1, calculating the angle θ1 between the first vector and the second vector, and calculating the local curvature b by the product of the angle and the lengths of the two vectors, the two vectors being the first vector V1 and the second vector V2, the formula for calculating the local curvature b is: , denotes the modulus of the vector.

[0047] Further, the normal vector change rate is calculated, comprising: For each boundary point Q, the normal vectors M1 and M2 of the two points Q1 and Q2 closest to the boundary point are obtained, and the formula for calculating the normal vector change rate r is: .

[0048] In a specific embodiment, the contour boundary points are optimized, specifically including the following steps: The mobile least square method is used to locally fit the boundary points, and the positions of the contour boundary points are adjusted.

[0049] Specifically, in order to improve the smoothness and accuracy of the pallet contour boundary points, the present application selects to use the mobile least square method to locally fit and adjust the positions of the boundary points, which aims to reduce noise and irregularities through local fitting while maintaining the overall shape features of the pallet contour, providing a higher quality data basis for subsequent contour fitting and analysis.

[0050] The above describes the AI-based pallet edge detection method in the embodiments of the present application, and the following describes the AI-based pallet edge detection system in the embodiments of the present application, please refer to Figure 4 An embodiment of the AI-based pallet edge detection system in the embodiments of the present application includes: The preprocessing module is configured to use a drone carrying a radar device to scan a target pallet area according to a preset flight path, obtain three-dimensional point cloud data of the target pallet area, the three-dimensional point cloud data including three-dimensional coordinate information and reflection intensity information, perform coordinate system conversion on the three-dimensional point cloud data, and then perform denoising processing on the converted three-dimensional point cloud data, and perform downsampling processing on the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data. The first detection module is configured to perform ground detection on the representative three-dimensional point cloud data, identify a ground plane where the ground is located, and divide points in the representative three-dimensional point cloud data located above the ground plane into a pallet candidate area. The second detection module is configured to select a certain point from the pallet candidate area as a seed three-dimensional point, set a belonging condition, and belong three-dimensional points meeting the belonging condition to a current growing pallet area until there is no three-dimensional point meeting a growing rule, and finally obtain a growing pallet area as a final pallet area. The boundary determination module is configured to detect the pallet point cloud area to obtain contour boundary points of the pallet, classify the contour boundary points, optimize the contour boundary points, fit the pallet contour based on the optimized boundary points, and obtain an accurate contour boundary line of the pallet.

[0051] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0052] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0053] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. The AI-based cargo stack edge detection method is characterized by: The method comprises: Step S1: Scanning a target cargo stack area using a radar device mounted on an unmanned aerial vehicle (UAV) along a preset flight path to obtain three-dimensional point cloud data of the target cargo stack area, the three-dimensional point cloud data including three-dimensional coordinate information and reflection intensity information, performing coordinate system conversion on the three-dimensional point cloud data, then performing denoising on the converted three-dimensional point cloud data, and downsampling the denoised three-dimensional point cloud data to obtain representative three-dimensional point cloud data; Step S2: performing ground detection on the representative three-dimensional point cloud data to identify the plane where the ground is located, and dividing the points in the representative three-dimensional point cloud data that are above the ground plane into candidate cargo stack areas; Step S3: Select any point from the candidate cargo stack area as a seed 3D point, set inclusion conditions, and include the 3D points that meet the inclusion conditions into the current growing cargo stack area until there are no more 3D points that meet the inclusion conditions. The resulting growing cargo stack area is used as the final cargo stack area. Step S4: Detect the cargo stack area to obtain the contour boundary points of the cargo stack, classify and optimize the contour boundary points, and fit the cargo stack contour based on the optimized contour boundary points to obtain the precise contour boundary line of the cargo stack.

2. The method according to claim 1, characterized in that Perform coordinate system conversion on 3D point cloud data, including: Determine the position and attitude information of the UAV in the world coordinate system, and also obtain the relative position and attitude relationship between the radar device and the UAV. Establish a coordinate transformation matrix based on the UAV position, attitude information and the relative position and attitude relationship between the radar device and the UAV. Through matrix operations, transform the coordinates of each three-dimensional point cloud data from the radar coordinate system to the world coordinate system.

3. The method according to claim 1, characterized in that The converted 3D point cloud data is subjected to denoising processing, including: With each 3D point in the 3D point cloud data as the center, a corresponding local neighborhood is constructed. The local neighborhood is a 3D sphere. The number of 3D points in the local neighborhood is counted, and the volume of the local neighborhood is calculated. The result value of the number of 3D points divided by the volume of the local neighborhood is used as the first eigenvalue. The standard deviation of the reflection intensity of all 3D points in the local neighborhood is calculated, and the standard deviation is used as the second eigenvalue. The first range and the second range are preset, and the 3D points whose first eigenvalue is not in the first range or the second eigenvalue is not in the second range are judged as noise points and eliminated.

4. The method according to claim 1, wherein Identify the ground plane on which the ground lies, including: The three-dimensional point cloud data is divided into multiple small cube grids of different scales according to different scales. In the cube grid of each scale, the neighborhood three-dimensional points in the first neighborhood of each small cube are collected, and the coordinate mean of the neighborhood three-dimensional points is calculated. The coordinate mean is subtracted from the coordinate of each three-dimensional point, and the covariance matrix of the three-dimensional point cloud data after subtracting the coordinate mean is calculated. The covariance matrix is ​​eigendecomposed to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the smallest eigenvalue is selected as the normal vector. The local plane of the corresponding small cube is determined based on the normal vector and any point in the small cube, and the ground plane is obtained based on the local plane.

5. The method according to claim 4, characterized in that Get the ground plane based on the local plane, including: Starting from the largest scale, at each scale, for each local plane of a small cube, calculate the normal vector angle and plane distance between the local planes of the adjacent small cubes, and determine whether any two local planes meet the preset conditions. If they meet the preset conditions, the preset conditions are that the normal vector angle is less than a preset first threshold and the plane distance is less than a preset second threshold, fit the two local planes into one local plane, repeat this step, and merge all local planes that meet the preset conditions to obtain multiple candidate planes; The candidate planes at the maximum scale and the local planes at the next scale that meet the preset conditions are merged until all the local planes at the minimum scale are merged to obtain the final multiple candidate global planes. The global plane with the largest number of 3D points among the multiple candidate global planes is used as the final ground plane.

6. The method according to claim 1, characterized in that The process of generating the point cloud area of ​​the cargo stack includes: Each time a three-dimensional point is included in the current growing cargo stack area, several three-dimensional points that are most recently included in the cargo stack area are selected, the covariance matrix of the selected three-dimensional points is calculated, the covariance matrix is ​​eigendecomposed to obtain eigenvalues ​​and eigenvectors, the eigenvector corresponding to the largest eigenvalue is selected as the main direction of the current growing cargo stack area, and the angles between several adjacent three-dimensional points and the main direction are calculated. The distance threshold, reflection intensity threshold and angle threshold are set in advance, and the adjacent three-dimensional points that meet the inclusion conditions are included in the current growing cargo stack area. The inclusion conditions are that the distance between the adjacent three-dimensional point and the three-dimensional point in the current growing area is less than the distance threshold, the difference between the reflection intensity and the reflection intensity in the current growing area is less than the reflection intensity threshold, and the angle between the adjacent three-dimensional point and the main direction is less than the angle threshold.

7. The method according to claim 1, characterized in that Detect the point cloud area of ​​the cargo stack to obtain the contour boundary points of the cargo stack, including: Calculate the normal vector of each three-dimensional point in the cargo stack area, and calculate the angle between the normal vector of the three-dimensional point and the three-dimensional points in its second neighborhood. If the angle is greater than a preset second threshold, the corresponding three-dimensional point is determined to be a boundary point. For each boundary point, the corresponding local curvature and normal vector change rate are also calculated. The first curvature threshold C1, the second curvature threshold C2, the first normal vector change threshold N1 and the second normal vector change threshold N2 are preset. If the local curvature is greater than C1 and the normal vector change rate is greater than N1, the corresponding boundary point is classified as a convex boundary point. If the local curvature is less than C2 and the normal vector change rate is less than N2, the corresponding boundary point is classified as a concave boundary point. Otherwise, the corresponding boundary point is classified as a smooth boundary point.

8. The method according to claim 1, characterized in that The contour boundary points are optimized, including: using a moving least squares method to perform local fitting on the contour boundary points and adjusting the positions of the contour boundary points.

9. An AI-based cargo stack edge detection system, configured to implement the AI-based cargo stack edge detection method according to any one of claims 1 to 8, characterized in that: The system comprises: A preprocessing module is used to use a drone equipped with a radar device to scan the target cargo stack area according to a preset flight path to obtain 3D point cloud data of the target cargo stack area. The 3D point cloud data includes 3D coordinate information and reflection intensity information. The 3D point cloud data is subjected to coordinate system conversion, and then the converted 3D point cloud data is subjected to denoising processing. The denoised 3D point cloud data is then downsampled to obtain representative 3D point cloud data. The first detection module is used to perform ground detection on the three-dimensional point cloud data, identify the plane where the ground is located, and divide the points above the ground plane in the three-dimensional point cloud data into candidate areas for cargo stacks; The second detection module is used to select any point from the candidate cargo stack area as a seed 3D point, set a classification condition, and classify the 3D points that meet the classification condition into the current growing cargo stack area until there are no more 3D points that meet the classification condition, and the finally obtained growing cargo stack area is used as the final cargo stack area; The boundary determination module is used to detect the cargo stack area to obtain the contour boundary points of the cargo stack, classify and optimize the contour boundary points, fit the cargo stack contour based on the optimized contour boundary points, and obtain the precise contour boundary line of the cargo stack.

Images