A three-dimensional target detection method based on optimal observation surface classification
By segmenting and clustering lidar point clouds and constructing observation surfaces, combined with a lightweight classification network and likelihood estimation, the robustness and real-time performance issues of 3D target detection in complex environments are solved, achieving efficient target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2024-04-01
- Publication Date
- 2026-07-07
AI Technical Summary
Existing 3D target detection technologies suffer from poor robustness, weak generalization ability, and low real-time performance in various complex environments. Camera-LiDAR fusion methods involve large computational loads and are also affected by their robustness.
By segmenting and clustering lidar point clouds, optimal and suboptimal observation coordinate systems are constructed, and point clouds are mapped to two-dimensional observation surfaces for classification. By combining a lightweight classification network and similarity estimation, ground point cloud interference is eliminated, thereby improving detection accuracy and real-time performance.
It improves the robustness and accuracy of 3D target detection, adapts to different scenarios and environments, reduces computational load, and improves real-time performance and detection accuracy.
Smart Images
Figure CN118429956B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of autonomous driving and mobile robot environmental perception technology, and more specifically, relates to a three-dimensional target detection method based on optimal observation surface classification. Background Technology
[0002] In the fields of autonomous driving and mobile robotics, one or more of visual sensors and LiDAR are often used as data sources for the development and application of algorithms for practical 3D target detection.
[0003] Image-based 3D target detection methods using cameras (monocular, binocular, or depth) as a single sensor are greatly affected by ambient lighting, making them unsuitable for low-light conditions. Furthermore, they struggle to accurately detect depth information in outdoor scenes, leading to significant target position errors and unreliable detection results. 3D target detection methods using lidar (3D) as a single sensor are unaffected by ambient lighting and offer high ranging accuracy. However, current general-purpose point cloud 3D target detection networks suffer from high computational costs, weak generalization ability, and significant differences between training and testing data that drastically reduce the prediction accuracy of the 3D detector. Additionally, the general-purpose network may lose detection of most targets when the lidar experiences large pitch angle changes. Combining cameras and lidar can improve the accuracy of 3D target detection to some extent, but the accuracy reduction due to data discrepancies persists. The camera-lidar coupling affects the system's robustness; interference with either sensor can degrade target detection performance. Moreover, increasing the data source increases the network's computational load, further reducing the real-time performance of the detection network.
[0004] It is evident that existing 3D target detection technologies suffer from poor robustness, weak generalization ability, and low real-time performance in various complex environments. Summary of the Invention
[0005] To address the shortcomings and improvement needs of existing technologies, this invention provides a three-dimensional target detection method based on optimal observation surface classification, aiming to improve the robustness of three-dimensional target detection in autonomous vehicles and mobile robots under different complex environments.
[0006] To achieve the above objectives, according to one aspect of the present invention, a three-dimensional target detection method based on optimal observation surface classification is provided, comprising:
[0007] A frame of 3D point cloud acquired by lidar is segmented and clustered to obtain the clustered target;
[0008] Determine the orientation vector of each cluster target in the spatial horizontal plane, determine an optimal observation point on the straight line axis that is perpendicular to the orientation vector in the spatial horizontal plane and passes through the average point of the cluster target, and construct an optimal observation coordinate system with the optimal observation point as the origin and the direction from the optimal observation point to the average point as the positive x-axis;
[0009] The point cloud of each cluster target is transformed from the current coordinate system to the corresponding optimal observation coordinate system. The horizontal yaw angle range and vertical pitch angle range of the cluster target point cloud in the optimal observation coordinate system are mapped to the yz plane of the optimal observation coordinate system to obtain the dimensions in the corresponding two directions, which are used as the virtual width and height of the cluster target. According to the virtual width and height and the required classification sample size, the point cloud of the cluster target located in the optimal observation coordinate system is scaled and mapped to a surface located on the yz plane that is the same size as the required classification sample size to obtain the optimal observation surface. The pixel value of each pixel is the Euclidean distance from the corresponding point of the cluster target to the origin of the optimal observation coordinate system.
[0010] The target category of a cluster is obtained by classifying the target based on the best observation surface of each cluster target.
[0011] Furthermore, before performing segmentation clustering, the method also includes:
[0012] A frame of original 3D point cloud acquired by lidar is filtered to remove ground point cloud data, resulting in a processed frame of 3D point cloud.
[0013] Then, the processed 3D point cloud is segmented and clustered to obtain the clustering target.
[0014] Furthermore, the ground point cloud filtering is implemented as follows: a frame of original 3D point cloud acquired by lidar is divided into planar grids; ground point clouds are identified and removed from each grid.
[0015] The implementation method for planar grid division is as follows:
[0016] On the xy-horizontal plane of the current coordinate system, with the origin of the current coordinate system as the center, N annular regions are divided according to a preset N radial distance. Based on a preset M sector angle, the circular planar region centered on the origin of the current coordinate system and divided into N annular regions is further divided into M sectors, thus obtaining a grid plane. The values of the preset N radial distances and / or the preset M sector angles satisfy the following: the area of the divided grid unit increases with the increase of the radial distance.
[0017] The points in the original 3D point cloud frame are mapped onto the grid plane to achieve the allocation of the original 3D point cloud frame in the grid and complete the planar grid division.
[0018] Furthermore, the method for identifying and removing ground point clouds from each grid cell is as follows:
[0019] From the original 3D point cloud corresponding to each grid, select the n points with the lowest vertical z-direction height values as the initial grounding point cloud, calculate the covariance matrix of the initial grounding point cloud and perform SVD decomposition on it to obtain the feature vectors on the x, y and z axes of the lidar coordinate system.
[0020] The feature vectors on the three axes are added together. It is then determined whether the angle between the added feature vector and the current coordinate system xy horizontal plane is less than a threshold. If so, the initial ground point cloud is determined to be true. The ground point cloud is expanded, and the feature vector calculation and angle determination are re-executed until the angle determination result is negative. At this point, the ground point cloud before the most recent expansion is taken as the ground point cloud in the original 3D point cloud corresponding to the current grid. If not, the initial ground point cloud is determined to be false, and there is no ground point cloud in the original 3D point cloud corresponding to the current grid.
[0021] The identified ground point cloud is deleted from the original 3D point cloud frame.
[0022] Furthermore, the implementation method for expanding the grounding point cloud is as follows:
[0023] Calculate the average three-dimensional coordinates of the current grounding point cloud to obtain the average point; this average point and the origin of the lidar coordinate system form the origin-average point vector, and calculate the vector magnitude after projecting this vector onto the z-axis of the current coordinate system, which is used as the extended tolerance threshold.
[0024] Calculate the difference between the z-coordinate value of each point in the original 3D point cloud (excluding the current ground point cloud) and the average point, and add all points whose difference is less than the expansion tolerance threshold to the current ground point cloud to complete the expansion of the current ground point cloud.
[0025] Furthermore, the orientation vector is determined as follows:
[0026] Calculate the covariance matrix of the point cloud corresponding to each cluster target, calculate the eigenvectors of the covariance matrix on the x, y, and z axes of the lidar coordinate system, add the eigenvectors on the three axes, and project them onto the xy horizontal plane of the lidar coordinate system to obtain the orientation vector of the corresponding cluster target on the spatial horizontal plane.
[0027] Furthermore, the distance between the best observation point of each cluster target and the average point of that cluster target is the same as the radial distance between the average point and the origin of the current coordinate system.
[0028] Furthermore, it also includes:
[0029] A suboptimal observation point is determined on a straight line axis located in the horizontal space that passes through the average point of each cluster target and is oriented in the same direction as the orientation vector. A suboptimal observation coordinate system is constructed with the suboptimal observation point as the origin and the direction from the suboptimal observation point to the average point as the positive x-axis.
[0030] The point cloud of each cluster target is transformed from the current coordinate system to the corresponding suboptimal observation coordinate system. The horizontal yaw angle range and vertical pitch angle range of the cluster target point cloud in the suboptimal observation coordinate system are mapped to the yz plane of the suboptimal observation coordinate system to obtain the dimensions in the corresponding two directions, which are used as the virtual width and height of the cluster target. According to the virtual width and height and the required classification sample size, the point cloud of the cluster target located in the suboptimal observation coordinate system is scaled and mapped to a surface located in the yz plane of the suboptimal observation coordinate system that is the same size as the required classification sample size to obtain the suboptimal observation surface. The pixel value of each pixel is the Euclidean distance from the corresponding point of the cluster target to the origin of the suboptimal observation coordinate system.
[0031] Then, based on the best and second-best observation surfaces of each cluster target, target classification is performed to obtain the target category of that cluster target.
[0032] Furthermore, it also includes: performing similarity estimation tests on the classified targets to obtain the final detection results.
[0033] The present invention also provides a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed by a processor, controls the device containing the storage medium to perform a three-dimensional target detection method based on optimal observation surface classification as described above.
[0034] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:
[0035] (1) This invention proposes constructing an optimal observation surface for clustered targets, using the optimal observation surface as the classification basis. The method used to extract targets from point clouds is a clustering method. The clustered targets are mostly individual objects and do not contain scene environment information. Based on the point cloud of clustered targets, the optimal observation surface is constructed according to the shape and contour information of the clustered targets. The constructed optimal observation surface is only related to the clustered targets and is not related to the environmental point cloud. The constructed optimal observation surface is used for target classification, and the classification accuracy is high. Therefore, the method of this invention can greatly improve the adaptability of the detection algorithm to different scenes and environments, and improve the robustness of target detection. Among them, when constructing the optimal observation surface, the three-dimensional target point cloud is reconstructed into two-dimensional image features, which realizes data dimensionality reduction while retaining the significant distinguishable features of the target, thereby improving the real-time performance and accuracy of target detection.
[0036] (2) Preferably, the method of the present invention first filters and removes the ground point cloud in the original point cloud of the lidar, which can reduce the interference of the ground on the target detection algorithm in different scenarios and further improve the robustness of target detection.
[0037] (3) This invention proposes to identify ground point clouds by adding the feature vectors on the three axes and judging whether the current ground point cloud is real based on the angle between the added feature vector and the xy horizontal plane of the lidar coordinate system. This is based on the following considerations: the feature vectors on the three axes basically correspond to the change trend of the original three-dimensional point cloud corresponding to the current grid on the x, y, and z axes in lidar coordinates, reflecting the magnitude of change in different directions. Therefore, the sum of their vectors indicates the transformation direction and trend of the overall point cloud corresponding to the current grid. When the transformation direction is basically parallel to the ground, that is, whether it is basically parallel is judged according to the threshold, it can be approximately considered that the point cloud has ground point cloud features and is regarded as a ground point cloud to ensure the robustness of target detection.
[0038] (4) The present invention also proposes to construct a suboptimal observation surface in another direction from the target. The construction method of the suboptimal observation surface is the same as that of the optimal observation surface. The three-dimensional target point cloud is reconstructed into two-dimensional image features, which realizes data dimensionality reduction while retaining the significant distinguishable features of the target. At the same time, target classification is realized based on the optimal and suboptimal observation surfaces, which can further improve the accuracy of target detection. Attached Figure Description
[0039] Figure 1 A flowchart of a three-dimensional target detection method based on optimal observation surface classification is provided in an embodiment of the present invention;
[0040] Figure 2 This is a schematic diagram illustrating the results of constructing the best and second-best observation surfaces for point clouds of cars, pedestrians, and bicycles, as provided in an embodiment of the present invention.
[0041] Figure 3 A flowchart of similarity estimation provided for embodiments of the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0043] Example 1
[0044] A 3D target detection method based on optimal observation surface classification, such as Figure 1 As shown, it includes:
[0045] A frame of 3D point cloud acquired by lidar is segmented and clustered to obtain the clustered target;
[0046] Determine the orientation vector of each cluster target in the spatial horizontal plane, determine an optimal observation point on the straight line axis that is perpendicular to the orientation vector in the spatial horizontal plane and passes through the average point of the cluster target, and construct an optimal observation coordinate system with the optimal observation point as the origin and the direction from the optimal observation point to the average point as the positive x-axis;
[0047] The point cloud of each cluster target is transformed from the current coordinate system to the corresponding optimal observation coordinate system. The horizontal yaw angle range and vertical pitch angle range of the cluster target point cloud in the optimal observation coordinate system are mapped to the yz plane of the optimal observation coordinate system to obtain the dimensions in the corresponding two directions, which are used as the virtual width and height of the cluster target. According to the virtual width and height and the required classification sample size, the point cloud of the cluster target located in the optimal observation coordinate system is scaled and mapped to a surface located on the yz plane that is the same size as the required classification sample size to obtain the optimal observation surface. The pixel value of each pixel is the Euclidean distance from the corresponding point of the cluster target to the origin of the optimal observation coordinate system.
[0048] The target category of a cluster is obtained by classifying the target based on the best observation surface of each cluster target.
[0049] Determining the optimal observation point involves determining the linear axis. In practice, the angle between the orientation vector and the positive x-axis of the current coordinate system can be calculated as the orientation angle from the bird's-eye view of the current cluster target. The slope of the line is then calculated based on this orientation angle, and a line with this slope is constructed. An optimal observation point is then determined on this line. For example, the angle between the orientation vector of the cluster target and the positive x-axis of the lidar coordinate system (i.e., the forward direction of the lidar) can be calculated as the orientation angle of the current cluster target, denoted by θ. f This means that θ f It can be calculated using the following formula: In the formula, β(1) represents the component of the clustered target orientation vector on the y-axis of the lidar coordinate system, and β(0) represents the component of the clustered target orientation vector on the x-axis of the lidar coordinate system.
[0050] The optimal observation surface is defined as a two-dimensional grayscale image with fixed width and height, used to carry the clustering targets projected onto it. The fixed width and height values are related to the image classification network used subsequently, and are generally set to the size of the input image of the classification network. The optimal observation surface lies on the yz plane of the optimal observation coordinate system, and its center position is indeterminate, to be determined later by the clustering targets.
[0051] Projecting the clustered target onto the optimal observation surface: Calculate the horizontal yaw angle range and vertical pitch angle range of the clustered target point cloud. These angle ranges characterize the three-dimensional size of the clustered target. Select the point with the median horizontal yaw angle and median vertical pitch angle in the clustered target point cloud as the clustered target center point. This center point has a three-dimensional coordinate value in the optimal observation coordinate system. Use its y and z values as the center position of the optimal observation surface, thus determining the position of the optimal observation surface. Since the width and height of the optimal observation surface are fixed, there are two projection strategies when projecting the clustered target onto it: one is to project using the y and z coordinate information of the clustered target point cloud, and the other is to project using the horizontal yaw angle and vertical pitch angle of the point calculated previously. In this method, the horizontal yaw angle and vertical pitch angle of the point are used for projection. The y-z coordinates of points are not used for projection because this projection method suffers from inconsistencies in representing objects at different distances. For clusters with greater radial distances, the point cloud is sparse, resulting in a scattered and discontinuous shape projected onto the optimal observation surface. Conversely, for clusters with closer radial distances, the point cloud is dense, resulting in a dense and continuous shape projected onto the optimal observation surface. This method aims to ensure that distant clusters have a relatively dense shape after projection, while closer clusters have a relatively sparse shape. Therefore, the horizontal yaw angle and vertical pitch angle of the points are used for projection. This reduces the angular difference between two points with significantly different y-z coordinates when projecting distant clusters, while increasing the angular difference between two points with similar y-z coordinates when projecting closer clusters. Following the previous step, the horizontal yaw angle range and vertical pitch angle range are mapped to the optimal observation plane width and height dimensions. This is achieved by dividing the horizontal yaw angle range by the optimal observation plane width and the vertical pitch angle range by the optimal observation plane height, yielding the width and height scaling factors for mapping the clustered target onto the optimal observation plane. To ensure the object shape remains unchanged, the width and height scaling factors should ideally be consistent; therefore, we take the smaller of the width and height scaling factors as the actual scaling factor used. Mapping the clustered target point cloud using these scaling factors yields the final optimal observation plane grayscale image.
[0052] This method proposes to construct the optimal observation surface for clustering targets and use the optimal observation surface as the classification basis. The classification basis construction method proposed in this method is only related to the clustering targets and is not related to the environmental point cloud. Moreover, the construction of the optimal observation surface depends on the shape and contour information of the clustering targets, and the classification accuracy is high after classification.
[0053] Because the method used for target extraction is clustering, the clustered targets are mostly individual objects and do not contain scene environment information. Furthermore, the target classification uses a best observation surface-based classification method, which is only related to the clustered targets and not to the scene point cloud. Therefore, this effectively improves the detection algorithm's adaptability to different scenes and environments, enhancing the robustness of target detection. By constructing the best observation surface, the 3D target point cloud is reconstructed into 2D image features, achieving data dimensionality reduction while preserving the target's significant distinguishable features, thus improving the real-time performance and accuracy of target detection.
[0054] The method involves segmentation clustering; for example, dynamic Euclidean clustering can be used to perform the segmentation clustering described above. Additionally, the method involves target classification; for example, a lightweight classification network, such as MobileNetV3, can be used for classification, which improves the classification speed while maintaining classification accuracy.
[0055] As a preferred embodiment, the method further includes the following steps before performing segmentation clustering:
[0056] A frame of original 3D point cloud acquired by lidar is filtered to remove ground point cloud data, resulting in a processed frame of 3D point cloud.
[0057] Then, the processed 3D point cloud is segmented and clustered to obtain the clustering target.
[0058] The method first filters and removes ground point clouds from the original point cloud of the LiDAR, which can reduce the interference of ground on the target detection algorithm in different scenarios and improve the robustness of target detection.
[0059] As a preferred implementation, the above-mentioned ground point cloud filtering is achieved by: dividing a frame of original 3D point cloud acquired by lidar into planar grids; identifying and removing ground point clouds from each grid; wherein, the planar grid division is implemented as follows:
[0060] On the xy-horizontal plane of the current coordinate system, with the origin of the current coordinate system as the center, N annular regions are divided according to a preset N radial distance. According to a preset M sector angle, the circular planar region divided into N annular regions with the origin of the current coordinate system as the center is divided into M sectors, thereby obtaining a grid plane. The values of the preset N radial distances and / or the values of the preset M sector angles satisfy the following: the area of the divided grid cells increases with the increase of the radial distance.
[0061] The points in the original 3D point cloud frame are mapped onto the grid plane to achieve the allocation of the original 3D point cloud frame in the grid and complete the planar grid division.
[0062] By dividing the ground plane into grids, ground point clouds are filtered within a small area of each grid. Since the real ground is often uneven, a single-plane model is insufficient to represent the real ground. Therefore, the ground plane is divided into grids, and the entire ground plane is split into multiple grid sub-planes for analysis. This makes the algorithm model more closely resemble the actual ground model, improving the overall ground point cloud recognition accuracy. In addition, the area of the divided grid unit increases with distance, fully considering the distribution characteristics of laser point clouds, which are denser closer and sparser farther away, thus improving the recognition accuracy and efficiency of subsequent ground point clouds.
[0063] For example, firstly, an xy circular plane with a radius of approximately 80m is constructed in the lidar coordinate system; then, according to the preset radial distance, the circular plane is divided into sixteen annular regions from near to far, wherein the radial length of the first four annular regions is 2.5m / region, the radial length of the fifth to eighth annular regions is 4.0m / region, the radial length of the ninth to twelfth annular regions is 6.0m / region, and the radial length of the thirteenth to sixteenth annular regions is 8.0m / region; furthermore, the circular plane is divided into sixteen sectors, and the angle range of each sector is 22.5° / sector.
[0064] It should be noted that the current coordinate system is generally the lidar coordinate system, which is a spatial coordinate system with the lidar's mechanical center as the origin and the lidar's forward direction as the positive x-axis.
[0065] Furthermore, the above-mentioned mapping of points in a frame of original 3D point cloud onto a grid plane, realizing the allocation of a frame of original 3D point cloud in the grid, can be implemented as follows:
[0066] Calculate the radial distance of each point in a frame of the original 3D point cloud from the origin of the current coordinate system when mapped onto the horizontal plane; assign the point cloud to the corresponding vertical region in the plane according to the magnitude of the radial distance; calculate the horizontal yaw angle of the point cloud on the horizontal plane, and assign the point cloud to the corresponding sector in the vertical region according to the magnitude of the horizontal yaw angle.
[0067] As a preferred implementation, taking the current coordinate system as the lidar coordinate system as an example, the method for filtering and removing ground point clouds is as follows:
[0068] (1) Perform the following steps on the original 3D point cloud corresponding to each grid:
[0069] From the original 3D point cloud corresponding to each grid, select the n points with the lowest vertical z-direction height values as the initial grounding point cloud, calculate the covariance matrix of the initial grounding point cloud and perform SVD decomposition on it to obtain the feature vectors on the x, y and z axes of the lidar coordinate system.
[0070] The feature vectors on the three axes are added together. It is then determined whether the angle between the added feature vector and the current coordinate system xy horizontal plane is less than a threshold. If so, the initial ground point cloud is determined to be true. The ground point cloud is expanded, and the feature vector calculation and angle determination are re-executed until the angle determination result is negative. At this point, the ground point cloud before the most recent expansion is taken as the ground point cloud in the original 3D point cloud corresponding to the current grid. If not, the initial ground point cloud is determined to be false, and there is no ground point cloud in the original 3D point cloud corresponding to the current grid.
[0071] (2) Summarize the ground point clouds identified from the original 3D point clouds corresponding to each grid, and delete them from the original 3D point cloud of the frame.
[0072] In this method, the feature vectors on the three axes are added together. The angle between the added feature vector and the xy-horizontal plane of the lidar coordinate system is used to determine whether the current ground point cloud is real. This is based on the following considerations: the feature vectors on the three axes basically correspond to the changing trends of the original 3D point cloud corresponding to the current grid on the x, y, and z axes in lidar coordinates, reflecting the magnitude of the changes in different directions. Therefore, the sum of their vectors indicates the transformation direction and trend of the overall point cloud corresponding to the current grid. When the transformation direction is basically parallel to the ground, that is, when the basic parallelism is judged based on the threshold, the point cloud can be approximately considered to have ground point cloud characteristics and is regarded as a ground point cloud. The above threshold is determined according to the actual accuracy requirements, for example, it can be set to 30°.
[0073] As a preferred implementation method, the extended grounding point cloud can be implemented as follows:
[0074] Calculate the average three-dimensional coordinates of the current grounding point cloud to obtain the average point; this average point and the origin of the lidar coordinate system form the origin-average point vector, and calculate the vector magnitude after projecting this vector onto the z-axis of the current coordinate system, which is used as the extended tolerance threshold.
[0075] Calculate the difference between the z-coordinate values of each point in the original 3D point cloud (excluding the current ground point cloud) and the average point. Add all points whose difference is less than the expansion tolerance threshold to the current ground point cloud to complete the expansion of the current ground point cloud.
[0076] As a preferred embodiment, the orientation vector is determined as follows:
[0077] Calculate the covariance matrix of the point cloud corresponding to each cluster target, calculate the eigenvectors of the covariance matrix on the x, y, and z axes of the lidar coordinate system, add the eigenvectors on the three axes, and project them onto the xy horizontal plane of the lidar coordinate system to obtain the orientation vector of the corresponding cluster target on the spatial horizontal plane.
[0078] This orientation vector determination scheme aims to accurately capture the primary orientation of clustered targets in 3D space, providing crucial information for constructing the optimal observation surface. By calculating the covariance matrix of the clustered target point cloud, eigenvectors reflecting the distribution characteristics of the point cloud in three main directions can be obtained. These eigenvectors reveal the extension and clustering trends of the point cloud in space, thus helping to identify the primary orientation of the targets. The scheme design also considers computational simplicity and robustness. By adding and projecting the eigenvectors, the orientation can be determined in a relatively simple and stable manner, allowing the algorithm to maintain high reliability even in the face of noise and data incompleteness.
[0079] To maintain the spatial distribution characteristics of each cluster target, the Euclidean distance of the average point of the cluster target in the lidar coordinate system is calculated. When transforming to a new coordinate system, the spatial distribution of the cluster target is kept unchanged by translating by a considerable distance. Therefore, as a preferred implementation, the distance between the best observation point of each cluster target and the average point of the cluster target is the same as the radial distance between the average point and the origin of the current coordinate system.
[0080] As a preferred embodiment, it may also include:
[0081] A suboptimal observation point is determined on a straight line axis located in the horizontal space that passes through the average point of each cluster target and is oriented in the same direction as the orientation vector. A suboptimal observation coordinate system is constructed with the suboptimal observation point as the origin and the direction from the suboptimal observation point to the average point as the positive x-axis.
[0082] The point cloud of each cluster target is transformed from the current coordinate system to the corresponding suboptimal observation coordinate system. The horizontal yaw angle range and vertical pitch angle range of the cluster target point cloud in the suboptimal observation coordinate system are mapped to the yz plane of the suboptimal observation coordinate system to obtain the dimensions in the corresponding two directions, which are used as the virtual width and height of the cluster target. According to the virtual width and height and the required classification sample size, the point cloud of the cluster target located in the suboptimal observation coordinate system is scaled and mapped to a surface located in the yz plane of the suboptimal observation coordinate system that is the same size as the required classification sample size to obtain the suboptimal observation surface. The pixel value of each pixel is the Euclidean distance from the corresponding point of the cluster target to the origin of the suboptimal observation coordinate system.
[0083] Then, based on the best and second-best observation surfaces of each cluster target, target classification is performed to obtain the target category of that cluster target.
[0084] In other words, a suboptimal observation coordinate system is constructed for the clustered targets. This coordinate system has the suboptimal observation point as the origin and the suboptimal observation angle as the forward coordinate axis. The suboptimal observation surface is then constructed using the optimal observation surface construction method to obtain the suboptimal observation surface of the target. Schematic diagrams illustrating the construction of optimal and suboptimal observation surfaces for targets such as cars, pedestrians, and bicycles are shown below. Figure 2 As shown.
[0085] In the actual construction of the optimal and suboptimal observation surfaces, the optimal and suboptimal observation angles can be calculated based on the orientation vector. These two angles are then used to determine the positive x-axis of the optimal and suboptimal observation coordinate systems. Specifically, for example, the average point of the clustering target can be represented as p. c =(x c ,y c ,z c Define the angle θ between the vector of the average point and the origin of the lidar coordinate system and the positive direction of the horizontal axis. l Define the horizontal orientation vector β of the clustering target. 3×1 With average point p c The angle between the vector of the line connecting the origin and the origin is θ. init , then θ l With θ init The following formula is used to calculate:
[0086]
[0087]
[0088] The optimal observation angle is defined as θ. The optimal observation angle can be calculated based on the angle quadrant of the clustered target in the lidar coordinate system.
[0089]
[0090] Define the rotation matrix corresponding to the optimal observation angle as a 3x3 matrix. Given θ, the rotation matrix R required to rotate the target point cloud to the optimal observation angle can be calculated:
[0091]
[0092] Given the optimal observation angle θ, and defining the second-best observation angle as θ+90°, the rotation matrix R1 corresponding to rotating the target point cloud to the second-best observation angle can be calculated:
[0093]
[0094] When classifying targets based on the best and second-best observation surfaces for each clustering objective, considering that deep learning classification networks generally have a fixed input image size, various image processing methods are used as inputs to the classification network for training and prediction. This embodiment provides two input image organization formats, specifically including:
[0095] (1) Arrange the best and second-best observation surfaces to form a single image: Place the best and second-best observation surfaces side by side to obtain a combined grayscale image. Resize this grayscale image to align with the input size of the deep learning classification network, thus enabling network training and prediction. For example... Figure 2 As shown.
[0096] (2) Stack the best and second-best observation surfaces in channels to form a single image: Stack the best and second-best observation surfaces on the image channels to obtain a dual-channel image. Using this type of image for network training and prediction is also a feasible solution.
[0097] The two approaches yielded similar results in this embodiment.
[0098] As a preferred implementation, the method further includes: performing a similarity estimation test on the classified targets to eliminate misclassified targets, thereby further improving the accuracy and robustness of target detection for autonomous vehicles and mobile robots in complex environments.
[0099] Specifically, a range of prior attribute values is set for the corresponding category, including length, width, height, and verticality; the attribute values of the classified targets are calculated, and the target attributes are verified to meet the range of prior attribute values; if they meet the range, the classification result of the target is trusted; if they do not meet the range, the target is ignored.
[0100] For example, taking the shape-size likelihood estimation test, the true target of each category should have a reasonable size. For instance, the height of a normal person is mostly between 1.4 meters and 2.3 meters, and the height of a car is mostly between 1.2 meters and 2.5 meters. Targets exceeding this reasonable range will be penalized with a penalty score. This penalty score directly affects the target's classification confidence. For each violation, the classification confidence is reduced by a penalty score. After completing all tests, the remaining classification confidence is calculated. If the score is less than a threshold, the target's classification result is not trusted; otherwise, the classification result is trusted, and the similarity estimation test for the next target continues.
[0101] The similarity estimation process is as follows: Figure 3As shown, the penalty score is σ = 0.1, the initial classification confidence η is set to 1, the classification confidence threshold is set to 0.9, and the verticality of the target is defined as φ. This method sets up a probability estimation test with two test groups and a total of four test items. The two test groups are the geometric shape test group and the verticality test group. The geometric shape test group includes three items: target area test, length and width range test, and minimum height test; the verticality test group includes one item: target verticality test. The verticality test is only for small target objects with obvious vertical features, such as pedestrians and utility poles. Principal component analysis of the point cloud in the (x, y, z) quadrant yields three pairwise orthogonal eigenvectors. When the magnitude of the eigenvector in the z-direction is greater than 0.8, the target is considered to satisfy the verticality test.
[0102] Similarity estimation tests screen out unreasonable classification targets and eliminate those that may be affected by interference or noise, thus ensuring the stability and accuracy of classification results.
[0103] In summary, existing 3D target detection algorithms based on cameras, LiDAR, or camera-LiDAR fusion all have various shortcomings. This embodiment proposes a 3D target detection algorithm that combines optimal observation surface classification. By processing the LiDAR point cloud, 3D target detection is achieved, improving the robustness, generalization, and real-time performance of target detection for autonomous vehicles and mobile robots in complex environments.
[0104] Example 2
[0105] A computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, it controls the device on which the storage medium is located to perform a three-dimensional target detection method based on optimal observation surface classification as described above.
[0106] The relevant technical solutions are the same as in Embodiment 1, and will not be repeated here.
[0107] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A three-dimensional target detection method based on optimal observation surface classification, characterized in that, include: A frame of 3D point cloud acquired by lidar is segmented and clustered to obtain the clustered target; Determine the orientation vector of each cluster target in the spatial horizontal plane, determine an optimal observation point on the straight line axis that is perpendicular to the orientation vector in the spatial horizontal plane and passes through the average point of the cluster target, and construct an optimal observation coordinate system with the optimal observation point as the origin and the direction from the optimal observation point to the average point as the positive x-axis; The point cloud of each cluster target is transformed from the current coordinate system to the corresponding optimal observation coordinate system. The horizontal yaw angle range and vertical pitch angle range of the cluster target point cloud in the optimal observation coordinate system are mapped to the yz plane of the optimal observation coordinate system to obtain the dimensions in the corresponding two directions, which are used as the virtual width and height of the cluster target. According to the virtual width and height and the required classification sample size, the point cloud of the cluster target located in the optimal observation coordinate system is scaled and mapped to a surface located on the yz plane that is the same size as the required classification sample size to obtain the optimal observation surface. The pixel value of each pixel is the Euclidean distance from the corresponding point of the cluster target to the origin of the optimal observation coordinate system. The target category of a cluster is obtained by classifying the target based on the best observation surface of each cluster target.
2. The three-dimensional target detection method according to claim 1, characterized in that, Before performing segmentation clustering, the method also includes: A frame of original 3D point cloud acquired by lidar is filtered to remove ground point cloud data, resulting in a processed frame of 3D point cloud. Then, the processed 3D point cloud is segmented and clustered to obtain the clustering target.
3. The three-dimensional target detection method according to claim 2, characterized in that, The method for filtering out the ground point cloud is as follows: a frame of original 3D point cloud acquired by lidar is divided into planar grids; ground point cloud is identified and removed from each grid. The implementation method for planar grid division is as follows: On the xy-horizontal plane of the current coordinate system, with the origin of the current coordinate system as the center, N annular regions are divided according to a preset N radial distance. Based on a preset M sector angle, the circular planar region centered on the origin of the current coordinate system and divided into N annular regions is further divided into M sectors, thus obtaining a grid plane. The values of the preset N radial distances and / or the preset M sector angles satisfy the following: the area of the divided grid unit increases with the increase of the radial distance. The points in the original 3D point cloud frame are mapped onto the grid plane to achieve the allocation of the original 3D point cloud frame in the grid and complete the planar grid division.
4. The three-dimensional target detection method according to claim 3, characterized in that, The implementation method for identifying and removing ground point clouds from each grid is as follows: From the original 3D point cloud corresponding to each grid, select the n points with the lowest vertical z-direction height values as the initial grounding point cloud, calculate the covariance matrix of the initial grounding point cloud and perform SVD decomposition on it to obtain the feature vectors on the x, y and z axes of the lidar coordinate system. The feature vectors on the three axes are added together. It is then determined whether the angle between the added feature vector and the current coordinate system xy horizontal plane is less than a threshold. If so, the initial ground point cloud is determined to be true. The ground point cloud is expanded, and the feature vector calculation and angle determination are re-executed until the angle determination result is negative. At this point, the ground point cloud before the most recent expansion is taken as the ground point cloud in the original 3D point cloud corresponding to the current grid. If not, the initial ground point cloud is determined to be false, and there is no ground point cloud in the original 3D point cloud corresponding to the current grid. The identified ground point cloud is deleted from the original 3D point cloud frame.
5. The three-dimensional target detection method according to claim 4, characterized in that, The method for expanding the grounding point cloud is as follows: Calculate the average three-dimensional coordinates of the current grounding point cloud to obtain the average point; this average point and the origin of the lidar coordinate system form the origin-average point vector, and calculate the vector magnitude after projecting this vector onto the z-axis of the current coordinate system, which is used as the extended tolerance threshold. Calculate the difference between the z-coordinate value of each point in the original 3D point cloud (excluding the current ground point cloud) and the average point, and add all points whose difference is less than the expansion tolerance threshold to the current ground point cloud to complete the expansion of the current ground point cloud.
6. The three-dimensional target detection method according to claim 1, characterized in that, The orientation vector is determined as follows: Calculate the covariance matrix of the point cloud corresponding to each cluster target, calculate the eigenvectors of the covariance matrix on the x, y, and z axes of the lidar coordinate system, add the eigenvectors on the three axes, and project them onto the xy horizontal plane of the lidar coordinate system to obtain the orientation vector of the corresponding cluster target on the spatial horizontal plane.
7. The three-dimensional target detection method according to claim 1, characterized in that, The distance between the best observation point of each cluster target and the average point of that cluster target is the same as the radial distance between the average point and the origin of the current coordinate system.
8. The three-dimensional target detection method according to claim 1, characterized in that, Also includes: A suboptimal observation point is determined on a straight line axis located in the horizontal space that passes through the average point of each cluster target and is oriented in the same direction as the orientation vector. A suboptimal observation coordinate system is constructed with the suboptimal observation point as the origin and the direction from the suboptimal observation point to the average point as the positive x-axis. The point cloud of each cluster target is transformed from the current coordinate system to the corresponding suboptimal observation coordinate system. The horizontal yaw angle range and vertical pitch angle range of the cluster target point cloud in the suboptimal observation coordinate system are mapped to the yz plane of the suboptimal observation coordinate system to obtain the dimensions in the corresponding two directions, which are used as the virtual width and height of the cluster target. According to the virtual width and height and the required classification sample size, the point cloud of the cluster target located in the suboptimal observation coordinate system is scaled and mapped to a surface located in the yz plane of the suboptimal observation coordinate system that is the same size as the required classification sample size to obtain the suboptimal observation surface. The pixel value of each pixel is the Euclidean distance from the corresponding point of the cluster target to the origin of the suboptimal observation coordinate system. Then, based on the best and second-best observation surfaces of each cluster target, target classification is performed to obtain the target category of that cluster target.
9. The three-dimensional target detection method according to any one of claims 1 to 8, characterized in that, Also includes: For the classified targets, a similarity estimation test is performed to obtain the final detection result.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, it controls the device on which the storage medium is located to perform a three-dimensional target detection method based on optimal observation surface classification as described in any one of claims 1 to 8.