Method, device and equipment for object detection in autonomous driving and storage medium

Abstract

This disclosure discloses a method, apparatus, device, and storage medium for object detection in autonomous driving, comprising: acquiring a first distance between all point clouds in a point cloud set of an object to be detected and a target vehicle; acquiring multiple point cloud processing ranges based on a comparison between the first distance and a first threshold; adjusting the downsampling retention rate of the point clouds within the first point cloud processing range to obtain a first sample set; performing feature enhancement on the point clouds within a second point cloud processing range to obtain a feature-enhanced second point cloud processing range; performing weighted processing on the bounding boxes within the feature-enhanced second point cloud processing range to obtain a second sample set; inputting the first sample set and the second sample set into a target student network, and outputting the detection result of the object to be detected. This disclosure adjusts the downsampling retention rate of the point clouds within the first point cloud processing range and enhances the point cloud features within the second point cloud processing range to ensure a balance between near and far samples.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to methods, apparatus, devices, and storage media for object detection in autonomous driving. Background Technology

[0002] With the development of autonomous driving perception technology, LiDAR-based 3D object detection technology has made significant progress. Because LiDAR point clouds offer the advantage of accurate 3D ranging, many autonomous driving companies have equipped their vehicles with LiDAR. LiDAR-based 3D object perception technology also plays a core role in the overall perception module of autonomous driving.

[0003] In the process of using LiDAR to perceive the surrounding environment, related technologies usually use AI algorithm models to perform 3D detection of surrounding objects, such as the PointPillars algorithm. However, this type of algorithm is difficult to detect objects that are far away from the vehicle (greater than 50 meters) and have relatively sparse point cloud features. The main reasons are as follows: (1) The point cloud of LiDAR becomes sparser the farther away it is. The different density of the LiDAR point cloud of objects at different distances leads to the difference in feature expression, which makes it difficult to detect distant objects; (2) In the process of manual annotation of the dataset, it is not easy to annotate the samples of distant objects, which makes the number of objects at different distances unbalanced in the dataset, which makes it difficult to detect distant objects.

[0004] Therefore, the related technologies suffer from poor performance in detecting distant objects when using LiDAR for 3D object detection. Summary of the Invention

[0005] In view of this, this disclosure provides a method, apparatus, device and storage medium for object detection in autonomous driving, to solve the problem of poor performance in detecting distant objects when using LiDAR for 3D object detection in related technologies.

[0006] In a first aspect, this disclosure provides a method for object detection in autonomous driving, the method comprising: Obtain the first distance between all point clouds in the point cloud set of the object to be detected and the target vehicle. The point cloud set carries a sample box, which is used to characterize the shape formed by the sample set of the object to be detected. Based on the comparison between the first distance and the first threshold, multiple point cloud processing ranges are obtained, wherein the first threshold is used to divide the point cloud in the point cloud set; The downsampling retention rate of the point cloud within the first point cloud processing range is adjusted to obtain the first sample set. The first point cloud processing range consists of near point clouds with a first distance less than a first threshold within the point cloud processing range. Feature enhancement is performed on the point clouds within the second point cloud processing range to obtain the feature-enhanced second point cloud processing range. The second point cloud processing range consists of far point clouds with a first distance greater than or equal to the first threshold within the point cloud processing range. The sample boxes within the processing range of the second point cloud after feature enhancement are weighted to obtain the second sample set; The first and second sample sets are input into the target student network, and the detection results of the objects to be detected are output. The target student network is a network formed after learning and inheriting the key features extracted by the teacher network.

[0007] In this embodiment, a first distance is obtained between all point clouds in the point cloud set of the object to be detected and the target vehicle. The point cloud set carries bounding boxes, which characterize the shape formed by the bounding boxes of the object to be detected. Multiple point cloud processing ranges are obtained based on a comparison between the first distance and a first threshold, where the first threshold is used to divide the point clouds in the point cloud set. The downsampling retention rate of the point clouds within the first point cloud processing range is adjusted to obtain a first sample set. The first point cloud processing range consists of near-range point clouds whose first distance is less than the first threshold. Feature enhancement is performed on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range. The second point cloud processing range consists of far-range point clouds whose first distance is greater than or equal to the first threshold. The bounding boxes within the feature-enhanced second point cloud processing range are weighted to obtain a second sample set. The first and second sample sets are input into a target student network, and the detection result of the object to be detected is output. Because this embodiment adjusts the downsampling retention rate of the point cloud within the first point cloud processing range, it ensures that the point cloud set maintains sparse consistency, enhances the point cloud features within the second point cloud processing range, supplements the point cloud features in sparse areas, and solves the problem of missing features in distant point clouds. At the same time, it performs weighted processing on the sample boxes within the second point cloud processing range after feature enhancement, ensuring that the number of samples in the distance is small and the number of samples in the near distance is balanced, thereby improving the detection performance of objects within the second point cloud processing range. In addition, this embodiment also involves a target student network, which realizes the transfer of key features of the teacher network, improves the detection accuracy of single-frame point clouds, and especially improves the recognition effect of distant objects.

[0008] In one optional implementation, the downsampling retention rate of the point cloud within the first point cloud processing range is adjusted to obtain a first sample set, including: The point cloud within the first point cloud processing range is divided according to the polar radius range and polar angle range to obtain multiple division regions; Obtain the centroid polar radius of the divided region to get the initial downsampling retention rate of the point cloud within the divided region; The initial downsampling retention rate is adjusted according to the polar angle direction of the divided region to obtain the target downsampling retention rate of the point cloud within the divided region; The point cloud within the defined region is downsampled based on the target downsampling retention rate to obtain the first sample set.

[0009] In this embodiment, the point cloud within the first point cloud processing range is divided into multiple regions according to the polar radius range and polar angle range, thereby achieving differentiated processing of the point cloud within the first point cloud processing range. The target downsampling retention rate of the point cloud within the divided region is determined based on the centroid polar radius value and polar angle direction, and downsampling is performed on the point cloud within the divided region based on the target downsampling retention rate. This maintains the sparsity consistency of the point cloud set, reduces the feature differences between near and far objects, and improves the detection performance of objects within the second point cloud processing range.

[0010] In one optional implementation, the point cloud within the first point cloud processing range is divided according to its polar radius and polar angle range to obtain multiple partitioned regions, including: The point cloud within the first point cloud processing range is divided according to the polar radius range to obtain the first division region; The point cloud within the first point cloud processing range is divided according to the polar angle range to obtain the second division region; Based on the first and second partitioned regions, multiple partitioned regions are obtained.

[0011] In this embodiment of the disclosure, the point cloud within the first point cloud processing range is divided according to its polar radius range to achieve differentiated processing of the point cloud within the first point cloud processing range in terms of near and far dimensions. Similarly, the point cloud within the first point cloud processing range is divided according to its polar angle range to achieve differentiated processing of the point cloud within the first point cloud processing range in terms of angular direction dimensions.

[0012] In one optional implementation, the centroid polar radius of the segmented region is obtained to determine the initial downsampling retention rate of the point cloud within the segmented region, including: Obtain the centroid radius of the target vehicle from the centroid of the divided region; Based on the centroid polar radius value, the initial downsampling retention rate of the point cloud within the divided region is obtained, where the initial downsampling retention rate is directly proportional to the centroid polar radius value.

[0013] In this embodiment of the disclosure, the initial downsampling retention rate of the point cloud within the divided region is obtained by using the centroid radius value of the distance between the centroid of the divided region and the target vehicle, thereby achieving sparse consistency of the point cloud set in the near and far dimensions and reducing the feature differences between near and far objects.

[0014] In one optional implementation, adjusting the initial downsampling retention rate according to the polar angle direction of the divided region to obtain the target downsampling retention rate of the point cloud within the divided region includes: Based on the polar angle direction of the divided regions, the correction parameters for the initial downsampling retention rate are obtained; The initial downsampling retention rate is adjusted based on the correction parameters to obtain the target downsampling retention rate of the point cloud within the divided region.

[0015] In this embodiment of the disclosure, the correction parameter of the initial downsampling retention rate is obtained according to the polar angle direction of the divided region, and the initial downsampling retention rate is adjusted according to the correction parameter to obtain the target downsampling retention rate of the point cloud within the divided region, thereby achieving the sparsity consistency of the point cloud set in the angular direction dimension and reducing the feature differences of objects in different angular directions.

[0016] In one optional implementation, feature enhancement is performed on the point cloud within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range, including: The point cloud within the second point cloud processing range is divided according to the polar radius range and polar angle range to obtain multiple far-distance division regions. Calculate the point cloud density value for each distantly divided region, and filter out sparse sub-regions based on the point cloud density value and a preset density threshold; Virtual point clouds are generated based on the coordinate and normal vector features of the point clouds corresponding to the same sample boxes within sparse sub-regions. The virtual point cloud is subjected to feature verification, and the valid virtual point cloud is added to the second point cloud processing range to obtain the feature-enhanced second point cloud processing range.

[0017] In one optional implementation, the sample boxes within the second point cloud processing range after feature enhancement are weighted to obtain a second sample set, including: Obtain the second distance between the sample box and the target vehicle within the second point cloud processing range after feature enhancement; Obtain the polar angle value of the sample box within the second point cloud processing range after feature enhancement; Based on the second distance, the polar angle value, and the preset weights of the sample boxes, the weighted weights of the sample boxes within the second point cloud processing range after feature enhancement are obtained. Based on the weighted weights and the sample boxes within the second point cloud processing range after feature enhancement, a second sample set is obtained.

[0018] In this embodiment of the disclosure, the weighted weight of the sample box within the second point cloud processing range after feature enhancement is calculated based on the second distance between the sample box and the target vehicle, the polar angle value of the sample box within the second point cloud processing range after feature enhancement, and the preset weight of the sample box. This adjusts the problem of insufficient samples within the second point cloud processing range, ensuring that the number of samples at a distance is small and the number of samples at a distance is large, thereby improving the detection performance of objects within the second point cloud processing range after feature enhancement.

[0019] In one alternative implementation, the method further includes: Before inputting the first and second sample sets into the target student network, historical single-frame point clouds are obtained, wherein the historical single-frame point clouds carry ground truth bounding box samples. Historical single-frame point clouds are stitched together to form multiple dense stitched point clouds; Input the densely stitched point cloud into the teacher network and output the first prediction box, the first feature map, and the first confidence distribution. Input the historical single-frame point cloud into the student network, and output the second prediction box, the second feature map, and the second confidence distribution; Based on the comparison results between the second predicted bounding box and the ground truth bounding box samples, the comparison results between the first feature map and the second feature map, and the comparison results between the first confidence distribution and the second confidence distribution, the network parameters of the student network are adjusted to obtain the target student network.

[0020] In this embodiment, the teacher network is responsible for high-precision modeling, while the student network maintains a lightweight structure. It utilizes densely spliced ​​point clouds to enhance feature learning, significantly improving the detection performance of sparse point cloud objects at a distance. Thus, after the student network fully learns the capabilities of the teacher network, the resulting target student network greatly improves the detection accuracy of sparse point clouds in a single frame, especially improving the recall and accuracy of small targets, distant objects, and sparse point clouds.

[0021] In one optional implementation, based on the comparison results of the second predicted bounding box and the ground truth bounding box samples, the comparison results of the first feature map and the second feature map, and the comparison results of the first confidence distribution and the second confidence distribution, the network parameters of the student network are adjusted to obtain the target student network, including: The error between the second predicted bounding box and the ground truth bounding box sample in the student network is obtained to obtain the first loss; The error between the first confidence distribution and the second confidence distribution is obtained to obtain the second loss; The error between the first feature map and the second feature map is obtained to obtain the third loss; Based on the first loss, the second loss, and the third loss, we obtain the knowledge distillation loss; By backpropagating the knowledge distillation loss, the network parameters of the student network are updated, making the student network closer to the teacher network, thus obtaining the target student network.

[0022] Secondly, this disclosure provides an apparatus for object detection in autonomous driving, the apparatus comprising: The first acquisition module is used to acquire the first distance between the point cloud set of the object to be detected and the target vehicle. The point cloud set carries a sample frame, which is used to characterize the shape formed by the sample set of the object to be detected. The second acquisition module is used to acquire multiple point cloud processing ranges based on the comparison result between the first distance and the first threshold, wherein the first threshold is used to divide the point cloud in the point cloud set; The first obtaining module is used to adjust the downsampling retention rate of the point cloud within the first point cloud processing range to obtain a first sample set, wherein the first point cloud processing range is composed of near point clouds with a first distance less than a first threshold within the point cloud processing range, and to perform feature enhancement on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range, wherein the second point cloud processing range is composed of far point clouds with a first distance greater than or equal to the first threshold within the point cloud processing range; The second module is used to perform weighted processing on the sample boxes within the processing range of the second point cloud after feature enhancement, to obtain the second sample set. The detection module is used to input the first sample set and the second sample set into the target student network and output the detection result of the object to be detected. The target student network is a network formed after learning and inheriting the key features extracted by the teacher network.

[0023] Thirdly, this disclosure provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the object detection method in autonomous driving described in the first aspect or any corresponding embodiment.

[0024] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method for object detection in autonomous driving as described in the first aspect or any corresponding embodiment.

[0025] Fifthly, this disclosure provides a computer program product, including computer instructions for causing a computer to perform the method for object detection in autonomous driving as described in the first aspect or any corresponding embodiment. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating a method for object detection in autonomous driving according to an embodiment of the present disclosure; Figure 2 This is a structural block diagram of an object detection device in autonomous driving according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0029] With the development of autonomous driving perception technology, LiDAR-based 3D object detection technology has made significant progress. Because LiDAR point clouds offer the advantage of accurate 3D ranging, many autonomous driving companies have equipped their vehicles with LiDAR. LiDAR-based 3D object perception technology also plays a core role in the overall perception module of autonomous driving.

[0030] In the process of using LiDAR to perceive the surrounding environment, related technologies usually use AI algorithm models to perform 3D detection of surrounding objects, such as the PointPillars algorithm. However, this type of algorithm is difficult to detect objects that are far away from the vehicle (greater than 50 meters) and have relatively sparse point cloud features. The main reasons are as follows: (1) The point cloud of LiDAR becomes sparser the farther away it is. The different density of the LiDAR point cloud of objects at different distances leads to the difference in feature expression, which makes it difficult to detect distant objects; (2) In the process of manual annotation of the dataset, it is not easy to annotate the samples of distant objects, which makes the number of objects at different distances unbalanced in the dataset, which makes it difficult to detect distant objects.

[0031] Therefore, the related technologies suffer from poor performance in detecting distant objects when using LiDAR for 3D object detection.

[0032] To address the aforementioned problems, according to an embodiment of this disclosure, a method for object detection in autonomous driving is provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a method for object detection in autonomous driving, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for object detection in autonomous driving according to an embodiment of the present disclosure. This process can be applied to a server and includes the following steps: Step S101: Obtain the first distance between all point clouds in the point cloud set of the object to be detected and the target vehicle. The point cloud set carries a sample frame, which is used to characterize the shape formed by the sample set of the object to be detected.

[0034] Optionally, this embodiment can be applied to driving scenarios equipped with multiple (e.g., three) solid-state LiDARs, where the horizontal field of view (HFOV) is 120 degrees and the vertical field of view (VFOV) is 25 degrees. Since this embodiment uses solid-state LiDARs to acquire the point cloud set of the object to be detected, and the number of solid-state LiDARs is limited, the sparsity of the LiDAR point cloud in the application scenario of this embodiment varies in the angular dimension. The object to be detected can be a vehicle, pedestrian, or obstacle, etc.

[0035] Specifically, the server uses a LiDAR system to emit and receive light in a driving scenario, acquiring a point cloud set of objects to be detected. Then, it obtains bounding boxes for these objects through manual annotation. Using the target vehicle as the origin of the coordinate axis, for the already annotated bounding boxes, it is assumed that their annotation data uses a 7-dimensional vector. It means that, among them, This indicates the horizontal coordinate of the center point of the annotation box in the camera coordinate system. This indicates the vertical height coordinates of the center point of the annotation box in the camera coordinate system. This indicates the depth coordinates of the center point of the annotation box in the camera coordinate system. Indicates the length of the sample frame. This indicates the width of the sample frame. Indicates the height of the sample frame. This represents the yaw angle of the sample bounding box from the bird's-eye-view (BEV) perspective. It is also assumed that the point cloud set of this sample is... ,in The coordinates of the 3D laser point cloud are (3D vectors). This represents the total number of points in a single frame of the point cloud, where... It is the first in the single-frame point cloud set i The three-dimensional coordinate components of each laser point They represent the first i The x-axis, y-axis, and z-axis coordinates of a laser point relative to the origin of the target vehicle's coordinate system in three-dimensional space.

[0036] From the BEV perspective, the server obtains the point cloud set of the object to be detected based on the coordinates of the point cloud. P The first distance between all point clouds and the target vehicle.

[0037] It should be noted that BEV is a perspective of viewing an object or scene from above. In the field of autonomous driving, data acquired by sensors (such as LiDAR) is usually converted into BEV representation, simplifying the complex three-dimensional environment into a two-dimensional image to better perform tasks such as object detection and path planning.

[0038] Step S102: Based on the comparison result between the first distance and the first threshold, obtain multiple point cloud processing ranges, wherein the first threshold is used to divide the point cloud in the point cloud set.

[0039] Optionally, in this embodiment of the disclosure, the first threshold is included within the range of values ​​for the first distance, and is used for the point cloud set. P The point cloud is divided into segments. For example, in this embodiment, the first threshold is selected as 50.

[0040] Specifically, the server obtains multiple point cloud processing ranges based on a first distance and a first threshold. For example, considering the impact of sparsity differences in point clouds at different distances on detection performance, the server sets the point clouds together. Point clouds are categorized by distance: distant point clouds (greater than 50 meters) and near point clouds (less than 50 meters). For each laser point cloud... The Euclidean distance between the point cloud and the origin of the target vehicle (i.e., the self-vehicle) is calculated using the following formula, and point cloud sets with a distance less than 50 meters are selected. Among them, the point cloud set within the first point cloud processing range is a point cloud set with a distance of less than 50 meters. .

[0041]

[0042] in, Point cloud The Euclidean distance from the origin of the vehicle. This refers to a collection of distant point clouds that are 50 meters or more away.

[0043] Step S103: Adjust the downsampling retention rate of the point cloud within the first point cloud processing range to obtain a first sample set. The first point cloud processing range consists of near-distance point clouds within the point cloud processing range whose first distance is less than a first threshold. Perform feature enhancement on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range. The second point cloud processing range consists of far-distance point clouds within the point cloud processing range whose first distance is greater than or equal to a first threshold.

[0044] Optionally, in embodiments of this disclosure, Sets and Differences in the sparsity of point clouds between sets can lead to inconsistent feature representations of samples, thereby reducing the detection performance of more distant samples. Therefore, to maintain consistency in sparsity between the two sets, [further measures are needed]. The point cloud in the data is downsampled. In this embodiment of the disclosure, a point cloud set is obtained. The polar radius range is 50 meters, and the polar angle range is 360 degrees. Adjustments are made based on the polar radius and polar angle ranges. Downsampling retention rate, based on the downsampling retention rate of the point cloud set The inner point cloud is downsampled to obtain the first sample set, where the downsampling retention rate refers to the proportion of the number of point clouds retained after the downsampling operation to the number of original point clouds.

[0045] The point cloud set within the second point cloud processing range is a point cloud set with a distance of 50 meters or more. .

[0046] For point clouds within the second point cloud processing range (≥50 meters), feature enhancement can be used to generate more virtual point clouds at the locations of sparse sub-regions of the point cloud within the second point cloud processing range. Then, the virtual point clouds are added to the original point cloud set within the second point cloud processing range to obtain the feature-enhanced point cloud set, and thus obtain the feature-enhanced second point cloud processing range.

[0047] Step S104: Weight the sample boxes within the processing range of the second point cloud after feature enhancement to obtain the second sample set.

[0048] Optionally, all sample boxes within the second point cloud processing range after feature enhancement are traversed to determine the position, size, confidence level, and distance from the vehicle of each sample box in the point cloud space. According to a preset weighting rule, weight values ​​corresponding to the position, size, confidence level, and distance from the vehicle are assigned to each sample box. The weight values ​​are then bound to the features, annotation information, and loss calculation factor of the corresponding sample box, so that the model gives higher optimization priority to high-weight sample boxes during training. All sample boxes that have completed weighting are then integrated to form a second sample set.

[0049] Step S105: Input the first sample set and the second sample set into the target student network and output the detection result of the object to be detected. The target student network is a network formed after learning and inheriting the key features extracted by the teacher network.

[0050] Optionally, in this embodiment, a teacher-student network architecture is designed, where the backbone architecture of the student network is consistent with that of the teacher network but with lighter parameters, for real-time detection tasks. During training, the student network not only needs to fit real labeled data (hard supervision) but also needs to align the feature maps and prediction results output by the teacher network (soft supervision). Through the knowledge distillation loss function, the key features of the teacher network are transferred to obtain the target student network. Then, the first sample set and the second sample set are input into the target student network to obtain the detection results of the object to be detected.

[0051] In this embodiment, a first distance is obtained between all point clouds in the point cloud set of the object to be detected and the target vehicle. The point cloud set carries bounding boxes, which characterize the shape formed by the bounding boxes of the object to be detected. Multiple point cloud processing ranges are obtained based on a comparison between the first distance and a first threshold, where the first threshold is used to divide the point clouds in the point cloud set. The downsampling retention rate of the point clouds within the first point cloud processing range is adjusted to obtain a first sample set. The first point cloud processing range consists of near-range point clouds whose first distance is less than the first threshold. Feature enhancement is performed on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range. The second point cloud processing range consists of far-range point clouds whose first distance is greater than or equal to the first threshold. The bounding boxes within the feature-enhanced second point cloud processing range are weighted to obtain a second sample set. The first and second sample sets are input into a target student network, and the detection result of the object to be detected is output. Because this embodiment adjusts the downsampling retention rate of the point cloud within the first point cloud processing range, it ensures that the point cloud set maintains sparse consistency, enhances the point cloud features within the second point cloud processing range, supplements the point cloud features in sparse areas, and solves the problem of missing features in distant point clouds. At the same time, it performs weighted processing on the sample boxes within the second point cloud processing range after feature enhancement, ensuring that the number of samples in the distance is small and the number of samples in the near distance is balanced, thereby improving the detection performance of objects within the second point cloud processing range. In addition, this embodiment also involves a target student network, which realizes the transfer of key features of the teacher network, improves the detection accuracy of single-frame point clouds, and especially improves the recognition effect of distant objects.

[0052] In some optional implementations, a first sample set is obtained by adjusting the downsampling retention rate of the point cloud within the first point cloud processing range, including: The point cloud within the first point cloud processing range is divided according to the polar radius range and polar angle range to obtain multiple division regions; Obtain the centroid polar radius of the divided region to get the initial downsampling retention rate of the point cloud within the divided region; The initial downsampling retention rate is adjusted according to the polar angle direction of the divided region to obtain the target downsampling retention rate of the point cloud within the divided region; The point cloud within the defined region is downsampled based on the target downsampling retention rate to obtain the first sample set.

[0053] Optionally, in this embodiment of the disclosure, the centroid polar radius of the divided region refers to the distance between the centroid of the divided region and the target vehicle, and the polar angle direction of the divided region may include front, left front, right front, rear, left rear and right rear.

[0054] Specifically, the server accesses the point cloud collection. The interior point cloud is divided into multiple regions based on its polar radius range (50 meters) and polar angle range (360 degrees). Then, the centroid polar radius values ​​of each region are obtained. According to the centroid polar radius value The initial downsampling retention rate of the point cloud within the defined region is obtained. Next, the initial downsampling retention rate is adjusted according to the polar angle direction of the divided region to obtain the target downsampling retention rate of the point cloud within the divided region. Finally, the point cloud within the divided region is downsampled based on the target downsampling retention rate to obtain the first sample set.

[0055] In this embodiment, the point cloud within the first point cloud processing range is divided into multiple regions according to the polar radius range and polar angle range, thereby achieving differentiated processing of the point cloud within the first point cloud processing range. The target downsampling retention rate of the point cloud within the divided region is determined based on the centroid polar radius value and polar angle direction, and downsampling is performed on the point cloud within the divided region based on the target downsampling retention rate. This maintains the sparsity consistency of the point cloud set, reduces the feature differences between near and far objects, and improves the detection performance of objects within the second point cloud processing range.

[0056] In some optional implementations, the point cloud within the first point cloud processing range is divided according to its polar radius and polar angle range to obtain multiple partitioned regions, including: The point cloud within the first point cloud processing range is divided according to the polar radius range to obtain the first division region; The point cloud within the first point cloud processing range is divided according to the polar angle range to obtain the second division region; Based on the first and second partitioned regions, multiple partitioned regions are obtained.

[0057] Optionally, in this embodiment of the disclosure, the server sets the point cloud according to the polar radius range (50 meters). Divide the data into the first partition region (e.g., 10 regions, each 5 meters wide), and aggregate the point cloud data according to the polar angle range (360 degrees). The system is divided into a second partitioned region (e.g., 6 regions, each interval 60°), and then the first and second partitioned regions are merged to obtain multiple partitioned regions (e.g., 60 regions). Each partitioned region can be represented as... (in, m This is the first division area. , n This is the second division area. ), For the extreme path of the cloud, The polar angle of the point cloud. In this embodiment of the disclosure, the point cloud within the first point cloud processing range is divided according to the polar radius range to achieve differentiated processing of the point cloud within the first point cloud processing range in the near and far dimensions. The point cloud within the first point cloud processing range is divided according to the polar angle range to achieve differentiated processing of the point cloud within the first point cloud processing range in the angular direction dimension.

[0058] In some optional implementations, the centroid polar radius of the segmented region is obtained to determine the initial downsampling retention rate of the point cloud within the segmented region, including: Obtain the centroid radius of the target vehicle from the centroid of the divided region; Based on the centroid polar radius value, the initial downsampling retention rate of the point cloud within the divided region is obtained, where the initial downsampling retention rate is directly proportional to the centroid polar radius value.

[0059] Optionally, in this embodiment of the disclosure, the downsampling retention rate is calculated based on the centroid radius and polar angle direction of the target vehicle from the centroid of each divided region, as shown in the following formula:

[0060] In the above formula, r The downsampling retention rate, For the first The centroid radius of each divided region; This is the base coefficient for retention rate (preferably, the value range is 0.4~0.6, used to adjust the overall range of retention rate). for The maximum polar diameter of the inner point cloud (i.e., 50 meters). The minimum retention rate is set (ideally 0.1 to avoid feature loss due to excessively low retention); this formula is a monotonically decreasing function with a polar radius. The larger the value (the greater the distance), the higher the downsampling retention rate. r The smaller it is, the more it perfectly meets the requirements of a monotonic function; This is the polar angle direction correction coefficient, used to avoid oversampling of the sparse point cloud behind: the retention rate of the front, left front, and right front division regions remains unchanged, the retention rate of the left rear and right rear division regions is multiplied by 1.2, and the retention rate of the rear division region is multiplied by 1.5, to ensure that the effective features of the rear point cloud are not lost.

[0061] In this embodiment of the disclosure, the initial downsampling retention rate of the point cloud within the divided region is obtained by using the centroid radius value of the distance between the centroid of the divided region and the target vehicle, thereby achieving sparse consistency of the point cloud set in the near and far dimensions and reducing the feature differences between near and far objects.

[0062] In some optional implementations, the initial downsampling retention rate is adjusted according to the polar angle direction of the segmented region to obtain the target downsampling retention rate of the point cloud within the segmented region, including: Based on the polar angle direction of the divided regions, the correction parameters for the initial downsampling retention rate are obtained; The initial downsampling retention rate is adjusted based on the correction parameters to obtain the target downsampling retention rate of the point cloud within the divided region.

[0063] Optionally, in this embodiment of the disclosure, the server obtains the correction parameters for the initial downsampling retention rate of the point cloud within the segmented region according to the polar angle direction of the segmented region (including front, left front, right front, rear, left rear, and right rear).

[0064] Specifically, the point cloud density in the front, left front, and right front regions is relatively high, and the server keeps the initial downsampling retention rate of these regions unchanged, with a correction parameter of 1; the point cloud density in the left rear and right rear regions is relatively sparse, and the server slightly increases the initial downsampling retention rate of these regions, with a correction parameter slightly greater than 1 (e.g., 1.2); the point cloud density in the rear region is even sparser, and the server significantly increases the initial downsampling retention rate of these regions, with a correction parameter greater than that of the left rear and right rear regions (e.g., 1.5).

[0065] Then, the server corrects the initial downsampling retention rate of the point cloud within the divided region by multiplying it by the corresponding correction parameter according to the polar angle direction of the divided region, and obtains the target downsampling retention rate.

[0066] In this embodiment of the disclosure, the correction parameter of the initial downsampling retention rate is obtained according to the polar angle direction of the divided region, and the initial downsampling retention rate is adjusted according to the correction parameter to obtain the target downsampling retention rate of the point cloud within the divided region, thereby achieving the sparsity consistency of the point cloud set in the angular direction dimension and reducing the feature differences of objects in different angular directions.

[0067] In some optional implementations, feature enhancement is performed on the point cloud within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range, including: The point cloud within the second point cloud processing range is divided according to the polar radius range and polar angle range to obtain multiple far-distance division regions. Calculate the point cloud density value for each distantly divided region, and filter out sparse sub-regions based on the point cloud density value and a preset density threshold; Virtual point clouds are generated based on the coordinate and normal vector features of the point clouds corresponding to the same sample boxes within sparse sub-regions. The virtual point cloud is subjected to feature verification, and the valid virtual point cloud is added to the second point cloud processing range to obtain the feature-enhanced second point cloud processing range.

[0068] Optionally, the point cloud within the second point cloud processing range is divided into multiple distant division regions corresponding to the first point cloud processing range according to the polar diameter range (e.g., 50-80 meters) and polar angle range (360 degrees) division rules in the above embodiments (e.g., the polar diameter is divided into 6 regions, the polar angle is divided into 6 regions, and a total of 36 distant division regions).

[0069] Then, the point cloud density value for each distantly segmented region is calculated. , =N / V, where N is the number of point clouds in the region and V is the spatial volume of the region. These values ​​can be obtained directly.

[0070] Set density threshold (e.g., 0.01 per cubic meter), filter out < sparse subregions.

[0071] For each sparse sub-region, extract valid point clouds with the same object (e.g., the same bounding box outline) (identified by connectivity analysis of the point clouds), and obtain their corresponding coordinate features. ) and normal vector features ( It should be noted that, ( () represents the three-dimensional coordinate features of the point cloud of the same object in the vehicle coordinate system, used to characterize the spatial position of the point cloud of the object. The normal vector is a vector perpendicular to the local plane of the point cloud. These are the components of the vector along the x, y, and z axes of the three-dimensional coordinate system. A virtual point cloud is generated in the blank areas of the sparse sub-region using Kriging interpolation. The coordinates and normal vector of the virtual point cloud are based on the aforementioned coordinate characteristics ( ) and normal vector features ( Interpolation calculations are performed on the spatial distribution of the virtual point cloud to ensure that the virtual point cloud is consistent with the contour features of the original object.

[0072] The generated virtual point cloud is subjected to feature verification. The deviation value between the virtual point cloud and the original object point cloud outline is calculated. If the deviation value exceeds the preset threshold (e.g., 5 cm), the virtual point cloud is removed. The effective virtual point cloud is added to the original point cloud set of the second point cloud processing range to obtain the feature-enhanced second point cloud processing range.

[0073] In some optional implementations, the sample boxes within the processed second point cloud after feature enhancement are weighted to obtain a second sample set, including: Obtain the second distance between the sample box and the target vehicle within the second point cloud processing range after feature enhancement; Obtain the polar angle value of the sample box within the second point cloud processing range after feature enhancement; Based on the second distance, the polar angle value, and the preset weights of the sample boxes, the weighted weights of the sample boxes within the second point cloud processing range after feature enhancement are obtained. Based on the weighted weights and the sample boxes within the second point cloud processing range after feature enhancement, a second sample set is obtained.

[0074] Optionally, based on statistical results, in the LiDAR sample annotation dataset, the number of samples within a range of less than 50 meters is more than four times the number of samples within a range of 50 meters or greater. Furthermore, the farther the distance, the fewer the samples, the greater the annotation difficulty, and the higher the "preciousness" of the samples. Therefore, samples within a range of 50 meters or greater (i.e., the second point cloud processing range) are weighted offline. The weight value is dynamically calculated by combining the second distance between the sample and the target vehicle and the polar angle of the sample bounding box. The weighting formula is as follows. This weight will be applied to the calculation of the sample detection loss, increasing the optimizer's attention to distant samples:

[0075] Explanation of the parameters in the above formula: For weighted weights, For the first i The Euclidean distance between the center point of each sample frame and the origin of the target vehicle (calculated from the center point in the BEV view). The second normal form of vectors, i.e. ),in, This indicates the horizontal coordinate of the center point of the annotation box in the camera coordinate system. This indicates the vertical height coordinates of the center point of the annotation box in the camera coordinate system. , Preset weights for long-distance samples. For maximum weight, To minimize the weight (preferably 2.5 and 1.2 respectively), ensure that the weight increases with distance and remains within a reasonable range; This is the polar angle correction factor, based on the polar angle of the sample frame. Zone division settings: front, left front, right front ( Take 1.0, left rear, right rear ( Take 1.1, then ( We set it to 1.3 to compensate for the sparsity of the subsequent samples; In this embodiment of the disclosure, by weighting the sample boxes within the second point cloud processing range after feature enhancement, it is ensured that the number of samples at a distance is small and the number of samples at a distance is large, thereby improving the detection performance of objects within the second point cloud processing range after feature enhancement.

[0076] As an optional embodiment, before inputting the first sample set and the second sample set into the target student network, the method includes: Obtain historical single-frame point clouds, where the historical single-frame point clouds carry ground truth bounding box samples; Historical single-frame point clouds are stitched together to form multiple dense stitched point clouds; Input the densely stitched point cloud into the teacher network and output the first prediction box, the first feature map, and the first confidence distribution. Input the historical single-frame point cloud into the student network, and output the second prediction box, the second feature map, and the second confidence distribution; Based on the comparison results between the second predicted bounding box and the ground truth bounding box samples, the comparison results between the first feature map and the second feature map, and the comparison results between the first confidence distribution and the second confidence distribution, the network parameters of the student network are adjusted to obtain the target student network.

[0077] Optionally, acquire consecutive historical single-frame point clouds collected by the LiDAR of the autonomous vehicle, with each frame of point cloud carrying a 3D ground truth bounding box sample of the corresponding object (including center point, size, and yaw angle annotation information).

[0078] Based on the vehicle IMU (Inertial Measurement Unit) positioning data, coordinate alignment and inter-frame registration are performed on continuous multi-frame historical single-frame point clouds. Duplicate and redundant points are removed, effective feature points are retained, and the points are stitched together to form a dense stitched point cloud with richer information and more complete features.

[0079] The densely stitched point cloud is input into a teacher network using an improved PointNet++ with an added attention mechanism. After feature extraction, it outputs a high-precision first prediction box, a highly expressive first feature map, and a high-confidence first confidence distribution, which serve as the standard supervision signal for knowledge distillation.

[0080] The historical single-frame point cloud, after polar coordinate partitioning and variable downsampling, is input into the student network, which has the same backbone structure as the teacher network but with lighter parameters. The output includes the second prediction box, the second feature map, and the second confidence distribution.

[0081] Based on the differences between the second predicted bounding box and the ground truth bounding box, the differences between the first feature map and the second feature map, and the differences between the first confidence distribution and the second confidence distribution, the parameters of the student network are iteratively updated through backpropagation. After convergence, a target student network that can be used for real-time sparse point cloud 3D detection is obtained.

[0082] In this embodiment of the disclosure, a student network is trained on input sparse point cloud samples, and a teacher network with multiple frames of dense point cloud samples is used to provide high-quality supervision, thereby solving the problems of blurred features in single-frame point clouds and inaccurate detection of distant objects.

[0083] As an optional embodiment, based on the comparison results of the second predicted bounding box and the ground truth bounding box samples, the comparison results of the first feature map and the second feature map, and the comparison results of the first confidence distribution and the second confidence distribution, the network parameters of the student network are adjusted to obtain the target student network, including: The error between the second predicted bounding box and the ground truth bounding box sample in the student network is obtained to obtain the first loss; The error between the first confidence distribution and the second confidence distribution is obtained to obtain the second loss; The error between the first feature map and the second feature map is obtained to obtain the third loss; Based on the first loss, the second loss, and the third loss, we obtain the knowledge distillation loss; By backpropagating the knowledge distillation loss, the network parameters of the student network are updated, making the student network closer to the teacher network, thus obtaining the target student network.

[0084] Optionally, the process of adjusting student network parameters and obtaining the target student network architecture based on multi-dimensional comparison results is as follows: Calculate the first loss (hard-supervised loss): based on the second predicted bounding box output by the student network and the ground truth bounding box samples carried in the point cloud, use the Focal Loss function. Loss and IoU Loss Function Loss combination yields the first loss. ,in, The error is calculated for the weight coefficients (preferably 0.5) to obtain the first loss, which is used to constrain the student network to fit the real annotations.

[0085] Calculate the second loss (soft supervision loss): Use KL divergence to calculate the error between the first confidence distribution of the teacher network output and the second confidence distribution of the student network output, i.e., the second loss. ,in, , The confidence vectors of the student and teacher networks are respectively used to obtain the second loss, which is used to constrain the student network to learn the predicted distribution of the teacher network.

[0086] Calculate the third loss (feature alignment loss): Use the MSE loss to calculate the error between the first feature map output by the teacher network and the second feature map output by the student network. This error is used to align the feature maps of the intermediate layers between the student and teacher networks, i.e., the third loss. ,in, For feature map size, H Represents the height of the feature map. W The width of the feature map. i The representative feature map in the height dimension ( H Pixel index on axis j The representative feature map in the width dimension ( W Pixel index on axis The intermediate layer feature maps of the student and teacher networks are respectively used to obtain the third loss, which is used to constrain the alignment of the intermediate features of the student network with those of the teacher network.

[0087] The knowledge distillation loss is obtained by fusion: the first loss, the second loss, and the third loss are weighted and summed to obtain the total knowledge distillation loss, calculated using the following formula:

[0088] in, For total knowledge distillation loss, The loss weight coefficients are preferably set to 1.0, 0.3, and 0.5, and can be adaptively adjusted according to the actual training effect.

[0089] Backpropagation to update parameters: Backpropagation is performed based on the total knowledge distillation loss to iteratively update the student network parameters, so that the prediction results, feature representations, and confidence distribution of the student network continuously approximate the teacher network, and finally converges to obtain a high-precision and lightweight target student network architecture.

[0090] In this embodiment, a three-branch loss consisting of hard supervision, soft supervision, and feature alignment is used. Through multi-loss weighted joint optimization, the student network stably approximates the teacher network, ensuring that the student network can fully learn the teacher network's capabilities.

[0091] This embodiment also provides an object detection device for autonomous driving, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0092] This embodiment provides a device for object detection in autonomous driving, such as... Figure 2 As shown, it includes: The first acquisition module 201 is used to acquire the first distance between the point cloud set of the object to be detected and the target vehicle. The point cloud set carries a sample frame, which is used to characterize the shape formed by the sample set of the object to be detected. The second acquisition module 202 is used to acquire multiple point cloud processing ranges based on the comparison result between the first distance and the first threshold, wherein the first threshold is used to divide the point cloud in the point cloud set; The first obtaining module 203 is used to adjust the downsampling retention rate of the point cloud within the first point cloud processing range to obtain a first sample set, wherein the first point cloud processing range is composed of near point clouds with a first distance less than a first threshold within the point cloud processing range, and to perform feature enhancement on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range, wherein the second point cloud processing range is composed of far point clouds with a first distance greater than or equal to the first threshold within the point cloud processing range; The second module 204 is used to perform weighted processing on the sample boxes within the processing range of the second point cloud after feature enhancement to obtain the second sample set. The detection module 205 is used to input the first sample set and the second sample set into the target student network and output the detection result of the object to be detected. The target student network is a network formed after learning and inheriting the key features extracted by the teacher network.

[0093] In this embodiment, a first distance is obtained between all point clouds in the point cloud set of the object to be detected and the target vehicle. The point cloud set carries bounding boxes, which characterize the shape formed by the bounding boxes of the object to be detected. Multiple point cloud processing ranges are obtained based on a comparison between the first distance and a first threshold, where the first threshold is used to divide the point clouds in the point cloud set. The downsampling retention rate of the point clouds within the first point cloud processing range is adjusted to obtain a first sample set. The first point cloud processing range consists of near-range point clouds whose first distance is less than the first threshold. Feature enhancement is performed on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range. The second point cloud processing range consists of far-range point clouds whose first distance is greater than or equal to the first threshold. The bounding boxes within the feature-enhanced second point cloud processing range are weighted to obtain a second sample set. The first and second sample sets are input into a target student network, and the detection result of the object to be detected is output. Because this embodiment adjusts the downsampling retention rate of the point cloud within the first point cloud processing range, it ensures that the point cloud set maintains sparse consistency, enhances the point cloud features within the second point cloud processing range, supplements the point cloud features in sparse areas, and solves the problem of missing features in distant point clouds. At the same time, it performs weighted processing on the sample boxes within the second point cloud processing range after feature enhancement, ensuring that the number of samples in the distance is small and the number of samples in the near distance is balanced, thereby improving the detection performance of objects within the second point cloud processing range. In addition, this embodiment also involves a target student network, which realizes the transfer of key features of the teacher network, improves the detection accuracy of single-frame point clouds, and especially improves the recognition effect of distant objects.

[0094] In some alternative implementations, the first obtaining module 203 includes: The first submodule is used to divide the point cloud within the first point cloud processing range according to the polar radius range and polar angle range to obtain multiple division regions. The second submodule is used to obtain the centroid polar radius value of the divided region and obtain the initial downsampling retention rate of the point cloud within the divided region; The third submodule is used to adjust the initial downsampling retention rate according to the polar angle direction of the divided region, so as to obtain the target downsampling retention rate of the point cloud within the divided region; The fourth submodule is used to downsample the point cloud within the divided region based on the target downsampling retention rate to obtain the first sample set.

[0095] In some alternative implementations, the first obtaining submodule includes: The first obtaining unit is used to divide the point cloud within the first point cloud processing range according to the polar radius range to obtain the first division region; The second obtaining unit is used to divide the point cloud within the first point cloud processing range according to the polar angle range to obtain the second division region; The third obtaining unit is used to obtain multiple division regions based on the first division region and the second division region.

[0096] In some alternative implementations, the second obtaining submodule includes: The acquisition unit is used to acquire the centroid polar radius values ​​of the centroids of the divided regions relative to the target vehicle. The fourth unit is used to obtain the initial downsampling retention rate of the point cloud within the divided region based on the centroid polar radius value, wherein the initial downsampling retention rate is proportional to the centroid polar radius value.

[0097] In some alternative implementations, the third submodule includes: The fifth unit is used to obtain the correction parameters for the initial downsampling retention rate based on the polar angle direction of the divided region; The sixth unit is used to adjust the initial downsampling retention rate according to the correction parameters, so as to obtain the target downsampling retention rate of the point cloud within the divided area.

[0098] In some alternative implementations, the second obtaining module 204 includes: The fifth submodule is used to divide the point cloud within the second point cloud processing range according to the polar radius range and polar angle range to obtain multiple far-distance division regions. The filtering submodule is used to calculate the point cloud density value of each distantly divided region, and filter out sparse sub-regions based on the point cloud density value and a preset density threshold. The generation submodule is used to generate virtual point clouds based on the coordinate features and normal vector features of the point clouds corresponding to the same sample boxes within sparse sub-regions. The sixth submodule is used to perform feature verification on the virtual point cloud, add the valid virtual point cloud to the second point cloud processing range, and obtain the feature-enhanced second point cloud processing range.

[0099] In some alternative implementations, the second obtaining module 204 includes: The first acquisition submodule is used to acquire the second distance between the sample box and the target vehicle within the second point cloud processing range after feature enhancement; The second acquisition submodule is used to acquire the polar angle values ​​of the sample boxes within the second point cloud processing range after feature enhancement; The seventh submodule is used to obtain the weighted weight of the sample box within the second point cloud processing range after feature enhancement, based on the second distance, polar angle value, and preset weight of the sample box. The eighth submodule is used to obtain the second sample set based on the sample boxes within the second point cloud processing range after weighting and feature enhancement.

[0100] In some alternative embodiments, the device further includes: The third acquisition module is used to acquire historical single-frame point clouds before inputting the first sample set and the second sample set into the target student network, wherein the historical single-frame point clouds carry real bounding box samples. The stitching module is used to stitch together historical single-frame point clouds to form multiple dense stitched point clouds; The first output module is used to input the densely spliced ​​point cloud into the teacher network and output the first prediction box, the first feature map and the first confidence distribution. The second output module is used to input the historical single-frame point cloud into the student network and output the second prediction box, the second feature map and the second confidence distribution. The adjustment module is used to adjust the network parameters of the student network based on the comparison results between the second predicted bounding box and the ground truth bounding box samples, the comparison results between the first feature map and the second feature map, and the comparison results between the first confidence distribution and the second confidence distribution, so as to obtain the target student network.

[0101] In some alternative implementations, the adjustment module includes: The third acquisition submodule is used to obtain the error between the second predicted bounding box and the ground truth bounding box sample of the student network, and to obtain the first loss; The fourth acquisition submodule is used to obtain the error between the first confidence distribution and the second confidence distribution to obtain the second loss; The fifth acquisition submodule is used to obtain the error between the first feature map and the second feature map to obtain the third loss; The ninth submodule is used to obtain the knowledge distillation loss based on the first loss, the second loss, and the third loss. The tenth submodule is used to update the network parameters of the student network based on backpropagation of knowledge distillation loss, so that the student network is closer to the teacher network, and the target student network is obtained.

[0102] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0103] In this embodiment, the object detection device in autonomous driving is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0104] This disclosure also provides a computer device having the above-described features. Figure 2 The device shown is for object detection in autonomous driving.

[0105] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this disclosure, such as... Figure 3 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0106] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0107] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0108] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0110] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0111] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0112] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0113] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for object detection in autonomous driving, characterized in that, The method includes: Obtain the first distance between the point cloud set of the object to be detected and the target vehicle, wherein the point cloud set carries a sample frame, and the sample frame is used to characterize the shape formed by the sample set of the object to be detected; Based on the comparison result between the first distance and the first threshold, multiple point cloud processing ranges are obtained, wherein the first threshold is used to divide the point clouds in the point cloud set; The downsampling retention rate of point clouds within the first point cloud processing range is adjusted to obtain a first sample set, wherein the first point cloud processing range is composed of near point clouds whose first distance is less than the first threshold within the point cloud processing range. Feature enhancement is performed on the point clouds within the second point cloud processing range to obtain a feature-enhanced second point cloud processing range, wherein the second point cloud processing range is composed of far point clouds whose first distance is greater than or equal to the first threshold within the point cloud processing range. The sample boxes within the processing range of the second point cloud after feature enhancement are weighted to obtain a second sample set; The first sample set and the second sample set are input into the target student network, and the detection result of the object to be detected is output. The target student network is a network formed after learning and inheriting the key features extracted by the teacher network.

2. The method of claim 1, wherein, The adjustment of the downsampling retention rate of the point cloud within the first point cloud processing range to obtain the first sample set includes: The point cloud within the first point cloud processing range is divided according to the polar radius range and polar angle range to obtain multiple division regions; Obtain the centroid polar radius value of the divided region to obtain the initial downsampling retention rate of the point cloud within the divided region; The initial downsampling retention rate is adjusted according to the polar angle direction of the divided region to obtain the target downsampling retention rate of the point cloud within the divided region; Based on the target downsampling retention rate, the point cloud within the divided region is downsampled to obtain the first sample set.

3. The method of claim 2, wherein, The point cloud within the first point cloud processing range is divided according to its polar radius and polar angle range to obtain multiple partitioned regions, including: The point cloud within the first point cloud processing range is divided according to the polar radius range to obtain the first division region; The point cloud within the first point cloud processing range is divided according to the polar angle range to obtain a second division region; The plurality of division regions are obtained based on the first division region and the second division region.

4. The method according to claim 2 or 3, characterized in that, The step of obtaining the centroid polar radius value of the segmented region to obtain the initial downsampling retention rate of the point cloud within the segmented region includes: Obtain the centroid polar radius value of the distance between the centroid of the divided region and the target vehicle; Based on the centroid polar radius value, the initial downsampling retention rate of the point cloud within the divided region is obtained, wherein the initial downsampling retention rate is inversely proportional to the centroid polar radius value.

5. The method according to claim 2 or 3, characterized in that, The step of adjusting the initial downsampling retention rate according to the polar angle direction of the divided region to obtain the target downsampling retention rate of the point cloud within the divided region includes: Based on the polar angle direction of the divided region, the correction parameter for the initial downsampling retention rate is obtained; The initial downsampling retention rate is adjusted according to the correction parameters to obtain the target downsampling retention rate of the point cloud within the divided region.

6. The method of claim 1, wherein, The step of performing feature enhancement on the point cloud within the second point cloud processing range to obtain the feature-enhanced second point cloud processing range includes: The point cloud within the second point cloud processing range is divided according to the polar radius range and polar angle range to obtain multiple far-distance division regions; Calculate the point cloud density value for each of the distantly divided regions, and filter out sparse sub-regions based on the point cloud density value and a preset density threshold; Based on the coordinate features and normal vector features of the point cloud corresponding to the same sample box within the sparse sub-region, a virtual point cloud is generated; The virtual point cloud is subjected to feature verification, and the valid virtual point cloud is added to the second point cloud processing range to obtain the feature-enhanced second point cloud processing range.

7. The method of claim 6, wherein, The sample boxes within the second point cloud processing range after feature enhancement are weighted to obtain a second sample set, including: Obtain the second distance between the sample box and the target vehicle within the second point cloud processing range after feature enhancement; Obtain the polar angle value of the sample box within the second point cloud processing range after feature enhancement; Based on the second distance, the polar angle value, and the preset weights of the sample boxes, the weighted weights of the sample boxes within the second point cloud processing range after feature enhancement are obtained. The second sample set is obtained based on the weighted weights and the sample boxes within the second point cloud processing range after feature enhancement.

8. The method according to claim 1, characterized in that, Before inputting the first sample set and the second sample set into the target student network, the method includes: Obtain historical single-frame point clouds, wherein the historical single-frame point clouds carry real bounding box samples; The historical single-frame point clouds are stitched together to form multiple dense stitched point clouds; The densely stitched point cloud is input into the teacher network, and the first prediction box, the first feature map, and the first confidence distribution are output. The historical single-frame point cloud is input into the student network, and the second prediction box, the second feature map, and the second confidence distribution are output. Based on the comparison results between the second predicted bounding box and the ground truth bounding box sample, the comparison results between the first feature map and the second feature map, and the comparison results between the first confidence distribution and the second confidence distribution, the network parameters of the student network are adjusted to obtain the target student network.

9. The method of claim 8, wherein, The process of adjusting the network parameters of the student network based on the comparison results of the second predicted bounding box and the ground truth bounding box samples, the comparison results of the first feature map and the second feature map, and the comparison results of the first confidence distribution and the second confidence distribution to obtain the target student network includes: The error between the second predicted bounding box and the ground truth bounding box sample in the student network is obtained to obtain the first loss; The error between the first confidence distribution and the second confidence distribution is obtained to obtain the second loss; The error between the first feature map and the second feature map is obtained to obtain the third loss; Based on the first loss, the second loss, and the third loss, the knowledge distillation loss is obtained; The network parameters of the student network are updated by backpropagation based on the knowledge distillation loss, so that the student network is closer to the teacher network, thus obtaining the target student network.

10. A device for object detection in autonomous driving, characterized in that, The device includes: The first acquisition module is used to acquire the first distance between the point cloud set of the object to be detected and the target vehicle. The point cloud set carries a sample frame, which is used to characterize the shape formed by the sample set of the object to be detected. The second acquisition module is used to acquire multiple point cloud processing ranges based on the comparison result between the first distance and the first threshold, wherein the first threshold is used to divide the point cloud in the point cloud set; The first obtaining module is used to adjust the downsampling retention rate of point clouds within a first point cloud processing range to obtain a first sample set, wherein the first point cloud processing range is composed of near point clouds whose first distance is less than the first threshold within the point cloud processing range, and to perform feature enhancement on point clouds within a second point cloud processing range to obtain a feature-enhanced second point cloud processing range, wherein the second point cloud processing range is composed of far point clouds whose first distance is greater than or equal to the first threshold within the point cloud processing range; The second obtaining module is used to perform weighted processing on the sample boxes within the processing range of the second point cloud after feature enhancement to obtain a second sample set; The detection module is used to input the first sample set and the second sample set into the target student network and output the detection result of the object to be detected, wherein the target student network is a network formed after learning and inheriting the key features extracted by the teacher network.

Images