A method and system for 3D target detection using low-beam laser point clouds based on dual-view collaborative distillation

Abstract

This invention discloses a method and system for 3D target detection based on low-beam laser point clouds using dual-view collaborative distillation. The method includes: constructing a dataset; projecting the point cloud onto front and top view perspectives based on intrinsic and extrinsic parameters; extracting features from the top and front view representations using a 3D backbone network and a 2D backbone network respectively, obtaining top and front view feature representations for a high-beam teacher model and a low-beam student model; dynamically adjusting the distillation weights of different perspective features through a parallel height distillation module to guide the student model to prioritize learning perspective features with sufficient structural information; dividing the feature space into resonance, inspiration, and correction regions and implementing differentiated distillation constraints through a unified space selection distillation module to achieve feature alignment between teacher and student models; and using the optimized lightweight student model for inference to output the target's 3D information. This method effectively solves the problem of low detection accuracy caused by the sparseness of low-beam point clouds and significantly improves detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a method and system for three-dimensional target detection using low-beam laser point clouds based on dual-view collaborative distillation, which is particularly suitable for lightweight three-dimensional environment understanding in industrial sensing, autonomous driving and robotics applications. Background Technology

[0002] In recent years, with the rapid development of technologies such as autonomous driving, industrial robots, intelligent manufacturing, and unmanned inspection, 3D environmental perception and target detection technologies based on LiDAR have received widespread attention. LiDAR can directly acquire the spatial geometric information of targets and has advantages such as being unaffected by lighting conditions and having high ranging accuracy, making it one of the key sensors in industrial perception systems. Among them, 3D target detection, as an important foundation for environmental understanding, provides necessary support for path planning, obstacle avoidance decisions, and safety control.

[0003] Currently, high-beam lidar exhibits high accuracy and stability in 3D target detection tasks due to its dense point cloud distribution and complete spatial structure information. However, the high cost and power consumption of such devices limit their widespread application in large-scale industrial scenarios and low-cost equipment. In contrast, low-beam lidar has advantages such as low cost, small size, and easy deployment, showing promising engineering application prospects. However, the point cloud data it acquires suffers from significant sparsity, with severe loss of local geometric structure and spatial details of the target, leading to a significant decrease in the accuracy and robustness of existing detection methods. Existing research often relies on single-viewpoint features such as bird's-eye views for feature modeling, making it difficult to fully characterize the target's structural information in the vertical direction, thus limiting further improvements in the performance of low-beam lidar point cloud 3D target detection.

[0004] Therefore, how to effectively utilize limited spatial information and fully explore the complementary characteristics between different viewpoints under low-beam laser point cloud conditions to improve the accuracy and stability of 3D target detection has become a key technical problem that urgently needs to be solved in the fields of industrial sensing and robotics. There is an urgent need for a 3D target detection method for low-beam laser radar that, while ensuring computational efficiency and system practicality, enhances the ability to represent the spatial structure of the target, in order to meet the application requirements for high-reliability environmental sensing in complex industrial scenarios. Summary of the Invention

[0005] To address the above technical problems, this invention provides a method and system for detecting three-dimensional targets in low-beam laser point clouds based on dual-view collaborative distillation.

[0006] The technical solution adopted by this invention to solve its technical problem is: A method for detecting 3D targets in low-beam laser point clouds based on dual-view collaborative distillation includes the following steps: S100: Acquire 3D point cloud data of the environment to be detected by high-beam lidar and low-beam lidar simultaneously, as well as the corresponding sensor calibration information and target ground truth annotation information, and construct training and validation datasets. S200: Based on point cloud data, the three-dimensional point cloud is projected using sensor intrinsic and extrinsic parameters to construct a front view perspective representation, which is used to characterize the structural and semantic features of the target in the vertical direction; at the same time, the original three-dimensional point cloud data is voxelized or columnarized to construct a top view perspective representation, which is used to characterize the spatial distribution features of the target in the horizontal direction. S300: Features are extracted from the top view and front view representations using a three-dimensional backbone network and a two-dimensional backbone network, respectively, to obtain the top view feature representations and front view feature representations of the high-beam teacher model and the low-beam student model. S400: The parallel height distillation module performs joint distillation of features from different perspectives of the teacher model and the student model. Based on structural complexity or uncertainty, the weights of features from different perspectives are dynamically adjusted in the distillation process, guiding the low-beam student model to prioritize learning perspective features with more structural information. S500: The unified space selection distillation module divides the feature responses of the teacher model and the student model into regions, dividing the feature space into resonance region, inspiration region and correction region, and implementing differentiated distillation constraints for different regions to achieve effective alignment of the teacher model and the student model in the scene-level feature space. S600: Uses a distilled and optimized lightweight low-wire student model to infer the input point cloud and output the target's 3D position, size, and category information.

[0007] Preferably, the construction of the front view perspective representation in S200 includes: For any laser point in the point cloud First, calculate its spherical distance. Pitch angle and azimuth : ; ; ; in, This represents the three-dimensional spatial coordinates of the laser point in the radar coordinate system. This indicates the corresponding laser reflection intensity; Obtain the horizontal field of view of the front view of the lidar. and vertical field of view [ , Mapping the spherical coordinates onto the two-dimensional front view plane yields the corresponding pixel indices (u, v): ; ; where \(H\) and \(W\) are the height and width of the front - view feature map respectively, denotes the floor operation; When the pixel index satisfies \(0\leq u < W\) and \(0\leq v < H\), it is considered that the current laser point falls into the valid front - view area, and the radial distance of the current laser point in the horizontal plane is calculated : ; and a pixel - level front - view feature vector is constructed : ; where, is the height information of the laser point, is the horizontal radial distance of the laser point, is the reflection intensity of the laser point; Through the above - mentioned front - view projection and feature construction process, the original disordered and sparse 3D point cloud is transformed into a regular 2D representation.

[0008] Preferably, the parallel height distillation module in S400 includes an introduced structural entropy estimation module, which decouples geometric density and prediction uncertainty, dynamically generates spatial weights and information weights for each instance, and adaptively balances the feature contributions of the top - view and the front - view.

[0009] Preferably, the generation of spatial weights includes: Using the projection matrix to generate the 3D bounding box and the front - view bounding box of each key instance \(i\), and defining the volume point density and the area pixel density: ; where and respectively represent the number of points within the \(i\) - th instance in the teacher model and the student model, and are the volumes of the 3D bounding boxes corresponding to the \(i\) - th instance in the teacher model and the student model respectively, and are the number of valid pixels of the front - view projection of the \(i\) - th instance in the teacher model and the student model respectively, and are the projected areas of the front - view bounding boxes of the \(i\) - th instance in the teacher model and the student model respectively; Calculating the final spatial weight through batch normalization: ; where represents the spatial weight of the \(i\) - th instance in the top - view distillation, This represents the spatial weight of the i-th instance in the front view distillation. This indicates taking the maximum value among all instance volume point densities. This indicates taking the maximum value among all instance area pixel densities, where j represents the instance index.

[0010] Preferably, the generation of information weights includes: Calculate the prediction entropy of the teacher model and the student model on the i-th instance. , : ; Where C represents the total number of target categories. and These represent the probabilities predicted by the teacher model and the student model that the i-th instance belongs to the k-th class, respectively. Construct joint information weights: ; in, The joint information weight of the i-th instance is used to amplify the distillation supervision strength of samples where both teachers and students exhibit high uncertainty.

[0011] Preferably, the parallel high-level distillation module in S400 further includes feature-level dual-knowledge transfer, and its distillation loss function is: ; ; ; in, This represents the distillation loss from the top-view perspective. This represents the distillation loss from the front view perspective. This represents the total loss of the parallel high-distillation module. and Let these represent the top-view features of the teacher model and the student model on the i-th instance, respectively. and Let these represent the front view features of the teacher model and the student model on the i-th instance, respectively. This represents the set of instances corresponding to high-beam radar. This represents the square of the L2 norm.

[0012] Preferably, in S500, the feature response of the teacher model and the student model is divided into regions by a unified spatial selection distillation module, dividing the feature space into resonance regions, inspiration regions, and correction regions, including: Construct a joint response map: ; in and These represent the spatial coordinates of the teacher model and the student model in the feature map, respectively. Fusion feature values ​​at the location, It is a symbolic function; Based on the joint response mapping, spatial locations are divided into three categories of regions: Resonance region This area indicates that the teacher model and the student model are responding simultaneously at the same location; Revelation Area This area indicates that only the teacher model responded while the students failed to respond; Correction area This area indicates that only the student model responded while the teacher did not. An adaptive normalization factor is introduced to balance the area differences between regions: ; in, Indicates the region type. Indicates the resonance region. Indicates the area of ​​revelation. Indicates the correction area. The area of ​​the corresponding region. For the maximum area, This represents the normalized weight of the corresponding region.

[0013] Preferably, S500 implements differentiated distillation constraints for different regions to achieve effective alignment of the teacher model and the student model in the scene-level feature space, including: Element-wise amplitude alignment is performed in the resonance region using L2 loss: ; in, Indicates loss in the resonance region. For the teacher model at the feature map location The squared L2 norm of the eigenvector at that location. The mask representing the resonance region, For the teacher model at the feature map location The eigenvector at that location; Topological pattern recovery was performed on the revelation region using a centralized Pearson correlation distance: ; in, Indicates loss in the revelation area. This represents the number of channels in the feature map. The mask representing the revelation region, For the student model at the feature map location The eigenvector at that location; The L1 norm is used to impose sparsity constraints on the correction region to suppress noise response. ; in, Indicates the loss in the correction area. A mask representing the correction area; The overall optimization objective of the unified spatial selection distillation module is: ; in, To ensure a uniform selection of the total loss of the distillation module, , , These represent the normalization factors for the resonance region, the inspiration region, and the correction region, respectively.

[0014] A low-beam laser point cloud 3D target detection system based on dual-view collaborative distillation includes: The data acquisition and construction module is used to acquire the 3D point cloud data of the environment to be detected by the high-beam lidar and low-beam lidar simultaneously, as well as the corresponding sensor calibration information and target ground truth annotation information, and to construct the training and validation dataset. The dual-view construction module is used to construct a front view perspective representation by projecting 3D point cloud data based on point cloud data and using sensor intrinsic and extrinsic parameters, and to construct a top view perspective representation by voxelizing or columnar partitioning the original 3D point cloud data. The feature extraction module is used to extract features from the top view perspective using a 3D backbone network to obtain the top view feature representation, and to extract features from the front view perspective using a 2D backbone network to obtain the front view feature representation. The parallel height distillation module is used to jointly distill the features of the high-beam teacher model and the low-beam student model from different perspectives, and dynamically adjusts the weights of features from different perspectives in the distillation process based on structural complexity or uncertainty. The unified space selection distillation module is used to divide the feature response of the teacher model and the student model into regions, dividing the feature space into resonance region, inspiration region and correction region, and implementing differentiated distillation constraints for different regions. The lightweight inference module is used to infer the input point cloud using a distilled and optimized low-wire student model, and output the target's 3D position, size, and category information.

[0015] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a three-dimensional target detection method based on low-beam laser point cloud using dual-view collaborative distillation.

[0016] The aforementioned method and system for 3D target detection based on low-beam laser point clouds, using dual-view collaborative distillation, fully combines the advantages of dual-view collaborative distillation perception in representing 3D structures with the efficiency of knowledge distillation in model compression and performance transfer. This effectively solves the problem of limited 3D target detection accuracy caused by sparse point clouds under low-beam lidar conditions. Specifically, the method acquires 3D point cloud data using both high-beam and low-beam lidar, constructing top-view and front-view representations to characterize the spatial structural features of the target in the horizontal and vertical directions, respectively. Subsequently, a parallel height distillation mechanism is introduced to evaluate the structural observability of the target under different viewpoints based on structural entropy, and dynamically adjusts the distillation weights to guide the low-beam model to prioritize learning viewpoint features with more complete structural information. Furthermore, a unified spatial selection distillation module divides the feature region into resonance, inspiration, and correction regions, and implements differentiated distillation constraints to achieve effective alignment of teacher and student models in scene-level feature responses. Finally, a lightweight model optimized by distillation is used to complete 3D target detection inference, outputting stable and accurate detection results without increasing additional inference overhead. This solution achieves significant improvements in detection accuracy, model efficiency, and engineering deployability, providing a reliable and feasible technical solution for the large-scale application of low-beam lidar in industrial sensing, mobile robots, and automated equipment. Attached Figure Description

[0017] Figure 1 This is a flowchart of a low-beam laser point cloud three-dimensional target detection method based on dual-view collaborative distillation in one embodiment of the present invention; Figure 2 This is a schematic diagram of the overall structure of a low-beam laser point cloud three-dimensional target detection method based on dual-view collaborative distillation in one embodiment of the present invention. Figure 3 This is a flowchart of a parallel high-efficiency distillation module in one embodiment of the present invention; Figure 4 This is a flowchart of a unified space selection module in one embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] In one embodiment, such as Figure 1 As shown, a method for detecting 3D targets in low-beam laser point clouds based on dual-view collaborative distillation includes the following steps: S100: Acquire 3D point cloud data of the environment to be detected by high-beam and low-beam lidar simultaneously, as well as the corresponding sensor calibration information and target ground truth annotation information, and construct training and validation datasets.

[0020] Specifically, in industrial scenarios, high-beam and low-beam lidars are deployed separately to simultaneously acquire 3D point cloud data of the environment to be inspected. The two types of lidars are rigidly connected and fixed, and undergo joint calibration to ensure the geometric consistency of the acquired data in a unified coordinate system.

[0021] During data acquisition, corresponding sensor intrinsic and extrinsic parameter calibration information is acquired simultaneously. Meanwhile, 3D ground truth annotation information of the target is generated based on manual annotation, including target category, spatial location, and scale parameters. Finally, the high-beam point cloud, low-beam point cloud, calibration information, and ground truth annotation information are uniformly organized to construct a multi-beam point cloud dataset for model training, validation, and testing.

[0022] S200: Based on point cloud data, a front-view representation is constructed by projecting the 3D point cloud using sensor intrinsic and extrinsic parameters. This representation is used to characterize the target's structural and semantic features in the vertical direction. Simultaneously, the original 3D point cloud data is voxelized or columnarized to construct a top-view representation, used to characterize the target's spatial distribution features in the horizontal direction. A schematic diagram of the overall structure is shown below. Figure 2 As shown.

[0023] In one embodiment, the construction of the front view perspective representation in S200 includes: For any laser point in the point cloud First, calculate its spherical distance. Pitch angle and azimuth : ; ; ; in, This represents the three-dimensional spatial coordinates of the laser point in the radar coordinate system. This indicates the corresponding laser reflection intensity; Obtain the horizontal field of view of the front view of the lidar. and vertical field of view [ , Mapping the spherical coordinates onto the two-dimensional front view plane yields the corresponding pixel indices (u, v): ; ; Where H and W are the height and width of the front view feature map, respectively. This indicates the floor function; When the pixel index satisfies \(0\leq u < W\) and \(0\leq v < H\), it is considered that the current laser point falls into the effective front view area, and the radial distance of the current laser point in the horizontal plane is calculated. : ; And construct a pixel-level front view feature vector. : ; Where, is the height information of the laser point, is the horizontal radial distance of the laser point, is the reflection intensity of the laser point; Through the above front view projection and feature construction process, the original unordered and sparse 3D point cloud is transformed into a regular 2D representation.

[0024] Furthermore, the original 3D point cloud data is processed by voxelization or columnar division. The point cloud space is divided into uniform grids in the X-Y plane, and each grid cell retains the point cloud distribution information along the Z-axis direction. For each grid cell, statistical features such as the number of points falling into it, average height, maximum height, reflection intensity, etc. are statistically calculated to form a top view feature representation for characterizing the spatial distribution characteristics of the target in the horizontal direction.

[0025] S300: Respectively use a 3D backbone network and a 2D backbone network to extract features from the top view representation and the front view representation, and obtain the top view feature representations and front view feature representations of the high beam teacher model and the low beam student model; Specifically, use a 3D backbone network to perform feature encoding and aggregation on the top view representation, extract the spatial distribution characteristics, geometric contour information of the target in the horizontal direction, and the relative position relationship between targets, and obtain the top view feature representations of the high beam teacher model and the low beam student model.

[0026] Use a 2D backbone network to extract features from the constructed front view representation. By performing multi-layer convolution operations and feature downsampling on the front view plane, learn the height distribution, structural form and semantic difference features of the target in the vertical direction, and obtain the front view feature representations of the high beam teacher model and the low beam student model.

[0027] S400: Through a parallel height distillation module, jointly distill the features of the teacher model and the student model from different perspectives, dynamically adjust the weights of different perspective features in the distillation process based on structural complexity or uncertainty, and guide the low beam student model to preferentially learn the perspective features with more sufficient structural information.

[0028] Specifically, considering the significant differences in feature saliency and geometric integrity of different targets from two perspectives, direct average distillation may be dominated by low-quality instances. Therefore, this invention introduces a structure entropy estimation module (SEEM).

[0029] In one embodiment, the parallel height distillation module in S400 includes an introduced structural entropy estimation module that dynamically generates spatial weights and information weights for each instance by decoupling geometric density from prediction uncertainty, adaptively balancing the feature contributions of the top view and the front view.

[0030] To suppress the interference of empty shell instances with scarce geometric information in the coefficient point cloud on distillation, this invention designs a density-based three-dimensional point cloud spatial weighting. and front view space weights In one embodiment, the generation of spatial weights includes: Generate the 3D bounding box and front view bounding box for each key instance i using the projection matrix, and define the volume point density and area pixel density: ; in and Let these represent the number of points in the i-th instance of the teacher model and the student model, respectively. and Let be the 3D bounding box volumes corresponding to the i-th instance in the teacher model and the student model, respectively. and These represent the effective pixel counts of the front view projection of the i-th instance of the teacher model and the student model, respectively. and These are the projected areas of the front view bounding boxes of the i-th instance of the teacher model and the student model, respectively. The final spatial weights are calculated using batch normalization: ; in This represents the spatial weight of the i-th instance in the top-view distillation. This represents the spatial weight of the i-th instance in the front view distillation. This indicates taking the maximum value among all instance volume point densities. This indicates taking the maximum value among all instance area pixel densities, where j represents the instance index.

[0031] In addition to geometric consistency, we further introduce semantic-level difficulty modeling through entropy to characterize the uncertainty of model predictions. In one embodiment, the generation of information weights includes: Calculate the prediction entropy of the teacher model and the student model on the i-th instance. , : ; Where C represents the total number of target categories. and These represent the probabilities predicted by the teacher model and the student model that the i-th instance belongs to the k-th class, respectively. To facilitate collaborative learning between teachers and students on difficult samples, joint information weights are constructed: ; in, The joint information weight of the i-th instance is used to amplify the distillation supervision strength of samples where both teachers and students exhibit high uncertainty.

[0032] This mechanism enables the distillation process to dynamically and adaptively adjust the intensity of supervision based on the "cognitive state" of teachers and students, thereby prioritizing instances that are challenging for both.

[0033] Furthermore, the parallel high-level distillation module in the S400 also includes feature-level dual knowledge transfer, and its distillation loss function is: ; ; ; in, This represents the distillation loss from the top-view perspective. This represents the distillation loss from the front view perspective. This represents the total loss of the parallel high-distillation module. and Let these represent the top-view features of the teacher model and the student model on the i-th instance, respectively. and Let these represent the front view features of the teacher model and the student model on the i-th instance, respectively. This represents the set of instances corresponding to high-beam radar. This represents the square of the L2 norm.

[0034] S500: By using the unified space selection distillation module, the feature response of the teacher model and the student model is divided into regions, the feature space is divided into resonance region, inspiration region and correction region, and differentiated distillation constraints are implemented for different regions to achieve effective alignment of the teacher model and the student model in the scene-level feature space.

[0035] In one embodiment, in S500, the feature response of the teacher model and the student model is divided into regions by a unified spatial selection distillation module, dividing the feature space into resonance regions, inspiration regions, and correction regions, including: Construct a joint response map to characterize the spatial correlation between teacher and student characteristic responses: ; in and These represent the spatial coordinates of the teacher model and the student model in the feature map, respectively. Fusion feature values ​​at the location, It is a symbolic function; Based on the joint response mapping, spatial locations are divided into three categories of regions: Resonance region This area indicates that the teacher model and the student model respond simultaneously at the same location, suggesting that both have a consistent focus on this spatial location. Revelation Area This area indicates that only the teacher model responded while the student failed to respond; it reflects structural information that the student has not yet learned but the teacher has mastered. Correction area This region represents a response generated only by the student model and no response from the teacher; it typically corresponds to a noisy response or pseudostructure in the student model. To alleviate the imbalance in spatial area among different regions, this invention further introduces an adaptive normalization factor to ensure that each region makes a comparable contribution to the loss calculation. In other words, an adaptive normalization factor is introduced to balance the area differences among regions. ; in, Indicates the region type. Indicates the resonance region. Indicates the area of ​​revelation. Indicates the correction area. The area of ​​the corresponding region. For the maximum area, This represents the normalized weight of the corresponding region.

[0036] In one embodiment, S500 implements differentiated distillation constraints for different regions to achieve effective alignment of the teacher model and student model in the scene-level feature space, thereby ensuring robust knowledge transfer, including: Both the teacher and student models exhibited effective activation, indicating a consensus on feature localization. The main difference lies in the response intensity; therefore, L2 loss was used for element-wise amplitude alignment in the resonant regions to ensure high-fidelity reproduction of teacher features by the student model. ; in, Indicates loss in the resonance region. For the teacher model at the feature map location The squared L2 norm of the eigenvector at that location. The mask representing the resonance region, For the teacher model at the feature map location The eigenvector at that location; The revelation region represents the students' blind spots, where they fail to capture the complex feature distribution present in the teacher. To guide students in recovering these missing topological patterns, this invention utilizes a centered Pearson correlation distance for topological pattern recovery: ; in, Indicates loss in the revelation area. This represents the number of channels in the feature map. The mask representing the revelation region, For the student model at the feature map location The eigenvector at that location; The correction region consists of student-specific hallucinations from the background region. To eliminate these artifacts, a sparsity constraint using the L1 norm is applied to the correction region to suppress the noise response. ; in, Indicates the loss in the correction area. The mask represents the correction region; unlike L2, L1 penalty can effectively suppress redundant activation values ​​to absolute zero, thereby suppressing false positive noise. Ultimately, the overall optimization objective of the unified spatial selection distillation consists of three parts: resonant distillation, heuristic distillation, and corrective distillation. This enables the student model to achieve fine alignment in consistent regions, effective learning in missing regions, and proactive correction in noisy regions. The overall optimization objective of the unified spatial selection distillation module is: ; in, To ensure a uniform selection of the total loss of the distillation module, , , These represent the normalization factors for the resonance region, the inspiration region, and the correction region, respectively.

[0037] S600: Uses a distilled and optimized lightweight low-wire student model to infer the input point cloud and output the target's 3D position, size, and category information.

[0038] Specifically, the student model employs a lightweight network structure, enabling real-time processing of point cloud data acquired by low-beam LiDAR while maintaining low computational complexity and storage overhead. The inference phase only requires input of the low-beam point cloud data, eliminating the need for additional teacher models or auxiliary computation modules, thus avoiding any increase in system inference latency. The low-beam model performs feature extraction, feature fusion, and target prediction on the input point cloud, outputting the target's 3D spatial location, size parameters, and category information, forming the final 3D target detection result.

[0039] Compared with the prior art, the present invention has the following advantages: (1) Dual-view collaborative representation: Simultaneously construct top view and front view representation, make full use of the structural information of the target in the horizontal and vertical directions, and make up for the information loss caused by the sparsity of low-line point cloud.

[0040] (2) Parallel height-guided distillation: Spatial weights and information weights are dynamically generated through the structural entropy estimation module, guiding the student model to prioritize learning the perspective features with more complete structural information, thereby improving distillation efficiency.

[0041] (3) Differentiated spatial selection distillation: The feature space is divided into three types of regions: resonance, inspiration and correction. Customized constraints are applied to the characteristics of different regions to suppress noise response while preserving the teacher's semantic information, thereby achieving refined knowledge transfer.

[0042] (4) Lightweight inference: The student model optimized by distillation does not need to introduce teacher model or auxiliary module in the inference stage, maintaining low computational complexity and storage overhead, and has good engineering deployability.

[0043] (5) Significantly improved accuracy: Under the condition of low beam lidar, the accuracy of three-dimensional target detection is significantly improved, providing a reliable technical solution for application scenarios such as industrial sensing and mobile robots.

[0044] In one embodiment, a low-beam laser point cloud 3D target detection system based on dual-view collaborative distillation is also provided, comprising: The data acquisition and construction module is used to acquire the 3D point cloud data of the environment to be detected by the high-beam lidar and low-beam lidar simultaneously, as well as the corresponding sensor calibration information and target ground truth annotation information, and to construct the training and validation dataset. The dual-view construction module is used to construct a front view perspective representation by projecting 3D point cloud data based on point cloud data and using sensor intrinsic and extrinsic parameters, and to construct a top view perspective representation by voxelizing or columnar partitioning the original 3D point cloud data. The feature extraction module is used to extract features from the top view perspective using a 3D backbone network to obtain the top view feature representation, and to extract features from the front view perspective using a 2D backbone network to obtain the front view feature representation. The parallel height distillation module is used to jointly distill the features of the high-beam teacher model and the low-beam student model from different perspectives, and dynamically adjusts the weights of features from different perspectives in the distillation process based on structural complexity or uncertainty. The unified space selection distillation module is used to divide the feature response of the teacher model and the student model into regions, dividing the feature space into resonance region, inspiration region and correction region, and implementing differentiated distillation constraints for different regions. The lightweight inference module is used to infer the input point cloud using a distilled and optimized low-wire student model, and output the target's 3D position, size, and category information.

[0045] Specific limitations regarding the low-beam laser point cloud 3D target detection system based on dual-view collaborative distillation can be found in the above-described limitations of the low-beam laser point cloud 3D target detection method based on dual-view collaborative distillation, and will not be repeated here. Each module in the aforementioned low-beam laser point cloud 3D target detection system based on dual-view collaborative distillation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0046] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a method for detecting three-dimensional targets in low-beam laser point clouds based on dual-view collaborative distillation.

[0047] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0048] The foregoing has provided a detailed description of a method and system for three-dimensional target detection based on low-beam laser point clouds using dual-view collaborative distillation, as provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A method for detecting three-dimensional targets using low-beam laser point clouds based on dual-view collaborative distillation, characterized in that, Includes the following steps: S100: Acquire 3D point cloud data of the environment to be detected by high-beam lidar and low-beam lidar simultaneously, as well as the corresponding sensor calibration information and target ground truth annotation information, and construct training and validation datasets. S200: Based on point cloud data, the three-dimensional point cloud is projected using sensor intrinsic and extrinsic parameters to construct a front view perspective representation, which is used to characterize the structural and semantic features of the target in the vertical direction; at the same time, the original three-dimensional point cloud data is voxelized or columnarized to construct a top view perspective representation, which is used to characterize the spatial distribution features of the target in the horizontal direction. S3 00: Features are extracted from the top view and front view representations using a three-dimensional backbone network and a two-dimensional backbone network, respectively, to obtain the top view feature representations and front view feature representations of the high-beam teacher model and the low-beam student model; S400: The parallel height distillation module performs joint distillation of features from different perspectives of the teacher model and the student model. Based on structural complexity or uncertainty, the weights of features from different perspectives are dynamically adjusted in the distillation process, guiding the low-beam student model to prioritize learning perspective features with more structural information. S500: The unified space selection distillation module divides the feature responses of the teacher model and the student model into regions, dividing the feature space into resonance region, inspiration region and correction region, and implementing differentiated distillation constraints for different regions to achieve effective alignment of the teacher model and the student model in the scene-level feature space. S600: Uses a distilled and optimized lightweight low-wire student model to infer the input point cloud and output the target's 3D position, size, and category information.

2. The method according to claim 1, characterized in that, The construction of the front view perspective representation in S200 includes: For any laser point in the point cloud First, calculate its spherical distance. Pitch angle and azimuth : ； ； ； in, This represents the three-dimensional spatial coordinates of the laser point in the radar coordinate system. This indicates the corresponding laser reflection intensity; Obtain the horizontal field of view of the front view of the lidar. and vertical field of view [ , Mapping the spherical coordinates onto the two-dimensional front view plane yields the corresponding pixel indices (u, v): ； ； Where H and W are the height and width of the front view feature map, respectively. This indicates the floor function; When the pixel index satisfies 0 ≤ u < W and 0 ≤ v < H, it is considered that the current laser point falls into the effective front view area, and the radial distance of the current laser point in the horizontal plane is calculated : ； And construct pixel-level front view feature vectors : ； in, For the height information of the laser point, The horizontal radial distance of the laser point. The intensity of the reflected light from the laser point; Through the aforementioned front view projection and feature construction process, the original disordered and sparse 3D point cloud is transformed into a regular 2D representation.

3. The method according to claim 2, characterized in that, The parallel height distillation module in S400 includes an introduced structural entropy estimation module, which dynamically generates spatial weights and information weights for each instance by decoupling geometric density and prediction uncertainty, and adaptively balances the feature contributions of the top view and the front view.

4. The method according to claim 3, characterized in that, The generation of spatial weights includes: Generate the 3D bounding box and front view bounding box for each key instance i using the projection matrix, and define the volume point density and area pixel density: ； in and Let these represent the number of points in the i-th instance of the teacher model and the student model, respectively. and Let be the 3D bounding box volumes corresponding to the i-th instance in the teacher model and the student model, respectively. and These represent the effective pixel counts of the front view projection of the i-th instance of the teacher model and the student model, respectively. and These are the projected areas of the front view bounding boxes of the i-th instance of the teacher model and the student model, respectively. The final spatial weights are calculated using batch normalization: ； in This represents the spatial weight of the i-th instance in the top-view distillation. This represents the spatial weight of the i-th instance in the front view distillation. This indicates taking the maximum value among all instance volume point densities. This indicates taking the maximum value among all instance area pixel densities, where j represents the instance index.

5. The method according to claim 4, characterized in that, The generation of information weights includes: Calculate the prediction entropy of the teacher model and the student model on the i-th instance. , : ； Where C represents the total number of target categories. and These represent the probabilities predicted by the teacher model and the student model that the i-th instance belongs to the k-th class, respectively. Construct joint information weights: ； in, The joint information weight of the i-th instance is used to amplify the distillation supervision strength of samples where both teachers and students exhibit high uncertainty.

6. The method according to claim 5, characterized in that, The parallel high-level distillation module in S400 further includes feature-level dual-knowledge transfer, and its distillation loss function is: ； ； ； in, This represents the distillation loss from the top-view perspective. This represents the distillation loss from the front view perspective. This represents the total loss of the parallel high-distillation module. and Let these represent the top-view features of the teacher model and the student model on the i-th instance, respectively. and Let these represent the front view features of the teacher model and the student model on the i-th instance, respectively. This represents the set of instances corresponding to high-beam radar. This represents the square of the L2 norm.

7. The method according to claim 6, characterized in that, In S500, the unified spatial selection distillation module partitions the feature responses of the teacher and student models into regions, dividing the feature space into resonance regions, inspiration regions, and correction regions, including: Construct a joint response map: ； in and These represent the spatial coordinates of the teacher model and the student model in the feature map, respectively. Fusion feature values ​​at the location, It is a symbolic function; Based on the joint response mapping, spatial locations are divided into three categories of regions: Resonance region This area indicates that the teacher model and the student model are responding simultaneously at the same location; Revelation Area This area indicates that only the teacher model responded while the students failed to respond; Correction area This area indicates that only the student model responded while the teacher did not. An adaptive normalization factor is introduced to balance the area differences between regions: ； in, Indicates the region type. Indicates the resonance region. Indicates the area of ​​revelation. Indicates the correction area. The area of ​​the corresponding region. For the maximum area, This represents the normalized weight of the corresponding region.

8. The method according to claim 7, characterized in that, S500 implements differentiated distillation constraints for different regions to achieve effective alignment of teacher and student models in the scene-level feature space, including: Element-wise amplitude alignment is performed in the resonance region using L2 loss: ； in, Indicates loss in the resonance region. For the teacher model at the feature map location The squared L2 norm of the eigenvector at that location. The mask representing the resonance region, For the teacher model at the feature map location The eigenvector at that location; Topological pattern recovery was performed on the revelation region using a centralized Pearson correlation distance: ； in, Indicates loss in the revelation area. This represents the number of channels in the feature map. The mask representing the revelation region, For the student model at the feature map location The eigenvector at that location; The L1 norm is used to impose sparsity constraints on the correction region to suppress noise response. ； in, Indicates the loss in the correction area. A mask representing the correction area; The overall optimization objective of the unified spatial selection distillation module is: ； in, To ensure a uniform selection of the total loss of the distillation module, , , These represent the normalization factors for the resonance region, the inspiration region, and the correction region, respectively.

9. A three-dimensional target detection system based on low-beam laser point cloud using dual-view collaborative distillation, characterized in that, include: The data acquisition and construction module is used to acquire the 3D point cloud data of the environment to be detected by the high-beam lidar and low-beam lidar simultaneously, as well as the corresponding sensor calibration information and target ground truth annotation information, and to construct the training and validation dataset. The dual-view construction module is used to construct a front view perspective representation by projecting 3D point cloud data based on point cloud data and using sensor intrinsic and extrinsic parameters, and to construct a top view perspective representation by voxelizing or columnar partitioning the original 3D point cloud data. The feature extraction module is used to extract features from the top view perspective using a 3D backbone network to obtain the top view feature representation, and to extract features from the front view perspective using a 2D backbone network to obtain the front view feature representation. The parallel height distillation module is used to jointly distill the features of the high-beam teacher model and the low-beam student model from different perspectives, and dynamically adjusts the weights of features from different perspectives in the distillation process based on structural complexity or uncertainty. The unified space selection distillation module is used to divide the feature response of the teacher model and the student model into regions, dividing the feature space into resonance region, inspiration region and correction region, and implementing differentiated distillation constraints for different regions. The lightweight inference module is used to infer the input point cloud using a distilled and optimized low-wire student model, and output the target's 3D position, size, and category information.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images