Point cloud target detection method and system fusing 2D features, and electronic equipment

Abstract

The invention relates to a point cloud target detection method and system fusing 2D features, and electronic equipment. The method comprises the following steps: performing voxelization processing on collected original point cloud data to generate a three-dimensional voxel feature tensor; respectively projecting the three-dimensional voxel feature tensors to three orthogonal plane view angles of front view, bird's-eye view and side view to generate a multi-view angle feature map; performing cross-view feature alignment and splicing on the multi-view feature map to generate a composite feature map; optimizing the composite feature map by using a channel attention weighting mechanism to obtain an optimized composite feature map; and target detection is carried out based on the optimized composite feature map, so that the technical problem that the detection recall rate of the target is reduced due to three-dimensional space information loss in a BEV perspective in a traditional method is solved, and the technical effect of improving the detection recall rate of the target in a sparse point cloud scene is achieved.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a point cloud target detection method, system and electronic equipment integrating 2D features. Background Art

[0002] With the rapid development of autonomous driving technology, high-precision environmental perception has become a core challenge in achieving Level 4 / L5 autonomous driving. LiDAR and millimeter-wave radar, as key sensors, provide vehicles with three-dimensional spatial perception through point cloud data and electromagnetic wave reflection information. Point cloud detection algorithms such as CenterPoint convert point clouds into BEV (bird's-eye view) features, enabling accurate detection and tracking of vehicles, pedestrians, and other targets. Radar perception is irreplaceable in speed measurement, obstacle classification, and robustness against adverse weather conditions. Existing technologies, such as patent (CN119600561A), utilize CenterPoint for point cloud object detection, but their shortcomings include: 1) sensitivity to the sparse point clouds generated by low-line-count LiDARs (e.g., 16 lines), making them prone to missed detections at long distances or in dense traffic; 2) reliance on a single motion model for trajectory prediction, significantly reducing tracking accuracy when the target suddenly changes lanes or brakes; and 3) ineffective integration of radar velocity information, resulting in significant deviations in heading estimates for high-speed moving targets. Another example is the patent (CN119600264A) that proposes a multi-radar joint calibration scheme, but its limitations are: 1) the millimeter-wave radar point cloud density is low, and classification relies on manually designed features, resulting in a high pedestrian false detection rate; 2) the hardware requires a 4D imaging radar and a high-precision inertial navigation system, which is costly.

[0003] In the field of 3D target detection based on point clouds, traditional algorithms represented by CenterPoint usually use voxelized feature extraction methods, that is, converting the original point cloud data into a regular three-dimensional voxel grid to extract features, and then projecting the voxel features to the bird's-eye view (BEV) perspective for target positioning regression and loss calculation. However, this feature extraction method has inherent defects: when detecting targets directly in front of the radar, the point cloud density is insufficient because the target can only reflect a single-sided point cloud. At this time, relying solely on the feature projection from the BEV perspective will cause partial loss of three-dimensional spatial information. Especially in the feature extraction stage, a single-sided point cloud cannot effectively represent the complete geometric features of the target, making it difficult for the network to learn discriminative feature expressions, and ultimately resulting in a significant reduction in the model's detection recall rate for such targets. The essence of this phenomenon is that traditional methods find it difficult to effectively fuse multi-plane point cloud features from the BEV perspective. When faced with detection scenarios with sparse point clouds, the representation ability of the feature space is severely restricted.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a point cloud target detection method, system and electronic device that integrate 2D features, so as to at least solve the technical problem that the traditional method lacks three-dimensional spatial information from the BEV perspective, resulting in a reduced target detection recall rate.

[0006] According to one aspect of an embodiment of the present application, a point cloud target detection method integrating 2D features is provided, including: performing voxel processing on the collected original point cloud data to generate a three-dimensional voxel feature tensor; projecting the three-dimensional voxel feature tensor to three orthogonal plane perspectives of front view, bird's-eye view and side view respectively to generate a multi-perspective feature map; performing cross-perspective feature alignment and splicing on the multi-perspective feature map to generate a composite feature map; optimizing the composite feature map using a channel attention weighting mechanism to obtain an optimized composite feature map; and performing target detection based on the optimized composite feature map.

[0007] Optionally, the collected original point cloud data is voxelized to generate a three-dimensional voxel feature tensor, including: dividing the original point cloud data into a regular three-dimensional voxel grid, and extracting the geometric features of non-empty voxels to generate the three-dimensional voxel feature tensor; wherein the geometric features include at least one of the following: point density, reflection intensity.

[0008] Optionally, the multi-view feature map includes a front view (FV) feature map, a bird's-eye view (BEV) feature map and a side view (SV) feature map, and the three-dimensional voxel feature tensor is projected onto three orthogonal plane viewpoints of the front view, bird's-eye view and side view respectively to generate a multi-view feature map, including: performing a layer-by-layer slicing operation on the three-dimensional voxel feature tensor along the Y-axis to generate the front view feature map; performing maximum pooling on the three-dimensional voxel feature tensor along the Z-axis to generate the bird's-eye view feature map; and performing maximum pooling on the three-dimensional voxel feature tensor along the X-axis to generate the side view feature map.

[0009] Optionally, cross-view feature alignment and splicing are performed on the multi-view feature maps to generate a composite feature map, including: unifying the multi-view feature maps to the same size using bilinear interpolation or deconvolution, and performing feature map splicing along the channel axis to synthesize the composite feature map.

[0010] Optionally, the composite feature map is optimized using a channel attention weighting mechanism to obtain an optimized composite feature map, including: performing channel attention weighting on the composite feature map to generate channel attention weights; and multiplying the channel attention weights by the composite feature map to obtain the optimized composite feature map.

[0011] Optionally, the method also includes: optimizing the detection target through a direction-sensitive loss function; the direction-sensitive loss function includes the following components: classification loss, regression loss, 3D intersection over union (IoU) alignment loss and 2D projection IoU loss; wherein, the direction-sensitive loss function dynamically adjusts the regression loss weights of targets with different orientations through the composite sine value of the rotation angle of the target box and the azimuth angle of the position vector.

[0012] Optionally, the method further includes: adopting an improved multi-target tracking framework to track targets through motion-compensated Kalman prediction matching, IoU association matching of high-threshold detection frames, and appearance feature similarity matching of low-threshold detection frames.

[0013] Optionally, the improved multi-target tracking framework adopts the following trajectory state management mechanism: when the confidence of the detection frame exceeds the tracking threshold, a new trajectory is created and the state parameters are set. In subsequent frame processing, only unmatched detection frames with confidence higher than a high threshold are allowed to initialize new trajectories, and the active state trajectory and the lost state trajectory are fused through a filter to complete the motion state estimation; a multi-level matching mechanism is used to associate detection frames of different confidence levels respectively; when the trajectory is in the lost state for more than a set maximum number of frames, the trajectory is deleted, and the ID duplication detection mechanism is started when the trajectory state is transferred to forcibly delete redundant trajectories with the same identity.

[0014] According to another aspect of an embodiment of the present application, a point cloud target detection system that integrates 2D features is provided, including: a first generation module for voxelizing the collected original point cloud data to generate a three-dimensional voxel feature tensor; a second generation module for projecting the three-dimensional voxel feature tensor to three orthogonal plane perspectives of front view, bird's-eye view and side view, respectively, to generate a multi-perspective feature map; a third generation module for performing cross-perspective feature alignment and splicing on the multi-perspective feature map to generate a composite feature map; a feature optimization module for optimizing the composite feature map using a channel attention weighting mechanism to obtain an optimized composite feature map; and a target detection module for performing target detection based on the optimized composite feature map.

[0015] According to another aspect of an embodiment of the present application, an electronic device is provided, including: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the method described above.

[0016] In an embodiment of the present application, voxelization is performed on the collected original point cloud data to generate a three-dimensional voxel feature tensor; the three-dimensional voxel feature tensor is projected to three orthogonal plane perspectives of front view, bird's eye view and side view respectively to generate a multi-perspective feature map; the multi-perspective feature map is aligned and spliced ​​across perspectives to generate a composite feature map; the composite feature map is optimized using a channel attention weighting mechanism to obtain an optimized composite feature map; target detection is performed based on the optimized composite feature map, that is, the present application realizes target detection through multi-perspective feature fusion, thereby solving the technical problem that the traditional method lacks three-dimensional spatial information under the BEV perspective, resulting in a reduced detection recall rate of the target, and achieves the technical effect of improving the detection recall rate of the target in the sparse point cloud scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other embodiments can be derived from these drawings without inventive effort.

[0018] Figure 1 A flowchart of a point cloud target detection method integrating 2D features provided in an embodiment of the present application;

[0019] Figure 2 A flowchart of an improved vehicle tracking algorithm provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a point cloud object detection system integrating 2D features provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the embodiments of the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the embodiments of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0023] According to one aspect of an embodiment of the present application, a point cloud target detection method integrating 2D features is provided. Figure 1A flowchart of a point cloud target detection method integrating 2D features provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes the following steps:

[0024] Step S102 : voxelize the collected original point cloud data to generate a three-dimensional voxel feature tensor.

[0025] In step S104 , the three-dimensional voxel feature tensor is projected onto three orthogonal plane perspectives: front view, bird's eye view, and side view, to generate a multi-perspective feature map.

[0026] Step S106: perform cross-view feature alignment and splicing on the multi-view feature maps to generate a composite feature map.

[0027] Step S108: Use the channel attention weighting mechanism to optimize the composite feature map to obtain an optimized composite feature map.

[0028] Step S110: performing target detection based on the optimized composite feature map.

[0029] In an embodiment of the present application, voxelization is performed on the collected original point cloud data to generate a three-dimensional voxel feature tensor; the three-dimensional voxel feature tensor is projected to three orthogonal plane perspectives of front view, bird's eye view and side view respectively to generate a multi-perspective feature map; the multi-perspective feature map is aligned and spliced ​​across perspectives to generate a composite feature map; the composite feature map is optimized using a channel attention weighting mechanism to obtain an optimized composite feature map; target detection is performed based on the optimized composite feature map, that is, the present application realizes target detection through multi-perspective feature fusion, thereby solving the technical problem that the traditional method lacks three-dimensional spatial information under the BEV perspective, resulting in a reduced detection recall rate of the target, and achieves the technical effect of improving the detection recall rate of the target in the sparse point cloud scenario.

[0030] As an optional embodiment, the collected original point cloud data is voxelized to generate a three-dimensional voxel feature tensor, including: dividing the original point cloud data into a regular three-dimensional voxel grid, and extracting the geometric features of non-empty voxels to generate a three-dimensional voxel feature tensor; wherein the geometric features include at least one of the following: point density, reflection intensity.

[0031] Optionally, the original point cloud data is divided into a regular three-dimensional voxel grid. For example, the space can be divided into voxel units with a side length of 0.1m×0.1m×0.2m. Each voxel unit corresponds to a fixed spatial region. For each non-empty voxel (i.e., a voxel containing point cloud data), its geometric features are extracted. The geometric features may include geometric feature information such as point density and reflection intensity, which are used to describe the point cloud characteristics within the voxel. Among them, the point density is calculated by calculating the number of point clouds in each voxel to reflect the density of the point clouds in the area. The reflection intensity is calculated by calculating the average reflection intensity of the point clouds in each voxel to reflect the reflection characteristics of the area. The extracted geometric features are combined into a three-dimensional tensor, where each voxel corresponds to a feature vector, which contains information such as point density and reflection intensity. For example, the three-dimensional voxel feature tensor F 3D ∈R H×W×D×C , where H, W, and D are the voxel space dimensions and C is the number of channels.

[0032] It should be noted that voxelization is to divide the three-dimensional space into regular grid units (voxels) and allocate point cloud data to these voxels.

[0033] In an embodiment of the present application, irregular point cloud data is converted into a regular three-dimensional voxel grid through voxelization, which facilitates subsequent feature extraction and processing. In particular, for sparse point clouds generated by low-line-count lidars, voxelization can effectively organize data and avoid detection problems caused by uneven distribution of point clouds. Extracting geometric features such as point density and reflection intensity can describe the characteristics of point clouds from multiple angles. Point density can reflect the shape and size of the target, while reflection intensity can provide information about the material and surface properties of the target. The combination of these features can significantly improve the model's ability to recognize targets, especially in complex scenarios. Voxelization discretizes continuous point cloud data, reducing the amount of data that needs to be processed, and also making the feature extraction process more efficient. This is especially important in autonomous driving scenarios with high real-time requirements.

[0034] As an optional embodiment, the multi-view feature map includes a front view feature map, a bird's-eye view feature map and a side view feature map, and the three-dimensional voxel feature tensor is projected onto three orthogonal plane viewpoints of the front view, the bird's-eye view and the side view, respectively, to generate a multi-view feature map, including: performing a layer-by-layer slicing operation on the three-dimensional voxel feature tensor along the Y-axis to generate a front view feature map; performing maximum pooling on the three-dimensional voxel feature tensor along the Z-axis to generate a bird's-eye view feature map; and performing maximum pooling on the three-dimensional voxel feature tensor along the X-axis to generate a side view feature map.

[0035] Optionally, the 3D voxel feature tensor is sliced ​​layer by layer along the Y-axis. Specifically, the 3D voxel feature tensor is sliced ​​layer by layer along the Y-axis, with each slice retaining information from the X- and Z-axes, forming a 2D feature map. Convolutional encoding is performed on each slice to extract features. The convolution operation captures local features and enhances feature expressiveness. The 2D feature maps of all layers are stacked to form a front-view feature map.

[0036] Optionally, a max pooling operation is performed on the 3D voxel feature tensor along the Z axis. This operation retains the maximum value of each voxel along the Z axis, generating a 2D feature map that reflects the perspective from above. Convolutional encoding is performed on the resulting 2D feature map to extract features and form a bird's-eye view feature map.

[0037] Optionally, a max pooling operation is performed on the 3D voxel feature tensor along the X-axis. This operation retains the maximum value of each voxel along the X-axis, generating a 2D feature map that reflects the side view. Convolutional encoding is performed on the resulting 2D feature map to extract features and form a side view feature map.

[0038] Traditional methods typically rely solely on a bird's-eye view for target detection. However, when detecting targets directly in front of the radar, the target only reflects a single-sided point cloud, resulting in insufficient point cloud density and an inability to effectively characterize the target's complete geometric features. In the embodiments of this application, by introducing front-view and side-view feature maps, target features can be captured from multiple angles, reducing missed detections caused by sparse point clouds. This significantly improves the detection recall rate of targets directly in front of low-line-count lidars, especially at long distances and in dense traffic scenarios.

[0039] In addition, multi-view feature maps (front view, bird's eye view, and side view) can capture the geometric features of the target from different directions. The front view feature map can capture the front information of the target, the bird's eye view feature map can capture the top information of the target, and the side view feature map can capture the side information of the target. In the embodiment of the present application, by fusing these feature maps, the three-dimensional geometric features of the target can be more comprehensively characterized, thereby improving the model's ability to recognize the target, especially when dealing with targets of complex shapes, it can more accurately describe the target's geometric structure.

[0040] In the embodiments of this application, the dimensionality of the feature map is reduced by projecting the 3D voxel feature tensor onto a 2D plane. The processing complexity of a 2D feature map is much lower than that of a 3D feature map, while retaining key geometric information. This significantly reduces computational complexity without compromising detection accuracy, improving the system's real-time performance and efficiency.

[0041] It should be noted that the layer-by-layer slicing operation is to slice the 3D voxel feature tensor layer by layer along a certain axis (such as the Y axis), and each layer of slices retains the information of the other two axes (such as the X axis and the Z axis). This method can generate a 2D feature map to capture features in a specific direction.

[0042] Max pooling is a downsampling operation that preserves important features by taking the maximum value of a local region. In this embodiment, max pooling is used to compress a 3D voxel feature tensor into a 2D feature map while preserving key information.

[0043] Multi-view feature maps are feature maps generated from different viewpoints (such as front view, bird's eye view, and side view). These feature maps can provide different geometric information and help to more comprehensively characterize the 3D features of the target.

[0044] As an optional embodiment, cross-view feature alignment and splicing are performed on multi-view feature maps to generate a composite feature map, including: using bilinear interpolation or deconvolution to unify the multi-view feature maps to the same size, and splicing the feature maps along the channel axis to synthesize the composite feature map.

[0045] Since feature maps from different perspectives may have different resolutions, they need to be resized to the same size for subsequent stitching operations. One of the following two methods can be used: 1) Upsample or downsample the feature map through bilinear interpolation to reach the target size. Bilinear interpolation is a commonly used image scaling method that can adjust the resolution while maintaining image smoothness. 2) Use deconvolution operations to upsample the feature map. Deconvolution can more effectively restore high-resolution feature maps while retaining more detail information by learning the inverse mapping relationship of feature maps. Align the resized multi-view feature maps to the same coordinate system to ensure that they are spatially consistent.

[0046] The aligned multi-view feature maps are concatenated along the channel axis. Specifically, the number of channels in the front-view feature map, the bird's-eye view feature map, and the side-view feature map are summed to form a composite feature map. For example, if each view's feature map has C channels, the concatenated composite feature map will have 3C channels. By concatenating along the channel axis, the resulting composite feature map incorporates feature information from different viewpoints, enabling a more comprehensive representation of the target's 3D geometry.

[0047] In an embodiment of the present application, by splicing multi-perspective feature maps into a composite feature map, the model can simultaneously utilize feature information from front view, bird's-eye view, and side view perspectives. This multi-perspective fusion method can more comprehensively describe the three-dimensional geometric structure of the target, especially when processing sparse point cloud data, and can effectively supplement the missing feature information under a single perspective, thereby significantly improving the model's detection accuracy for the target, especially in complex scenes, such as target detection and small target detection directly in front of a low-line-count lidar. In addition, the fusion of multi-perspective feature maps can reduce the problem of missed detection caused by sparse point clouds. For example, the target directly in front may not be effectively detected from a bird's-eye view, but more feature information may be captured from front view and side view perspectives. Through cross-perspective feature alignment and splicing, this information is integrated, thereby improving the recall rate of detection. In sparse point cloud scenarios, the detection recall rate is significantly improved, especially in long-distance and dense traffic scenarios.

[0048] As an optional embodiment, a channel attention weighting mechanism is used to optimize the composite feature map to obtain an optimized composite feature map, including: performing channel attention weighting on the composite feature map to generate channel attention weights; and multiplying the channel attention weights with the composite feature map to obtain an optimized composite feature map.

[0049] A global average pooling operation is performed on the composite feature map F (assuming its shape is H×W×C, where H and W are the height and width of the feature map, and C is the number of channels). This operation compresses the feature map of each channel into a scalar value. The vector obtained by global average pooling is then passed through a fully connected layer (also known as a "Squeeze" operation), a ReLU activation function, a fully connected layer (also known as an "Expand" operation), and finally a Sigmoid activation function to generate channel attention weights. Finally, the generated channel attention weights are multiplied by the composite feature map channel by channel to obtain the optimized composite feature map.

[0050] In an embodiment of the present application, the channel attention mechanism can automatically identify feature channels that are more helpful for target detection tasks by learning the importance weights of each channel, and enhance the features of these channels while suppressing those unimportant channels. This mechanism enables the model to pay more attention to key features and reduce the interference of redundant information. This significantly improves the model's detection accuracy for targets, especially in complex backgrounds and multi-target scenarios, and can more accurately identify the key features of the target, reducing false detections and missed detections. In addition, the channel attention mechanism can enable the model to adaptively adjust the representation of features under different input conditions by dynamically adjusting the channel weights. This adaptive capability enables the model to better extract and utilize effective features when facing different scenarios and targets. The generalization ability in different scenarios is significantly improved, and target detection can be performed more stably, especially when facing sparse point cloud data and complex traffic scenarios.

[0051] It should be noted that the optimized composite feature map is a feature map after channel attention weighting, in which key features are enhanced and redundant information is suppressed, which can be more effectively used for target detection tasks.

[0052] As an optional embodiment, the above method also includes: optimizing the detection target through a direction-sensitive loss function; the direction-sensitive loss function includes the following components: classification loss, regression loss, 3D IoU alignment loss and 2D projection IoU loss; wherein, the direction-sensitive loss function dynamically adjusts the regression loss weights of targets with different orientations through the composite sine value of the rotation angle of the target box and the azimuth angle of the position vector.

[0053] Optionally, in the embodiment of the present application, the improved CenterPoint loss function adds 3D IoU loss and 2D projection IoU loss on the basis of the original classification loss and regression loss. The overall formula can be expressed as:

[0054]

[0055] Among them, λ1 and λ2 are the weight coefficients of the corresponding loss components.

[0056] Furthermore, the expression of Heatmap classification loss (Focal Loss) is:

[0057]

[0058] Where n is the total number of samples; p xyc To predict the probability of the heat map at the (x, y) position category c; y xyc is the real heat map annotation; α, γ are balance parameters (default α=0.25, γ=2).

[0059] Furthermore, the expression of target regression loss (Smooth L1 Loss) is:

[0060]

[0061] Among them, N pos is the total number of positive samples; p k is the prediction parameter (center offset / size / rotation, etc.); g k is the real annotation parameter; ω k is the parameter weight (code_weights); x, y, z, w, l, h, θ are the 3D target box information, the center point coordinates x, y, z, width, height w, l, h, and the orientation angle θ.

[0062] Furthermore, the expression of 3D IoU alignment loss is:

[0063]

[0064] Among them, N pos is the total number of positive samples; To predict the 3D IoU value; To predict 3D box information; Label 3D frame information.

[0065] Furthermore, the expression of 2D projection IoU loss (YZ plane projection) is:

[0066]

[0067] Among them, N pos is the total number of positive samples; To predict the 2D IoU value; It is the projected 2D frame information.

[0068] In an embodiment of the present application, the direction-sensitive loss function dynamically adjusts the regression loss weights of targets in different orientations by introducing the synthetic sine value of the rotation angle of the target frame and the azimuth angle of the position vector as a weight. This mechanism enables the model to pay more attention to the orientation information of the target, thereby improving the detection accuracy of multi-angle distributed targets, and significantly improving the model's robustness in orientation perception of multi-angle distributed targets on the axial side of the lidar, especially when dealing with high-speed moving targets (such as vehicles that change lanes suddenly), reducing the deviation of heading estimation. In addition, by combining 3D IoU alignment loss and 2D projection IoU loss, not only can the prediction of the 3D target frame be optimized, but the detection accuracy can also be further improved by optimizing the 2D projection frame. The robustness in complex scenes is significantly improved, and target detection and tracking can be performed more stably, especially in bad weather and dense traffic scenes.

[0069] As an optional embodiment, the above method also includes: adopting an improved multi-target tracking framework to track targets through motion-compensated Kalman prediction matching, IoU association matching of high-threshold detection boxes, and appearance feature similarity matching of low-threshold detection boxes.

[0070] For each detected object, a Kalman filter is initialized to predict the object's position and velocity in the next frame. Based on the Kalman filter's predictions, the object's motion is compensated. This step can reduce prediction bias caused by rapid object movement or sensor noise. In the next frame, the Kalman filter's predictions are used to match the detected object box. If the IoU between the detected and predicted boxes exceeds a set threshold (e.g., 0.5), they are considered a match.

[0071] For predicted tracks that were not successfully matched in the Kalman prediction matching, their IoU values ​​are calculated with the detection boxes of the current frame. Detection boxes with IoU values ​​above a high threshold (such as 0.7) are selected and associated with the predicted tracklets. This step ensures that only highly similar objects are matched, thus reducing the possibility of false matches.

[0072] For predicted tracks that fail to match the high-threshold IoU match, their appearance features (such as color and texture) are extracted. The similarity between the predicted track's appearance features and the appearance features of the detection box in the current frame is calculated. Detection boxes with a similarity above a low threshold (such as 0.3) are selected and associated with the predicted track. This step is used to handle cases where the targets have similar appearance but large positional deviations, further improving tracking robustness.

[0073] In this embodiment of the application, by combining motion-compensated Kalman predictive matching, high-threshold IoU correlation matching, and low-threshold appearance feature similarity matching, the framework can associate objects from multiple dimensions. This multi-stage matching mechanism can effectively reduce missed detections and false matches caused by object occlusion, rapid movement, or sensor noise.

[0074] As an optional embodiment, the improved multi-target tracking framework adopts the following trajectory state management mechanism: when the confidence of the detection frame exceeds the tracking threshold, a new trajectory is created and the state parameters are set. In subsequent frame processing, only unmatched detection frames with confidence higher than the high threshold are allowed to initialize new trajectories, and the active state trajectory and the lost state trajectory are fused through a filter to complete the motion state estimation; a multi-level matching mechanism is used to associate detection frames of different confidence levels respectively; when the trajectory is in the lost state for more than a set maximum number of frames, the trajectory is deleted, and the ID duplication detection mechanism is activated when the trajectory state is transferred to forcibly delete redundant trajectories with the same identity.

[0075] When the confidence score of the detection box exceeds the tracking threshold (e.g. track_thresh = 0.6), a new track is created. The track state (State) is set to "Tracked". The activation flag (is_activated) is set to True, indicating that the track is active.

[0076] In subsequent frames, only unmatched detection boxes with a confidence level higher than a high threshold (e.g., high_thresh = 0.7) are allowed to initialize new trajectories. The state of the new trajectory is set to "Tracked," but the activation flag (is_activated) is set to False, indicating that the trajectory has not yet been fully activated. For activated trajectories (is_activated = True) and lost trajectories (state = Lost), a Kalman filter is used to complete motion state estimation. The Kalman filter predicts the position and velocity of the target in the current frame based on the target's motion model.

[0077] IoU matching is performed on detection boxes with confidence higher than the tracking threshold (track_thresh), and the IoU threshold is set to 0.5. If the IoU between the detection box and the predicted track is greater than 0.5, they are associated.

[0078] For the predicted tracks that are not successfully matched in the primary matching, IoU matching is performed again with the detection boxes whose confidence is lower than the tracking threshold (track_thresh).

[0079] For unmatched tracks, we use appearance feature similarity matching. We calculate the similarity between the track's appearance features and the appearance features of the detection box, and select the detection boxes with a similarity above a low threshold (e.g., 0.3) for matching.

[0080] If a track is not matched to any detection box within a continuous number of frames (such as 30 frames), its status is set to "Lost".

[0081] If a track remains in the "lost" state for more than a maximum number of frames (e.g. 30 frames), it is removed from the tracking list.

[0082] When the track status changes from "Lost" to "Tracking" or "Deleted", the ID duplicate detection mechanism is activated to check whether there are redundant tracks with the same identity and forcibly delete them.

[0083] In an embodiment of the present application, through a multi-level matching mechanism, the framework can associate the target from multiple dimensions, reducing the problems of missed detection and mismatching caused by target occlusion, rapid movement or sensor noise. By limiting the initialization conditions of the new trajectory (only allowing high-confidence detection boxes to initialize new trajectories), unnecessary trajectory creation and update operations are reduced. At the same time, the motion compensation of the Kalman filter reduces the need for global search. Compared with traditional methods (such as DeepSort), the computational complexity is reduced and the tracking accuracy (MOTA index) is improved. By introducing appearance feature similarity matching, the framework can better handle situations where the target appearance is similar but the position deviation is large. This mechanism can effectively reduce mismatching and improve tracking accuracy. In complex scenarios, the tracking accuracy is significantly improved, especially when dealing with high-speed moving targets (such as sudden lane change vehicles), the motion trajectory of the target can be tracked more accurately. By starting the ID duplication detection mechanism when the trajectory state is transferred, the framework can promptly discover and delete redundant trajectories with the same identity, thereby reducing the possibility of mismatching and repeated tracking, and further improving the accuracy and stability of tracking.

[0084] An optional embodiment provided in this application is described in detail below.

[0085] To address the above technical issues, an optional embodiment of the present application fuses point cloud projections onto 3-axis 2D image features, combined with a loss function, to improve the detection rate of objects directly in front of the lidar and small targets. The tracking algorithm uses the more advanced ByteTrack algorithm for target tracking. The tracking process has a triple matching process, which can effectively solve the problem of missed detection. Detection performance is improved through multimodal feature fusion and optimized training strategies, specifically including the following three core optimizations:

[0086] 1. Multi-view feature fusion mechanism

[0087] The 3D voxel features are synchronously projected onto three orthogonal plane perspectives: front view (FV), bird's eye view (BEV), and side view (SV). Through cross-view feature splicing and channel attention weighted fusion, a composite feature map with complete spatial perception capabilities is reconstructed. The specific steps are as follows:

[0088] 1) Voxelization: Divide the original point cloud data into a regular 3D voxel grid (e.g. 0.1m×0.1m×0.2m). For each non-empty voxel, extract the geometric features (e.g. point density, reflection intensity) and generate a 3D voxel feature tensor F. 3D ∈R H×W×D×C (H, W, D are the voxel space dimensions, and C is the number of channels).

[0089] 2) Multi-view orthogonal projection: Project the feature map F along the Z axis 3DPerform maximum pooling to generate BEV feature map; along the Y axis, the feature map F 3D Perform slice operation layer by layer, retain the X-axis and Z-axis, generate FV feature map through convolution coding; 3D Perform maximum pooling, retain the height Z-axis width Y-axis dimensions, and generate the SV feature map.

[0090] 3) Cross-view feature alignment and feature splicing: Through bilinear interpolation or deconvolution, the feature maps of the three views are unified to the same size (H*W), and then the feature maps are spliced ​​along the channel (C-axis) to synthesize a composite feature map.

[0091] 4) Channel attention weighting: Use the Squeeze-and-Expand module to perform channel weighting. The specific operation is to perform global average pooling on the feature map to obtain information describing the global importance of each channel, activate it through ReLu and Sigmoid, generate the channel attention weight W, and then multiply W by the feature map to enhance the response of the key channel and suppress redundant information to obtain the weighted feature.

[0092] This design effectively addresses the loss of 3D geometric information caused by single-view projection, particularly enhancing the ability to represent edge features in sparse point cloud objects directly in front of you. Specifically, when constructing the training target, the 2D IoU between the predicted box and the label from three different perspectives is calculated as the optimization target for the loss function.

[0093] 2. Direction-Sensitive Loss Function

[0094] By introducing a direction-adaptive weighting factor, the rotation angle θ of the target frame is combined with the azimuth angle of the position vector The sine of the resultant angle This mechanism dynamically adjusts the contribution of regression loss to targets with different orientations during training, significantly improving the model's robustness in azimuth perception of targets distributed at multiple angles along the LiDAR axis.

[0095] The improved CenterPoint loss function adds 3D IoU loss and 2D projection IoU loss on the basis of the original classification loss and regression loss.

[0096] 3. Robust Trajectory Association Algorithm

[0097] Integrate the improved ByteTrack multi-target tracking framework and design a three-stage matching process:

[0098] ① Kalman prediction matching based on motion compensation;

[0099] ② IoU correlation matching of high threshold detection boxes;

[0100] ③Low threshold detection box appearance feature similarity matching.

[0101] Through the multi-dimensional spatiotemporal correlation of trajectory segments, the trajectory breakage phenomenon caused by instantaneous missed detection can be effectively suppressed.

[0102] The embodiment of the present application adopts the high-performance ByteTrack tracking framework, which has significant advantages over the traditional DeepSort algorithm: by eliminating the feature extraction network and cosine similarity calculation links, the computational complexity is reduced while maintaining the same level of tracking accuracy. Figure 2 This is a flow chart of an improved vehicle tracking algorithm provided in an embodiment of the present application, such as Figure 2 As shown in the figure, the execution process of the improved vehicle tracking algorithm includes three core modules:

[0103] 1) Trajectory initialization strategy

[0104] In the initial state, when the confidence of the detection box exceeds the tracking threshold (track_thresh=0.6), a new track is created and the state parameters are set: track state state=Tracked, activation flag is_activated=True.

[0105] In subsequent frame processing, only unmatched detection boxes with a confidence level higher than a high threshold (high_thresh=0.7) are allowed to initialize new tracks. At this time, state=Tracked and is_activated=False.

[0106] In the trajectory prediction stage, the activated state trajectory (is_activated = True) and the lost state trajectory (state = Lost) are fused and the motion state estimation is completed through the Kalman filter.

[0107] 2) Multi-level matching mechanism

[0108] ① Primary matching: Perform IoU matching (threshold 0.5) on detection boxes with confidence > track_thresh to establish detection-track association.

[0109] ② Secondary matching: For the unmatched predicted tracks and the detection boxes with confidence ≤ track_thresh, IoU association is performed again.

[0110] ③ Final matching: associate the inactive track (is_activated = False) with the high confidence (> high_thresh) unmatched detection box.

[0111] 3) Track status management

[0112] Lifecycle control: When a track remains in the Lost state for more than max_time_lost (30 frames by default), the state changes to Removed and the track is deleted.

[0113] Redundant track elimination: When the state changes from Lost to Tracked / Removed, the ID duplication detection mechanism is activated to forcibly delete redundant tracks with the same identity.

[0114] In summary, the embodiments of the present application have the following advantages over the prior art:

[0115] (1) Multi-view feature fusion enhances 3D perception: By simultaneously projecting 3D voxel features onto three orthogonal planes: front view (FV), bird's-eye view (BEV), and side view (SV), combined with cross-view feature stitching and channel attention weighted fusion, the geometric feature representation capability of sparse point clouds (especially targets directly in front of the radar with low line counts) is significantly improved. This mechanism effectively solves the problem of missing 3D information in the traditional BEV perspective and improves the detection recall rate in long-range and low-density point cloud scenes.

[0116] (2) Direction-Sensitive Loss Function Optimizes Heading Estimation Accuracy: An innovative direction-adaptive weighting factor (based on the composite sine value of the target's rotation angle and position azimuth) is introduced to dynamically adjust the regression loss weights for targets with different orientations. This design significantly improves the model's robustness in azimuth perception of targets distributed at multiple angles along the LiDAR axis, especially reducing deviations in heading estimation for high-speed moving targets (such as vehicles that suddenly change lanes).

[0117] (3) Triple matching tracking algorithm improves robustness in complex scenarios: Using an improved ByteTrack framework, a three-stage process consisting of motion-compensated matching, high-threshold IoU correlation, and low-threshold appearance feature matching, combined with trajectory state lifecycle management, effectively suppresses trajectory breaks caused by instantaneous missed detections. Compared to traditional methods, this algorithm reduces computational complexity while improving tracking continuity (increasing the MOTA index), making it suitable for stable tracking in dense traffic and inclement weather.

[0118] According to another aspect of an embodiment of the present application, a point cloud object detection system integrating 2D features is provided. Figure 3 A schematic diagram of a point cloud target detection system integrating 2D features provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the point cloud object detection system integrating 2D features includes: a first generation module 302, a second generation module 304, a third generation module 306, a feature optimization module 308 and an object detection module 310. The point cloud object detection system integrating 2D features is described in detail below.

[0119] The first generating module 302 is used to perform voxel processing on the collected original point cloud data to generate a three-dimensional voxel feature tensor.

[0120] The second generation module 304 is connected to the first generation module 302 and is used to project the three-dimensional voxel feature tensor to three orthogonal plane perspectives: front view, bird's eye view, and side view, to generate a multi-view feature map.

[0121] The third generating module 306 is connected to the second generating module 304 and is used to perform cross-view feature alignment and splicing on the multi-view feature maps to generate a composite feature map.

[0122] The feature optimization module 308 is connected to the third generation module 306 and is used to optimize the composite feature map using a channel attention weighting mechanism to obtain an optimized composite feature map.

[0123] The target detection module 310 is connected to the feature optimization module 308 and is used to perform target detection based on the optimized composite feature map.

[0124] In an embodiment of the present application, the system performs voxel processing on the collected original point cloud data to generate a three-dimensional voxel feature tensor; projects the three-dimensional voxel feature tensor to three orthogonal plane perspectives of front view, bird's eye view and side view respectively to generate a multi-perspective feature map; performs cross-perspective feature alignment and splicing on the multi-perspective feature map to generate a composite feature map; uses a channel attention weighted mechanism to optimize the composite feature map to obtain an optimized composite feature map; performs target detection based on the optimized composite feature map, that is, the present application realizes target detection through multi-perspective feature fusion, thereby solving the technical problem that the traditional method lacks three-dimensional spatial information under the BEV perspective, resulting in a reduced target detection recall rate, and achieves the technical effect of improving the detection recall rate of targets in sparse point cloud scenarios.

[0125] It should be noted here that the above-mentioned first generation module 302, second generation module 304, third generation module 306, feature optimization module 308 and target detection module 310 correspond to steps S102 to S110 in the method embodiment. The examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment.

[0126] Optionally, the above-mentioned first generation module 302 includes: a first generation unit, used to divide the original point cloud data into a regular three-dimensional voxel grid, and extract the geometric features of non-empty voxels to generate a three-dimensional voxel feature tensor; wherein the geometric features include at least one of the following: point density, reflection intensity.

[0127] Optionally, the multi-view feature map includes a front-view feature map, a bird's-eye view feature map and a side-view feature map. The above-mentioned second generation module 304 includes: a second generation unit, used to perform layer-by-layer slicing operations on the three-dimensional voxel feature tensor along the Y-axis to generate a front-view feature map; a third generation unit, used to perform maximum pooling on the three-dimensional voxel feature tensor along the Z-axis to generate a bird's-eye view feature map; and a fourth generation unit, used to perform maximum pooling on the three-dimensional voxel feature tensor along the X-axis to generate a side-view feature map.

[0128] Optionally, the third generation module 306 includes: a fifth generation unit, configured to unify the multi-view feature maps to the same size using bilinear interpolation or deconvolution, and perform feature map splicing along the channel axis to synthesize a composite feature map.

[0129] Optionally, the feature optimization module 308 includes: a sixth generation unit, used to perform channel attention weighting on the composite feature map to generate channel attention weights; and a calculation unit, used to multiply the channel attention weights with the composite feature map to obtain an optimized composite feature map.

[0130] Optionally, the above system also includes: a loss function optimization module, which is used to optimize the detection target through a direction-sensitive loss function; the direction-sensitive loss function includes the following components: classification loss, regression loss, 3D IoU alignment loss and 2D projection IoU loss; wherein, the direction-sensitive loss function dynamically adjusts the regression loss weights of targets with different orientations through the composite sine value of the rotation angle of the target box and the azimuth angle of the position vector.

[0131] Optionally, the above system also includes: a target tracking module, which is used to adopt an improved multi-target tracking framework to track targets through motion-compensated Kalman prediction matching, IoU association matching of high-threshold detection frames, and appearance feature similarity matching of low-threshold detection frames.

[0132] Optionally, the improved multi-target tracking framework adopts the following trajectory state management mechanism: when the confidence of the detection frame exceeds the tracking threshold, a new trajectory is created and the state parameters are set. In subsequent frame processing, only unmatched detection frames with confidence higher than a high threshold are allowed to initialize new trajectories, and the active state trajectory and the lost state trajectory are fused through a filter to complete the motion state estimation; a multi-level matching mechanism is used to associate detection frames of different confidence levels respectively; when a trajectory is in a lost state for more than a set maximum number of frames, trajectory deletion is executed, and the ID duplication detection mechanism is started when the trajectory state is transferred to forcibly delete redundant trajectories with the same identity.

[0133] According to another aspect of an embodiment of the present application, an electronic device is provided, including: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the method of the embodiment of the present application.

[0134] An embodiment of the present application further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0135] An embodiment of the present application further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0136] refer to Figure 4 , a structural block diagram of an electronic device that can be used as a server or client of an embodiment of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0137] like Figure 4 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device can also be stored in the RAM 403. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0138] Multiple components within the electronic device are connected to the I / O interface 405, including an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device capable of inputting information into the electronic device. The input unit 406 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 407 can be any type of device capable of presenting information and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include, but is not limited to, a magnetic disk or an optical disk. The communication unit 409 allows the electronic device to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, and a wireless communication transceiver, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0139] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing units, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present application may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0140] The computer programs for implementing the methods of the embodiments of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of embodiments of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared systems, devices or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] It should be noted that the term "including" and its variations used in the embodiments of the present application are open inclusions, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present application are illustrative and not restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0143] The various steps described in the method implementation schemes provided in the embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method implementation schemes may include additional steps and / or omit the steps shown. The scope of protection of the present application is not limited in this respect.

[0144] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments are referenced to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts are referred to the partial description of the method embodiment.

[0145] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A point cloud target detection method integrating 2D features, characterized in that: include: Perform voxel processing on the collected original point cloud data to generate a three-dimensional voxel feature tensor; Projecting the three-dimensional voxel feature tensor onto three orthogonal plane perspectives, namely, front view, bird's eye view, and side view, to generate a multi-view feature map; Performing cross-view feature alignment and splicing on the multi-view feature maps to generate a composite feature map; Optimizing the composite feature map using a channel attention weighting mechanism to obtain an optimized composite feature map; Target detection is performed based on the optimized composite feature map.

2. The method according to claim 1, characterized in that Perform voxel processing on the collected raw point cloud data to generate a 3D voxel feature tensor, including: The original point cloud data is divided into a regular three-dimensional voxel grid, and geometric features of non-empty voxels are extracted to generate the three-dimensional voxel feature tensor; wherein the geometric features include at least one of the following: point density and reflection intensity.

3. The method according to claim 1, characterized in that The multi-view feature map includes a front view feature map, a bird's-eye view feature map, and a side view feature map. The three-dimensional voxel feature tensor is projected onto three orthogonal plane viewpoints of the front view, the bird's-eye view, and the side view to generate a multi-view feature map, including: Slicing the three-dimensional voxel feature tensor layer by layer along the Y axis to generate the front view feature map; Performing maximum pooling on the three-dimensional voxel feature tensor along the Z axis to generate the bird's-eye view feature map; Perform maximum pooling on the three-dimensional voxel feature tensor along the X-axis to generate the side view feature map.

4. The method according to claim 1, wherein Performing cross-view feature alignment and splicing on the multi-view feature maps to generate a composite feature map, including: The multi-view feature maps are unified to the same size by using bilinear interpolation or deconvolution, and the feature maps are spliced ​​along the channel axis to synthesize the composite feature map.

5. The method according to claim 1, wherein The composite feature map is optimized using a channel attention weighting mechanism to obtain an optimized composite feature map, including: Performing channel attention weighting on the composite feature map to generate channel attention weights; Multiplying the channel attention weight by the composite feature map to obtain the optimized composite feature map.

6. The method according to claim 1, characterized in that The method further comprises: The detection target is optimized through a direction-sensitive loss function; the direction-sensitive loss function includes the following components: classification loss, regression loss, 3D IoU alignment loss and 2D projection IoU loss; wherein, the direction-sensitive loss function dynamically adjusts the regression loss weights of targets with different orientations through the composite sine value of the rotation angle of the target box and the azimuth angle of the position vector.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: An improved multi-target tracking framework is adopted to track targets through motion-compensated Kalman prediction matching, IoU correlation matching of high-threshold detection boxes, and appearance feature similarity matching of low-threshold detection boxes.

8. The method according to claim 7, characterized in that The improved multi-target tracking framework adopts the following trajectory state management mechanism: When the confidence of the detection box exceeds the tracking threshold, a new trajectory is created and the state parameters are set. In subsequent frame processing, only unmatched detection boxes with confidence higher than the high threshold are allowed to initialize new trajectories. The active state trajectory and the lost state trajectory are fused through the filter to complete the motion state estimation. Use a multi-level matching mechanism to associate detection boxes of different confidence levels; When a track is in the lost state for more than the set maximum number of frames, track deletion is performed, and the ID duplication detection mechanism is activated when the track state changes, forcibly deleting redundant tracks with the same identity.

9. A point cloud target detection system integrating 2D features, characterized in that: include: The first generation module is used to voxelize the collected original point cloud data to generate a three-dimensional voxel feature tensor; A second generation module is used to project the three-dimensional voxel feature tensor onto three orthogonal plane perspectives, namely, front view, bird's eye view, and side view, to generate a multi-view feature map; A third generation module is used to perform cross-view feature alignment and splicing on the multi-view feature maps to generate a composite feature map; A feature optimization module, configured to optimize the composite feature map using a channel attention weighting mechanism to obtain an optimized composite feature map; The target detection module is used to perform target detection based on the optimized composite feature map.

10. An electronic device comprising: A processor and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

Images