An Extended Target Association Method for Vehicle-Mounted Radar in Urban Road Scenarios

By introducing prior track information on the basis of PointNet++ and DBSCAN, and using deep learning technology to enhance the difference in target features, the problem of extended target association for millimeter-wave radar in complex urban road environments is solved, and the association accuracy is improved.

CN117494039BActive Publication Date: 2026-11-10BEIJING INST OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311443758.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-02
Publication Date
2026-11-10
Estimated Expiration
2043-11-02

AI Technical Summary

Technical Problem

In complex urban road environments, millimeter-wave radar extended target association methods face problems such as decreased association accuracy due to sparse scattering points and missing target information caused by occlusion. Existing methods have limited association performance under occlusion conditions.

Method used

Based on the PointNet++ and DBSCAN framework, we introduce track prior fusion features to enhance the feature differences between targets through track prior information and use deep learning technology to compensate for the lack of target information caused by occlusion, thereby improving the association accuracy.

Benefits of technology

It improves the accuracy of extended target association in complex urban road environments, overcomes the impact of occlusion, and enhances the association accuracy between scattering points and targets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117494039B_ABST
    Figure CN117494039B_ABST
Patent Text Reader

Abstract

This invention proposes an extended target association method for vehicle-mounted radar in urban road scenarios. It utilizes the target number and location information in the track prior to calculate the target and scattering point matching relationship between adjacent frames and complete the scattering point feature fusion of adjacent frames, thus solving the problem of target association performance degradation caused by occlusion in complex urban road scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of automotive radar technology, specifically relating to a method for extended target association of vehicle-mounted radar in urban road scenarios. Background Technology

[0002] Currently, countries worldwide are actively promoting the development of vehicle-mounted high-resolution millimeter-wave radar. With the increasing resolution of millimeter-wave radar, targets occupy multiple resolution cells, presenting as spatially expanded multiple scattering points. To meet the demands of high-precision environmental perception, it is necessary to determine the correlation between scattering points and targets to achieve the observation and generation of expanded targets. In complex urban road environments, extended target association methods for vehicle-mounted millimeter-wave radar still face the following challenges. On the one hand, compared to lidar, millimeter-wave radar has sparser scattering points. When large traffic participants turn, the varying radar speeds at different locations reduce association accuracy, leading to target fragmentation. On the other hand, frequent occlusion between road obstacles and traffic participants causes partial loss of target information, significantly reducing the performance of association algorithms and resulting in frequent instances of single target fragmentation or multiple target merging. Therefore, an extended target association algorithm that can adapt to complex urban road environments is needed.

[0003] Based on different methods of processing scattering points, extended target association schemes can be categorized as follows: point cloud voxelization-based association schemes, convolutional neural network-based association schemes, and clustering-based association schemes. Point cloud voxelization divides the entire scene into cells, each cell also called a voxel, which can contain zero or more scattering points. The algorithm then performs target association on a voxel-by-voxel basis. This approach is suitable for dense point clouds but has high memory and computational requirements. Furthermore, the conversion between points and voxels introduces quantization errors, making it unsuitable for sparse point clouds. Convolutional neural network-based schemes directly process scattering points to generate features for the association network. The physical coordinates of the scattering points are also included as part of the features in the association process. Clustering-based schemes use clustering algorithms to estimate which scattering points belong to the same target. Compared to the previous two types, this method has limited performance, but due to its low computational cost, it is often used for real-time data processing.

[0004] The aforementioned association methods perform poorly when adjacent scattering points of the same target are far apart and when multiple targets are densely distributed within a traffic area. A non-end-to-end association scheme for millimeter-wave radar scattering point observations aims to overcome this problem. This scheme compensates for the original scattering point observations by calculating the center-shifted vector (CSV) from each scattering point to its respective target center. However, this scheme does not consider occlusion; its association performance is limited when road obstacles or traffic participants partially obscure the target. Therefore, a new association algorithm capable of addressing these issues is needed. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a method for extending target association in vehicle-mounted radar using track prior fusion features. This invention leverages the temporal continuity of actual driving scenarios, introducing track prior information containing the number and location of targets at both historical and current times, based on the PointNet++ and DBSCAN framework. The method utilizes track prior fusion to enhance the feature differences between different targets by fusing target features from historical frames.

[0006] An extended target association method for urban road scenarios includes the following steps:

[0007] Step 1, Feature Generation: This includes generating scattering point features in frame t and target features in frame t-1.

[0008] Scattering point feature generation: Input the radar scattering point set Y of frame t t The PointNet++ network divides the sampling range for each scattering point based on the sampling radius and number of sampling points set by the Set Abstraction (SA) layer. The scattering points within the sampling range are then passed through the Feature Propagation (FP) layer to obtain high-dimensional semantic features γ. t , Where Q represents the semantic feature dimension. Target feature generation: Assuming that in frame t-1, the target number to which each scattering point belongs is known, the average of the physical information and features of the scattering points with the same target number is used to approximate the physical information X of the target center. t-1 and target feature map Ω t-1 .

[0009] Step 2, Inter-frame matching. Traditional algorithms only use single-frame scattering point information, ignoring the correlation information between adjacent frames. To further improve the correlation accuracy, this algorithm establishes the target center X... t-1 With scattering point Y t The algorithm mines the correlation information between adjacent frames by establishing a mapping relationship. Since directly establishing this relationship is difficult, the algorithm introduces a track prior as a bridge between the two. The track prior for frame t is defined as... in This represents the position and velocity of the s-th track, s∈[1, S]. t ], D z For the prior dimension of the trajectory, S t Let Z be the number of tracks in frame t. At this point, the track prior Z for frame t-1... t-1 The target center X of frame t-1 t-1 With the same physical information, the trajectory prior Z of frame t t and the scattering point Y of frame tt Similarly. Furthermore, Z t-1 and Z t The correspondence is directly obtained from the track number s. Therefore, this module will X t-1 and Y t The matching problem is transformed into X t-1 and Z t-1 Z t and Y t The matching problem. X t-1 and Z t-1 The matching problem is solved using the nearest neighbor algorithm. t and Z t The matching problem is solved using the threshold method.

[0010] Step 3, Feature Fusion. Based on the above matching relationship Ψ ab That is, the scattering point features of frame t can be fused. t and the target feature Ω of frame t-1 t-1 The final input of the interconnected network is obtained. in, This represents the fused feature of the b-th scattering point.

[0011] Step 4, Network Connection. The feature map obtained in the t-frame feature fusion module. The scattering point classification matrix P is obtained by processing the data through two multi-layer perceptrons (MLPs), namely MLP1 and MLP2. SM and center offset vector P CSV Both the MLP1 and MLP2 modules are implemented using conv layers with 1×1 kernels. The activation function chosen is ReLU (Rectified Linear Unit), which is used in conjunction with batch normalization and dropout to alleviate model overfitting.

[0012] Step 5, setting the loss function. The network's loss function consists of the classification loss function L. SEM The direct loss function L of the center offset vector SHIFT We obtain the result by weighted calculation, i.e., L = L SEM +αL SHIFT , where α is the weighting coefficient. Assume the true class value of the m-th scattering point is l. m The category to which the scattering points predicted by the network belong is The classification error L of the scattering point category SEM It can be calculated by l m and The cross-entropy is obtained, Assume Δx gtΔx is the true center offset vector of each scattering point. pred The prediction error L of the center offset vector, which is calculated by the network to compensate for its position, is... SHIFT It is the characterization vector Δx gt and Δx pred Angular loss L between CS And characterizes the length loss L between two vectors NIP The combination of L SHIFT =L CS +L NIP .

[0013] This completes all the steps.

[0014] The beneficial effects of this invention are as follows:

[0015] This invention, based on the PointNet++ and DBSCAN framework, introduces trajectory priors to obtain information about the target before it is occluded. Based on this information, deep learning and other techniques are used to compensate for the loss of target information caused by occlusion, thereby improving the accuracy of the association between scattering points and the target. Attached Figure Description

[0016] Figure 1 A diagram of the extended target association architecture based on trajectory priors;

[0017] Figure 2 A schematic diagram for generating scattering point features;

[0018] Figure 3 This is a schematic diagram of inter-frame matching;

[0019] Figure 4 This is a diagram of a network of connections.

[0020] Figure 5 This is a visualization of the correlation results over continuous time. Detailed Implementation

[0021] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0022] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions;

[0023] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Appendix Figure 1 The architecture of this invention is illustrated. The result obtained by the radar sensor in frame t is Y. t Y t Represented as M t The set of scattering points, i.e. yt,m This represents the physical information of the m-th scattering point, including its position and velocity information, D. y Represents the dimension of physical information. Y t The high-dimensional semantic feature γ of the scattering point is obtained through the feature generation module. t The target feature Ω from the previous moment t Perform feature fusion to obtain fused features After branching through a multi-task network, a classification matrix P is obtained to characterize the semantic category of each scattering point. SM and the center offset vector P used to compensate for the original scattering point CSV Compensated observations The target number of each scattering point is obtained through DBSCAN, and the category information obtained by the classification matrix can be combined to obtain the association result X. t This completes the target association. In the above process, feature fusion depends on the matching relationship Ψ. ab The target center X is calculated by the inter-frame matching module. t-1 With scattering point Y t The mapping relationship between them. The target feature Ω from the previous time step. t-1 Based on γ t-1 It is calculated from the target number.

[0025] Step 1, Feature Generation: This includes generating scattering point features in frame t and target features in frame t-1.

[0026] Scattering point feature generation: as shown in the appendix Figure 2 As shown, the set of radar scattering points Y in input frame t is... t The PointNet++ network divides the sampling range for each scattering point based on the sampling radius and number of sampling points set by the Set Abstraction (SA) layer. The scattering points within the sampling range are then passed through the Feature Propagation (FP) layer to obtain high-dimensional semantic features γ. t , Here, Q represents the semantic feature dimension. This invention incorporates a gating multi-layer perceptron (gMLP) module into each SA and FP layer, using gating units to control the throughput of scattering points within the sampling radius. Therefore, the feature generation part is more robust in terms of parameter settings such as the sampling radius, and can adaptively retrieve information from high-value scattering points.

[0027] Target feature generation: Assuming that the target number of each scattering point is known in frame t-1, the physical information and features of the scattering points with the same target number are averaged to approximate the physical information X of the target center. t-1and target feature map Ω t-1 The formula is as follows:

[0028]

[0029]

[0030] in ID t-1 (i) represents all scattering points with target number i in frame t-1.

[0031] Step 2, Inter-frame matching. Traditional target association only uses single-frame scattering point information, ignoring the association information between adjacent frames. To further improve the association effect, this algorithm establishes the target center X... t-1 With scattering point Y t The algorithm mines the correlation information between adjacent frames by establishing a mapping relationship. Since directly establishing this relationship is difficult, the algorithm introduces track priors as a bridge between the two. (See attached...) Figure 3 As shown, the prior track of frame t is defined as follows: in This represents the position and velocity of the s-th track, s∈[1, S]. t ], D z For the prior dimension of the trajectory, S t Let Z be the number of tracks in frame t. At this point, the track prior Z for frame t-1... t-1 The target center X of frame t-1 t-1 With the same physical information, the trajectory prior Z of frame t t and the scattering point Y of frame t t Similarly. Furthermore, Z t-1 and Z t The correspondence is directly obtained from the track number s. Therefore, this module will X t-1 and Y t The matching problem is transformed into X t-1 and Z t-1 Z t and Y t The matching problem.

[0032] X t-1 and Z t-1 The matching problem is solved using the nearest neighbor algorithm, where Θ is defined. ai Let x represent the a-th target center. t-1,a and the i-th track z t-1,i The matching relationship, Θ ai =1 indicates a successful match, otherwise Θ ai =0. Y t and Z t The matching problem is solved using the threshold method. The solution steps are as follows:

[0033] Step 2-1: Calculate the y-th scattering point b. t,b With the i-th track z t,i cosine similarity ω i,b The formula is as follows:

[0034]

[0035] Step 2-2: Find the prior track number Γ corresponding to the maximum cosine similarity of the b-th scattering point. b :

[0036]

[0037] Steps 2-3: Establish the matching relationship Φ between the i-th track and the b-th scattering point based on the following decision criteria. ib :

[0038]

[0039] Where, Φ ib =1 indicates a successful match, otherwise Φ ib =0.

[0040] Steps 2-4, Target Center X t-1 With scattering point Y t The mapping relationship between Ψ ab By Θ ai and Φ ib The calculation yielded:

[0041]

[0042] Among them, Ψ ab =1 indicates a successful match, otherwise Ψ ab =0.

[0043] Step 3, Feature Fusion. Based on the above matching relationship Ψ ab That is, the scattering point features of frame t can be fused. t and the target feature Ω of frame t-1 t-1 The final input to the network association module is obtained. Among them, the fused features of the b-th scattering point Characteristic γ of the b-th scattering point t,b and the target feature Ω that was successfully matched t-1,a The calculation yielded:

[0044]

[0045] Step 4, Network Connection. The feature map obtained in the t-frame feature fusion module. The scattering point classification matrix P is obtained by processing the data through two multi-layer perceptrons (MLPs), namely MLP 1 and MLP 2. SM and center offset vector P CSV As attached Figure 4 As shown. Both MLP1 and MLP2 modules are implemented using conv layers with 1×1 kernels. The activation function chosen is ReLU (Rectified Linear Unit), combined with batch normalization and dropout to alleviate overfitting. The head structures of different branches of the network are shown below:

[0046]

[0047] In formula (8), the constant 4 in the first row represents the probability that the scattering point belongs to each category; D is the physical information dimension of the original scattering point, including the x-coordinate offset, y-coordinate offset, and velocity offset of the scattering point, etc. SM For each scattering point, P CSV The scattering points are then used for compensation of the original scattering points, and the compensated scattering points are further clustered using the DBSCAN algorithm. The clustering results of each scattering point, combined with the classification results, constitute the target association result for that frame.

[0048] Step 5, setting the loss function. The network's loss function consists of the classification loss function L. SEM The direct loss function L of the center offset vector SHIFT We obtain the result by weighted calculation, i.e., L = L SEM +αL SHIFT , where α is the weighting coefficient. Assume the true class value of the m-th scattering point is l. m The category to which the scattering points predicted by the network belong is The classification error L of the scattering point category SEM It can be calculated by l m and The cross-entropy is obtained, Assume Δx gt Let Δx be the true center offset vector for each observation point. pred The prediction error L of the center offset vector, which is calculated by the network to compensate for its position, is... SHIFT It is the characterization vector Δx gt and Δx pred Angular loss L between CS And characterizes the length loss L between two vectors NIP The combination of L SHIFT =L CS +L NIPAmong them, L CS and L NIP The calculations are as follows:

[0049]

[0050]

[0051] Where cos(Δx) pred,i Δx gt,i ) represents Δx pred and Δx gt The cosine of the angle between the vectors of the i-th (1≤i≤f)-th feature dimension, ε is generally taken as 10. -5 This ensures that the denominator is not zero.

[0052] The following provides an implementation example with specific parameters.

[0053] In this example, we generated the dataset using the MATLAB software Driving Scenario Designer (DSD). Installing a millimeter-wave radar on a detection vehicle allows us to directly obtain continuous-time scattering point cloud data. This data includes the position, velocity, azimuth angle, category number, and target number of each scattering point in both Cartesian and polar coordinate systems.

[0054] The parameters of the vehicle-mounted radar were set according to the Radarscenes dataset. A 77GHz near-field millimeter-wave radar was used, with range, velocity, and azimuth resolutions of 15cm, 0.36m / s, and 2°, respectively. The azimuth F0V was ±60°, and the frame period was 70ms. The scene was set as a 100m×100m traffic intersection, simulating the movement of various targets. There were four target types: cars, large vehicles, two-wheeled vehicles, and pedestrians, with a ratio of approximately 0.46:0.17:0.09:0.28. The dataset was divided into training and test sets in a 4:1 ratio, and the specific sample distribution is shown in Table 1.

[0055] The training batch size is 5, and the initial learning rate is 10. -2 The optimizer is Adam. A cosine annealing iterator is used, and the learning rate is restarted every 25 training epochs. This is to address the issue of the number of scattering points N in each frame. t For different problems, we count the number of scattering points in each frame of each scene to obtain the max(N) t ) and mean(N t To determine the sample size, [reference was provided].

[19] The processing method involves fixing the number of scattering points to 200 during training and testing, slightly larger than mean(N). t This ensures that most of the sample information is retained without increasing computational costs.

[0056] Table 1. Sample distribution of the simulation dataset

[0057] Sample type small car truck bike pedestrian training set 10780 4919 2520 7678 test set 2700 1230 630 1919 total 13480 6149 3150 9597

[0058] Based on the number of scattering points per frame and the scene size, we require the PointNet++ network to have two SA layers and two corresponding FP layers. The first SA level has 64 sampling points and a sampling radius of 8m. These parameters are designed so that all sampling periods cover the entire field of view with appropriate overlap. SA layer parameters SA(N sample R sample [c1, ...c] d ]) represents the sampling radius R at each scattering point. sample N is selected from within. sample Each sampling point is processed by a d-layer convolutional layer with a 1×1 kernel, and the number of output channels is [c1, ... c2]. d The convolutional layer completes the sampling; the FP layer parameters are FP(c1, ... c). d ) indicates that the d-layer convolutional kernel is 1×1 and the number of output channels is [c1, ... c2]. d The convolutional layers are called [[]. In addition, BatchNorm and ReLU are used between two consecutive convolutional layers, and the gMLP block does not change the dimension of the output feature vector, so it can be directly appended to the end of each SA or FP.

[0059] Table 2 Network Parameters

[0060] Network layer Parameter value First SA layer SA(64, 8, (8, 32, 64)) Second SA layer SA(16,16,(64,128,256)) First FP layer FP(64, 32) Second FP layer FP(32, 32, 16)

[0061] Appendix Figure 5 The visualization results of the proposed algorithm over consecutive time intervals are shown. The first row shows the ground truth of the scene. The second row shows the association results of the algorithm. (a) is the association visualization result of the algorithm in frame 8; (b) is the association visualization result of the algorithm in frame 10; (c) is the association visualization result of the algorithm in frame 12. In frame 12, the radar's right front view is obstructed by a truck, thus greatly reducing the number of scattering points in the right front. However, due to the constraint of track prior, the algorithm can still correctly associate two spatially adjacent real targets. It can be seen that by introducing track prior, the algorithm can effectively overcome the above-mentioned obstruction problem and improve the association accuracy.

[0062] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for extended target association using vehicle-mounted radar in urban road scenarios, characterized in that, The association of extended targets observed by vehicle-mounted radar includes the following steps: Step 1, Feature Generation: Includes Frame scattering point feature generation and Frame target feature generation; Step 2, Inter-frame matching: Introducing track priors as... The target center of the frame and The matching bridge between scattering points of the frame is defined as the first The frame's trajectory prior, which includes the position and velocity information of each trajectory, will... The target center of the frame and The problem of matching scattering points of frames is transformed into The problem of matching the target center of a frame with the track prior, and the relationship between track prior and... The problem of matching scattering points in a frame, where The problem of matching the target center of a frame with the prior track information is solved using the nearest neighbor algorithm. The prior track information and... The problem of matching scattering points in a frame is solved using a threshold method, and the results are obtained based on the two matching outcomes described above. Frame target center and Mapping relationship between frame scattering points; Step 3, Feature Fusion: Based on the mapping relationship obtained in Step 2, the feature fusion is performed... Frame scattering point features and The target features of the frames are fused to obtain the input features of the associated network; Step 4, Association Network: Input the fused features obtained in Step 3 into the association network for processing to obtain the association results of the extended target; Step 5, Loss function setting: Set a weighted combination of the classification loss function and the center offset vector loss function as the loss function for training the association network.

2. The method for extended target association of vehicle-mounted radar in urban road scenarios as described in claim 1, characterized in that, The scattering point feature generation method described in step 1 is as follows: Input The PointNet++ network uses a set of radar scattering points from a frame to divide a sampling range for each scattering point based on the sampling radius and number of sampling points set by the set abstraction layer. Scattering points within the sampling range are then passed through a feature propagation layer to obtain high-dimensional semantic features. ; Q Represents the semantic feature dimension. Indicates the first The number of scattering points in a frame; a gated multilayer perceptron module is added to each set abstraction layer and feature propagation layer, and the pass rate of scattering points within the sampling radius is controlled by the gate unit.

3. The method for extended target association of vehicle-mounted radar in urban road scenarios as described in claim 1, characterized in that, The target feature generation method described in step 1 is as follows: In a frame, the target number of each scattering point is known. The physical information and characteristics of the scattering points with the same target number are averaged to approximate the physical information and target feature map of the target center.

4. The method for extended target association of vehicle-mounted radar in urban road scenarios as described in claim 1, characterized in that, In step 2, the solution steps of the threshold method include: Step 2-1, calculate the first... The scattering point and the th Cosine similarity of the flight paths; Step 2-2, find the first The prior track number corresponding to the maximum cosine similarity of each scattering point; Steps 2-3: Establish the first [judgment criteria] track and the first Matching relationships among scattering points; Steps 2-4: Calculate the mapping relationship between the target center and the scattering point based on the matching relationship between the target center and the track, and the matching relationship between the track and the scattering point.

5. The method for extended target association of vehicle-mounted radar in urban road scenarios as described in claim 1, characterized in that, In step 3, the first The fused features of the scattering point are derived from the scattering point. The features of each scattering point and the features of the successfully matched target are calculated, where This is the scattering point number.

6. The method for extended target association of vehicle-mounted radar in urban road scenarios as described in claim 1, characterized in that, In step 4, the fused feature map is processed by two multilayer perceptrons, MLP1 and MLP2, to obtain the scattering point classification matrix and the center offset vector. The original scattering points are compensated based on the center offset vector and then clustered by DBSCAN to obtain the target number of each scattering point. The category information obtained by combining the classification matrix is ​​used to obtain the association result of the extended target.

7. The method for extended target association of vehicle-mounted radar in urban road scenarios as described in claim 1, characterized in that, In step 5, the prediction error of the center offset vector It is a characterization vector and angular loss between and characterize the length loss between two vectors The combination ;in, This represents the true center offset vector for each scattering point. The center offset vector calculated for the network to compensate for its position.