Extended target tracking method based on Swinin-transformer

Abstract

The invention discloses an extended target tracking method based on a swi-transformer, and the method comprises the steps: processing a radar point cloud image with noise through a deep learning network of the swi-transformer and a Kalman filter, and employing a visual converter; wherein the swi n-transformer is used for deducing shape parameters of a target and at least comprises a standard multi-head self-attention module based on a shift window, a standard multi-head self-attention module based on a window and a structure, the interior of the structure is composed of two layers of multi-layer sensors, and Gaussian error linear unit nonlinearity exists in the middle of the structure; and the Kalman filter is used for estimating kinematics parameters of the target. According to the scheme, the motion parameter and the shape parameter of the single extended target can be estimated from the noisy point cloud at the same time, so that target tracking is more accurate and efficient.

Description

Technical Field

[0001] The invention belongs to the technical field of navigation, signal processing and target tracking, and mainly relates to an extended target tracking method based on Swin-transformer. Background Art

[0002] With the development of high-resolution sensors such as automotive radar and lidar, extended target tracking (ETT) has become a hot topic in the field of autonomous driving. ETT usually requires estimating the motion parameters and shape parameters of the tracked target. Among them, there are several typical algorithms for the estimation of shape parameters, such as independent axis estimation (IAE), extended Kalman filter based on multiplicative error model (MEM-EKF), random matrix, random hypersurface model, convolutional neural network (CNN) based methods, etc. Although the effectiveness of these methods has been proven in high measurement rate scenarios, their effectiveness will be reduced in low measurement rate scenarios. Summary of the invention

[0003] The present invention is aimed at the problem that the existing ETT method has poor performance in low measurement rate scenarios, and provides an extended target tracking method based on Swin-transformer. Through the deep learning network and Kalman filter of swin-transformer, a visual converter is used to process noisy radar point cloud images; wherein, the swin-transformer: realizes the inference of target shape parameters, and at least includes a standard multi-head self-attention module based on a shift window, a standard multi-head self-attention module based on a window, and a structure composed of two layers of multi-layer perceptrons, with Gaussian error linear unit nonlinearity in the middle; the Kalman filter: is used to estimate the kinematic parameters of the target. The method of this case can simultaneously estimate the motion parameters and shape parameters of a single extended target from the noisy point cloud, making the target tracking more accurate and efficient.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is: an extended target tracking method based on Swin-transformer, through the deep learning network and Kalman filter of Swin-transformer, using visual converter to process noisy radar point cloud images; wherein,

[0005] The swin-transformer: realizes the inference of the target shape parameters, and includes at least a standard multi-head self-attention module based on a shifted window, a standard multi-head self-attention module based on a window, and a structure composed of two layers of multi-layer perceptrons, with a Gaussian error linear unit nonlinearity in the middle;

[0006] The Kalman filter is used to estimate the kinematic parameters of the target.

[0007] As an improvement of the present invention, a Swin-transformer-based extended target tracking method includes the following steps:

[0008] S1, image conversion: normalize and discretize the noisy point cloud and convert the point cloud into an image;

[0009] S2, image update: updating the image obtained by the transformation in step S1 according to the passage of time to form an internal intensity measurement image (IIMI);

[0010] S3, shape estimation: use swin-transformer to perform visual transformation on the image formed in step S2 to infer the shape parameters of the target;

[0011] S4, kinematic estimation: using a Kalman filter on the shape parameters estimated in step S3, and estimating kinematic parameters based on the original noise point cloud and the inferred shape parameters;

[0012] S5, target tracking: The shape parameters and kinematic parameters of the target are obtained through the shape estimation in step S3 and the kinematic estimation in step S4, and the tracking of the target is completed.

[0013] As an improvement of the present invention, in step S1, the method for normalizing and discretizing the noise point cloud is specifically as follows:

[0014]

[0015]

[0016] Among them, Z' k are all newly arrived measurements z centered around the estimated target center k The Cartesian coordinates of k is the new coordinate after the measured value is aligned with the target value, are all newly arrived measurements, Representative with The rotation matrix of The measurement matrix H is is the kinematic parameter of the object, expressed as in, and Represents the horizontal and vertical coordinates of the object, and represents the horizontal and vertical speed of the object, and T represents the transpose of the matrix.

[0017] As another improvement of the present invention, the image update in step S2 is specifically as follows:

[0018]

[0019] in, represents the IIMI calculated within the latest time k, and 0≤γ≤1 represents the forgetting factor;

[0020] Normalized IIMI k Specifically:

[0021] IIMI k =255×IIMI k ÷max(IIMI k ).

[0022] As another improvement of the present invention, in step S3, the internal intensity measurement image formed in step S2 is used as the input of the Swin transformer, and the image is divided into non-overlapping blocks through the guided partitioning module, and then the image features are projected to the selected dimension through the linear embedding module, waiting for Swin transformer processing; the Swin transformer also includes a patch merging module to reduce the image resolution, and uses a fully connected layer to generate estimated shape parameters.

[0023] As another improvement of the present invention, the shape parameters in step S3 include and represents the angle of the shape's orientation, and Represents the minor and major axes of the ellipse. It can be predicted as:

[0024]

[0025] in, represents the vertical velocity of the object at time k, represents the horizontal velocity of the object at time k.

[0026] As another improvement of the present invention, in step S4, a Kalman filter is used to predict the kinematic parameters of the target:

[0027]

[0028]

[0029] Where F is the transfer matrix, are the kinematic parameters estimated at time k-1, are the kinematic parameters estimated at time k by the Kalman filter, is the estimated covariance corresponding to time k-1, Q is the noise, and after receiving the noise point cloud at time k, the kinematic parameters are updated by updating the equations. The update equations are as follows:

[0030]

[0031]

[0032]

[0033]

[0034] P k =(IG k H)P k|k-1

[0035]

[0036] Among them, P k|k-1 is the prediction error covariance, H is the observation matrix, which maps the state space to the observation space, is the covariance matrix of the observation noise, G k is the Kalman gain, which is used to weigh the weight of prediction error and measurement error. are all newly arrived measurements z centered around the estimated target center k The mean value, m k is the measured value The number of is an item related to the covariance matrix, used to expand the target correlation, and Represents the minor and major semi-axes of the ellipse before updating.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) The method of the present invention combines the neural network architecture with the Kalman filter, fully utilizing the high performance of SwinTransformer in low measurement rate scenarios, and proposes an extended target tracking method called ST-ETT, which simultaneously estimates the kinematic and shape parameters of a single extended target from a noisy point cloud, greatly improving the accuracy and efficiency of target tracking. The Swin Transformer network enables this architecture to be used in more scenarios, with stronger robustness and the ability to cope with various complex scenarios. At the same time, its model is more streamlined and training is more efficient.

[0039] (2) The method of the present invention proposes an elliptical extended target estimation method based on Swin Transformer, which shows powerful performance. In challenging scenarios such as low measurement rates, through deep learning methods, it can explore the general rules of target shape parameters behind sparse measurements, greatly increasing the success rate of shape parameter recognition.

[0040] (3) Compared with end-to-end target tracking neural networks, this architecture integrates the advantages of the Kalman filter. In some cases where the neural network cannot fully derive the internal model mechanism, the Kalman filter and the target shape and kinematics modeling are used to analyze the internal model of the target state, which reduces the amount of calculation, improves tracking accuracy, and also increases the interpretability of the neural network.

[0041] (4) The ellipse shape estimation method can be used in combination with any other kinematic estimation method, which shows that the ellipse estimation method is highly portable and modular. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the steps of the method of the present invention;

[0043] Figure 2 It is a schematic diagram of the architecture of the swin transformer in the present invention;

[0044] Figure 3 It is a schematic diagram of the architecture of the STB module in the swin transformer architecture;

[0045] Figure 4(a)-4(d) Schematic diagram of the comparison of the performance of the ST-ETT method and other methods in the test case;

[0046] Among them, Fig. 4(a) shows the average tracking performance of the ST-ETT method over time;

[0047] Fig. 4(b) shows the error diagram of the average tracking error minus IAE of ST-ETT under different measurement rates and noises;

[0048] Figure 4(c) shows the tracking error of ST-ETT minus the error of MEM-EKF;

[0049] Figure 4(d) shows the tracking error of ST-ETT minus the error of the CNN-based ETT method. DETAILED DESCRIPTION

[0050] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0051] Example 1

[0052] A method for extended target tracking based on swin-transformer, which uses a visual transformer to process noisy radar point cloud images through a swin-transformer deep learning network and a Kalman filter; SwinTransformer, as a high-performance neural network, is widely used in target tracking. This network can use a conversion mechanism to directly calculate the correlation between different image blocks without going through a hidden layer, which enables it to use images with less information to correctly estimate the target. With this ability, Swin Transformer becomes a potential way to improve the performance of extended target tracking in low measurement rate scenarios and realize the inference of target shape parameters; and the Kalman filter is used to estimate the kinematic parameters of the target.

[0053] A method for extended target tracking based on swin-transformer, such as Figure 1 As shown, the following steps are included:

[0054] Step S1: normalize and discretize the measured value, i.e., the noise point cloud, and convert it into an image.

[0055] The results of normalizing and discretizing the noise point cloud are as follows:

[0056]

[0057]

[0058] Among them, Z' k are all newly arrived measurements z centered around the estimated target center k The Cartesian coordinates of k is the new coordinate after the measured value is aligned with the target value. In the subsequent steps, the discrete Z k , forming an IIMI image.

[0059] Step S2: The image converted in step S1 is updated over time to form an internal intensity measurement image (IIMI).

[0060] The formed IIMI image is updated and calculated as follows:

[0061]

[0062] in, It represents the latest calculated IIMI, 0≤γ≤1 represents the forgetting factor, k represents the latest time, and u is the abbreviation of update, which represents the latest updated picture.

[0063] This method can also be used to obtain the normalized IIMI k :

[0064] IIMI k =255×IIMI k ÷max(IIMI k )

[0065] Step S3: Use swin-transformer to perform visual transformation processing on the image formed in S2 to infer the shape parameters of the target.

[0066] Normalized and updated IIMI k As the input of the swin transformer architecture, the entire swin transformer architecture inputs an H×W×2 image, where H and W represent the updated IIMI k The height and width of the image are then directed to the patch partitioning module. This module partitions the image into non-overlapping patches, after which the linear embedding module projects the image features into the selected dimensions for subsequent processing by the swin transformer block.

[0067] Swin transformer block (STB), such as Figure 3 As shown, it includes a shift window-based standard multi-head self-attention (SW-MSA) module, a window-based standard multi-head self-attention (W-MSA) module, and an internal structure consisting of two layers of multi-layer perceptron (MLP), with a Gaussian error linear unit (GELU) nonlinearity in the middle. The window size is set to M=7. The query dimension of each head is d=32, and the expansion layer of each MLP is α=4. The layer normalization (LN) technique is applied before each MSA module and each MLP module, and the residual connection is applied after each module. The output dimension of the swin transformer block is set to C=96. The patch merging module is used to further reduce the resolution of the image and seamlessly merge adjacent 2×2 pixel regions. After continuous processing by the STB module, a patch merging is completed. The specific process is as follows Figure 2 shown.

[0068] After completing all STB modules and patch merging processes, a fully connected layer is used to generate estimated shape parameters and Finish and After calculation, according to

[0069]

[0070] Generate pairs Estimates.

[0071] and the generated together as the shape parameter for the updated estimate, represents the angle of the shape's orientation, and Represents the minor and major axes of the ellipse.

[0072] Step S4: Use a Kalman filter on the shape parameters estimated in step S3 to estimate kinematic parameters based on the original noise point cloud and the inferred shape parameters.

[0073] Assume that the estimated kinematic parameters at time k-1 have been obtained The corresponding estimated covariance P k-1 and the estimated shape parameters First, a Kalman filter is used to predict the kinematic parameters of the target:

[0074]

[0075]

[0076] Z' k are all newly arrived measurements z centered around the estimated target center k The Cartesian coordinates of k are the new coordinates after the measured values ​​are aligned with the target values. are all newly arrived measurements, Representative with The rotation matrix of The measurement matrix H is

[0077] For the angle of the shape position, it can be predicted as

[0078]

[0079] After receiving the noise point cloud at time k, the kinematic parameters are updated by the following update equations:

[0080]

[0081]

[0082]

[0083]

[0084] P k =(IG k H)P k|k-1

[0085]

[0086] Among them, P k|k-1 is the prediction error covariance, H is the observation matrix, which maps the state space to the observation space, is the covariance matrix of the observation noise, G k is the Kalman gain, which is used to weigh the weight of prediction error and measurement error. are all newly arrived measurements z centered around the estimated target center k The mean value, m k is the measured value The number of is an item related to the covariance matrix, used to expand the target correlation, and Represents the minor and major semi-axes of the ellipse before updating.

[0087] Step S5: The shape parameters and kinematic parameters of the target are obtained through the shape estimation in step S3 and the kinematic estimation in step S4, and the tracking of the target is completed.

[0088] Test Case

[0089] In the MATLAB simulation environment with an Intel i9 CPU and a 3060 GPU, the process of a target moving independently from the origin is simulated. The target is tracked by using the ST-ETT algorithm and CNN, IAE, and MEM-EKF algorithms respectively, and the tracking effects are compared.

[0090] The squared Gauss-Wasserstein distance (GWD) is widely used to calculate the errors of the location parameters and shape parameters. The parameters of GWD are set as follows: the forgetting factor is set to γ ​​= 0.95, the covariance of the measured value is set to R = I·r, r = 2, the measurement rate follows a Poisson point process with λ = 2, and the process noise covariance Q is set to

[0091]

[0092] The ST-ETT method proposed in the present invention is further compared with the existing CNN-based ETT method, IAE and MEM-EKF methods, as shown in Figure 4. Figure 4(a) shows the average tracking performance of the ST-ETT method over time, which is compared in the form of average square GWD; Figure 4(b) shows the average tracking error of ST-ETT under different measurement rates and noises minus the error of IAE; Figure 4(c) shows the tracking error of ST-ETT minus the error of MEM-EKF; Figure 4(d) shows the tracking error of ST-ETT minus the error of the CNN-based ETT method.

[0093] Figure 4(a) shows the comparison of the average square GWD of different methods over time. The curve in the image shows that the ST-ETT method has the smallest tracking error compared with the existing methods. At the same time, the square GWD values ​​of these methods under different noise conditions and different measurement rates are calculated and recorded in Table 1. 3000 Monte Carlo (MC) simulations were performed for each scenario, and the final results were averaged.

[0094] Table 1 Square GWD of different ETT methods

[0095] Measurement rate ST-ETT CNN IAE MEM-EKF λ=0.75 2.0909 2.4946 5.4344 3.1147 λ=1 1.997 2.4095 5.4911 3.0815 λ=1.5 1.848 2.1624 5.3380 3.0401 λ=2.5 1.7784 2.0224 4.9897 2.8305 λ=4 1.5588 1.7145 4.0394 2.05731 λ=6 1.3792 1.2571 3.4535 1.9962

[0096] Figure 4(b) , 4(c) and 4(d) represent the average tracking error of ST-ETT minus the error of AE and MEM-EKF based on CNN ETT method, respectively. It can be seen from the figure that lower values ​​(blue) indicate that ST-ETT method outperforms other methods, while higher values ​​(red) indicate the opposite. It is obvious that the proposed ST-ETT method outperforms the existing three measurement methods, especially when the measurement rate λ is less than 6.

[0097] In summary, the core of this method is to combine the deep learning network of swin-transformer and Kalman filter, and use visual transformer to process noisy radar point cloud images. In this method, swin-transformer is used to realize the inference of target shape parameters, and Kalman filter is used to estimate the kinematic parameters of the target. Compared with existing methods, the proposed ST-ETT method has significant competitiveness in low measurement rate scenarios.

[0098] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. An extended target tracking method based on Swin-transformer, characterized in that : Through the deep learning network and Kalman filter of swin-transformer, the visual converter is used to process the noisy radar point cloud image; The swin-transformer: realizes the inference of the target shape parameters, and includes at least a standard multi-head self-attention module based on a shifted window, a standard multi-head self-attention module based on a window, and a structure composed of two layers of multi-layer perceptrons, with a Gaussian error linear unit nonlinearity in the middle; The Kalman filter is used to estimate the kinematic parameters of the target.

2. The extended target tracking method based on Swin-transformer according to claim 1, characterized in that: The steps include: S1, image conversion: normalize and discretize the noisy point cloud and convert the point cloud into an image; S2, image update: updating the image obtained by the transformation in step S1 according to the passage of time to form an internal intensity measurement image (IIMI); S3, shape estimation: use swin-transformer to perform visual transformation on the image formed in step S2 to infer the shape parameters of the target; S4, kinematic estimation: using a Kalman filter on the shape parameters estimated in step S3, and estimating kinematic parameters based on the original noise point cloud and the inferred shape parameters; S5, target tracking: The shape parameters and kinematic parameters of the target are obtained through the shape estimation in step S3 and the kinematic estimation in step S4, and the tracking of the target is completed.

3. The extended target tracking method based on Swin-transformer as claimed in claim 2, characterized in that: In step S1, the method for normalizing and discretizing the noise point cloud is specifically as follows: Among them, Z' k are all newly arrived measurements z centered around the estimated target center k The Cartesian coordinates of k is the new coordinate after the measured value is aligned with the target value, are all newly arrived measurements, Is with The rotation matrix of The measurement matrix H is are the kinematic parameters of the object, specifically in, and Represents the horizontal and vertical coordinates of the object, and represents the horizontal and vertical speed of the object, and T represents the transpose of the matrix.

4. The extended target tracking method based on Swin-transformer as claimed in claim 2, characterized in that: The image update in step S2 is specifically as follows: in, represents the IIMI calculated within the latest time k, and 0≤γ≤1 represents the forgetting factor; Normalized IIMI k Specifically: SEARCH k =255×IIMI k ÷max(IIMI) k )。 5. The extended target tracking method based on Swin-transformer as claimed in claim 2, characterized in that: In step S3, the internal intensity measurement image formed in step S2 is used as the input of the Swin transformer. After passing through the guided partitioning module, the image is divided into non-overlapping blocks, and then the image features are projected to the selected dimension through the linear embedding module, waiting for Swin transformer processing; the Swin transformer also includes a patch merging module to reduce the image resolution, and uses a fully connected layer to generate estimated shape parameters.

6. The extended target tracking method based on Swin-transformer as claimed in claim 5, characterized in that: The shape parameters in step S3 include and The angle representing the orientation of the shape, and Represents the minor and major axes of the ellipse. The prediction is: in, represents the vertical velocity of the object at time k, represents the horizontal velocity of the object at time k.

7. The extended target tracking method based on Swin-transformer according to claim 6, characterized in that: In step S4, a Kalman filter is used to predict the kinematic parameters of the target: Where F is the transfer matrix, are the kinematic parameters estimated at time k-1, are the kinematic parameters estimated at time k by the Kalman filter, is the estimated covariance corresponding to time k-1, Q is the noise; after receiving the noise point cloud at time k, the kinematic parameters are updated by updating the equation, and the update equation is as follows: P k =(I-G k H)P k|k-1 Among them, P k|k-1 is the prediction error covariance, H is the observation matrix, is the covariance matrix of the observation noise, G k is the Kalman gain, are all newly arrived measurements z centered around the estimated target center k The mean value, m k is the measured value The number of is a term related to the covariance matrix, and Represents the minor and major semi-axes of the ellipse before updating.

Images