A segmentation-assisted radar multi-frame target detection method

Abstract

The application discloses a kind of segmentation assisted radar multi-frame target detection methods, applied to radar target detection and deep learning technical field, for the global feature extraction capability insufficient, adjacent target distinguishing ability insufficient, segmentation and detection task mutual relationship is not sufficient for the problem of mining;The application first obtains range-azimuth two-dimensional data according to simulated and measured radar original echo data, establishes data set, and trains multi-frame target detection network MFDet according to the established data set, then utilizes the trained target detection network and outputs multi-frame detection results in parallel;The application uses region-level detection head, effectively solves the problem that pure semantic segmentation task cannot distinguish adjacent targets;By exploring the mutual relationship between semantic segmentation and target detection, the confidence of the segmentation result is used to assist the fine boundary box, effectively reducing the false alarm rate of target detection, and improving the target detection performance in clutter interference and adjacent target scene.

Description

Technical Field

[0001] This invention belongs to the fields of radar target detection and deep learning, and specifically relates to a radar multi-frame target detection technology. Background Technology

[0002] Target detection is a crucial task for radar. By processing echo signals, targets can be identified and located, enabling target detection in complex environments. Compared to other sensors such as cameras and lidar, radar offers several advantages, including longer operating range, immunity to low visibility and low-light conditions, and the ability to estimate velocity. Constant False Alarm Rate (CFAR) detection is an existing statistical distribution-based method that adaptively adjusts the detection threshold based on surrounding clutter elements. However, it is only suitable for clutter scenarios with specific statistical distributions. In complex environments, the high resolution of radar makes clutter statistical distributions even more complex, making it difficult for statistical distribution-based detection methods to accurately fit these distributions.

[0003] As a data-driven task, deep learning can automatically extract deep features without much consideration for the statistical characteristics of the input data, and without the need for manual design and explicit feature extraction. Specifically, in detection tasks, the field of deep learning has numerous end-to-end detection networks that can effectively solve the problem of performance degradation caused by model distribution mismatch in traditional techniques. Therefore, deep learning-based radar target detection methods have broad research space and application potential.

[0004] For feature extraction modules across multiple frames, the paper "RODNet: A real-time radar object detection network cross-supervised by camera-radar fused object 3d localization," IEEE Journal of Selected Topics in Signal Processing, vol.15, no.4, pp.954-967, 2021, utilizes convolutional layers instead of DFT to fuse chirp information within a frame to solve for Doppler velocity and employs time-deformable convolution to extract inter-frame motion features. However, the limited receptive field of the convolutional kernel implies insufficient overall perception. For radar target detection in a widely distributed clutter background, the ability to extract global features is required to accurately fit the clutter distribution.

[0005] For multi-frame target detection modules, the paper "RadarFormer: Lightweight and accurate real-time radar object detection model, Scandinavian Conference on Image Analysis. Springer, pp. 341-358, 2023" compresses multi-frame data in the temporal dimension before reconstructing the multi-frame data. The paper "T-RODNet: Transformer for vehicular millimeter-wave radar object detection, IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1-12, 2022" reconstructs multi-frame range-azimuth maps using spatiotemporal features. However, these studies primarily use semantic segmentation to indicate detection points. On one hand, extended targets occupying multiple cells correspond to multiple detection points, thus requiring subsequent clustering operations. On the other hand, semantic segmentation struggles to distinguish between adjacent or partially occluded targets. In contrast, bounding box-based methods can generate a box for each target instance to represent a holistic target region, rather than scattered detection points. Therefore, bounding box-based methods do not require clustering operations and can more robustly distinguish adjacent targets.

[0006] The paper "Region based single-stage interference mitigation and target detection, 2020 IEEE Radar Conference. IEEE, pp. 1–5, 2020" designs a dual-task network based on segmentation and detection to suppress clutter and distinguish adjacent radar targets. However, this study does not consider the combination of segmentation and detection results. The paper "RDSNet: A new deep architecture for reciprocal object detection and instance segmentation, Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 07, pp. 12208–12215, 2020" uses segmentation results to refine the bounding box boundaries. However, this work applies to optical images. In radar target detection, accurate confidence prediction is key to reducing false alarm bounding boxes caused by clutter. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a segmentation-assisted radar multi-frame target detection method, which effectively solves the problems of insufficient global feature extraction capability, insufficient ability to distinguish neighboring targets, and insufficient mining of the relationship between segmentation and detection tasks, thereby improving target detection performance and showing significant performance advantages in clutter interference scenarios and neighboring target scenarios.

[0008] The technical solution adopted in this invention is: a segmentation-assisted radar multi-frame target detection method, comprising:

[0009] S1. Perform fast Fourier transform on the measured multi-frame raw radar echo data in the range and azimuth dimensions respectively to obtain multi-frame RA (Range-Azimuth) two-dimensional data.

[0010] S2. Based on the multi-frame RA two-dimensional data from step S1, a dataset is established. The dataset includes: distance-azimuth two-dimensional data, corresponding distance-azimuth view bounding box labels and their mask labels. Specifically, the distance-azimuth view bounding box labels and their mask labels are obtained from the distance-azimuth two-dimensional data to obtain semantically separated mask labels and bounding box labels required for target detection, based on the target location. The dataset is divided into a training set and a test set.

[0011] S3. Construct the object detection network MFDet, and perform multi-task training on the object detection network MFDet based on the training set in step S2;

[0012] S4. Input the test set data from step S2 into the target detection network MFDet trained in step S3 to obtain the detection results, namely the target prediction boxes on the multi-frame range-orientation two-dimensional view.

[0013] The beneficial effects of this invention are as follows: The method of this invention first acquires multi-frame RA data from simulated and measured raw radar echo data, which is used as input to a neural network. A training set and a test set are established, and labels required for multi-frame detection are obtained based on the target location. A multi-frame detection network MFDet is constructed and trained. Then, the trained target detection network outputs the detection results. This method effectively solves the problems of insufficient global feature extraction capability, insufficient ability to distinguish nearby targets, and insufficient mining of the interrelationship between segmentation and detection tasks, thus improving target detection performance and showing significant performance advantages in cluttered and nearby target scenarios.

[0014] This invention overcomes the limitations of existing detection methods based on statistical models by using a data-driven approach to extract deep-level features, enabling intelligent radar target detection in low signal-clutter ratio (SCR) environments. It also overcomes the limitation of convolutional networks in fully perceiving global spatiotemporal features by utilizing the Swing Transformer architecture to fully extract spatiotemporal features of different sizes across multiple frames, thereby more fully extracting the differences between target and clutter features and adapting to multi-size target detection tasks, thus improving target detection performance in cluttered environments. Furthermore, it employs a region-level detection head to generate two-dimensional bounding boxes, achieving multi-frame parallel detection and overcoming the difficulty of distinguishing neighboring targets in pixel-level segmentation tasks, demonstrating significant performance advantages on neighboring target datasets. Finally, it introduces a pixel-level segmentation task to assist in refining bounding box confidence, and based on Bayes' theorem, guides the network to fully explore the interrelationship between segmentation and detection tasks, further improving detection performance. Attached Figure Description

[0015] Figure 1 This is a flowchart of a segmentation-assisted intelligent method for multi-frame target detection in radar according to the present invention.

[0016] Figure 2 This is a diagram of the overall network architecture of the multi-frame target detection network MFDet in this embodiment of the invention.

[0017] Figure 3 This is a label diagram of two simulated samples and a measured sample used for displaying results in an embodiment of the present invention.

[0018] Figure 4 The images show the detection results of the multi-frame target detection network MFDet in simulation and actual test scenarios in this embodiment of the invention.

[0019] Figure 5 The images show the detection results of existing target detection methods based on CFAR and clustering in simulation and real-world scenarios in this invention.

[0020] Figure 6 The images show the detection results of the T-RODNet network in simulation and real-world scenarios in this embodiment of the invention.

[0021] Figure 7 This is a comparison of the detection performance curves of the MFDet network of the present invention with existing CFAR detection and T-RODNet network under different SCR simulation scenarios in the embodiments of the present invention. Detailed Implementation

[0022] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.

[0023] like Figure 1 The flowchart of a segmentation-assisted intelligent method for multi-frame target detection in radar according to the present invention is shown below. The specific steps are as follows:

[0024] S1. Perform fast Fourier transform on the range and azimuth dimensions of the simulated and measured multi-frame raw radar echo data to obtain multi-frame RA two-dimensional data.

[0025] S2. The multi-frame RA two-dimensional data from step S1 is used as the input to the subsequent object detection neural network. This step also includes processing the multi-frame RA two-dimensional data from step S1 to establish a dataset; specifically: obtaining multi-frame semantic segmentation mask labels M and bounding box labels (including position size labels B) required for object detection based on the target location. gt Confidence label O gt ), and establish training and testing sets;

[0026] S3. Based on the training set from step S2, perform multi-task training using the MFDet target detection network, which is based on a 3D Transformer, a parallel multi-frame detection head, and a segmentation-assisted refinement module.

[0027] S4. Using the target detection network trained in step S3, output the detection results of the test set data in step S2, that is, the target prediction box on the multi-frame distance-azimuth two-dimensional view.

[0028] In this embodiment, step S1 is specifically as follows:

[0029] The measured multi-frame raw radar echo data are as follows:

[0030] The radar transmitted signal varies with a fast time t, denoted as s. T (t), s T (t) and the m-th received echo s R (t) Mixing yields the intermediate frequency signal s B (t), then sampled at a frequency of f s Fast time sampling is used to obtain the intermediate frequency signal s at the nth sampling time. B (n), the calculation process is as follows:

[0031]

[0032]

[0033]

[0034] Where μ represents the sweep slope, T c The sweep time width is represented by c, the speed of light is represented by f0, and the radar center frequency is represented by f. b and f vThis indicates that the target distance R0 and radial velocity v are included. r Frequency term of information;

[0035] Let the antenna distance be d and the target angle be θ, then the intermediate frequency signal s of the p-th antenna channel B (n,p) is represented as:

[0036]

[0037] Let N be the number of fast-time sampling points and P be the number of antenna channels. Then the intermediate frequency signal obtained from the m-th received echo can be represented as an N-row, P-column matrix S. B ,in,

[0038] B > n,p =s B (n,p),n=1,2,...N,p=1,2,...P (5)

[0039] For discrete signal matrix S B The fast time dimension and antenna channel dimension are respectively performed N r N a The fast Fourier transform of the point generates the RA two-dimensional data corresponding to the m-th received echo. That is, to perform spectral estimation for distance and orientation respectively;

[0040] Where, N r N is the number of distance sampling points, which is 128 in this embodiment. a This is the number of sampling points in the azimuth direction; in this embodiment, the value is 128. Let D represent the set of real numbers, then the coordinates of the target point in D are [k...]. r ,k a ] is represented as:

[0041]

[0042] Accumulate N continuously over a certain time interval f For each received echo, the above processing is performed to obtain N. f Two-dimensional RA data frames are ultimately converted into multi-frame RA data.

[0043] The process of acquiring multi-frame RA two-dimensional data of simulated radar echoes is the same as that in formulas (1)-(6). Subsequently, simulated radar echo data under different signal-to-clutter ratio scenarios are used to verify the performance of the method of this invention.

[0044] ​The simulated radar echo data is obtained through simulation using MATLAB software with specific parameter settings. These parameters include: radar operating frequency, sweep bandwidth, sweep duration, transmit gain, sampling rate, speed of light, element spacing, number of elements, and Fast Fourier Transform (FFT) sampling points. The FFT sampling points include settings for the number of sampling points in the range dimension, azimuth dimension, and Doppler dimension. In this embodiment, the specific parameter settings are: radar operating frequency: 77.3e9, sweep bandwidth: 0.5e9, sweep duration: 20.48e-6, transmit gain: 2, sampling rate: 12.5e6, speed of light: 3e8, element spacing: 1.5e-3, number of elements: 16, and FFT sampling points: 128 for range dimension, 128 for azimuth dimension, and 64 for Doppler dimension.

[0045] In this embodiment, step S2 is specifically as follows:

[0046] Based on the target's location and size, the format is (id) img ,id fra ,cls,c x ,c y The bounding box labels for (w,h);

[0047] Among them, id img The id represents the index of the sample to which the target belongs in the current batch. fra This indicates the frame index of the target within the current batch, cls indicates the target category, (c x ,c y (w,h) represents the center point coordinates and width and height values ​​of the target in the 2D view; cls is the confidence label O. gt The remaining six are location size labels B. gt .

[0048] The mask labels are pixel-level and have the same size as the input image.

[0049] The training set includes: RA two-dimensional data, corresponding RA view bounding box labels and their mask labels.

[0050] In this embodiment, step S3 is specifically as follows:

[0051] S31. Construct the MFDet network architecture;

[0052] The overall network architecture is as follows Figure 2As shown, it includes a feature extraction module, a semantic segmentation module, a multi-frame target detection module, and a segmentation-aided refinement module. The feature extraction module extracts spatiotemporal correlation features from multi-frame data to obtain two different scales of multi-frame bottleneck features F1 and F2 to adapt to multi-scale targets; the semantic segmentation module obtains the segmentation result M of the multi-frame RA data based on the multi-frame bottleneck features. s This enables pixel-level confidence prediction; the multi-frame object detection module, based on bottleneck features, predicts bounding box information D in parallel across multiple frames on the feature map. d This includes confidence level, center point location, and bounding box size; the segmentation-aided refinement module is based on the semantic segmentation result M. s Further correction of the bounding box confidence level yields refined bounding box information D. r The specific process of the proposed SARM is as follows: Figure 3 As shown. During the testing phase, the predicted bounding boxes are subjected to threshold filtering and non-maximum suppression algorithms to obtain the final multi-frame RA detection boxes.

[0053] a) Feature extraction module;

[0054] like Figure 2 As shown, the feature extraction module includes: l d Each downsampling module, l u An upsampling module based on a 3D convolutional layer;

[0055] The first downsampling block includes: image patch partitioning and linear embedding. For the input... First, the image is divided into a size s using a 3D convolutional layer. f ×s r ×s a The number of small pieces is Then perform linear mapping on the image patches to obtain There are several embedding vectors, the dimension m of which is determined by the specific situation. Image segmentation and linear embedding operations can be obtained by a three-dimensional convolutional layer, where the stride and size of the three-dimensional convolutional kernel are both set to s. f ×s r ×s a The process of outputting the 3D feature F1 of the first sampling block is as follows:

[0056]

[0057] in, Indicates rounding up, the size of the 3D feature F1 is... The number of channels is determined based on the actual situation; W represents a 3D convolution operation. D1 and BD1 This represents the 3D convolution kernel parameters and bias terms of the first downsampled block. This indicates the LayerNorm normalization layer;

[0058] The following l d The -1 downsampling block includes: a 3D shifted window self-attention module and a tile fusion module. The second downsampling block is used as an example below. In the 3D shifted window self-attention module, the 3D feature F1 is first divided into K non-overlapping 3D windows, and self-attention operations are performed within each window. The window size is determined based on the actual situation. Then, the window is moved, and self-attention operations are performed again within the shifted window. The shifted window contains elements from the original adjacent windows, thus realizing information interaction between the original adjacent windows and increasing the receptive field of the self-attention operation. The 3D feature F1 is processed by the 3D shifted window self-attention module to obtain feature F. s1 The process is as follows:

[0059]

[0060] in and This indicates the self-attention operations within the window and the shifted window in the second downsampling block.

[0061] In the above-described in-window self-attention mechanism, for N within the k-th window... w Embedded vectors From matrix W Q W K and W V We obtain Query(Q), key(K), and Value(V).

[0062] Q = A × W Q K = A × W K V = A × W V (9)

[0063] Among them, Q, K, The i-th embedding vector With the j-th embedding vector Relevance is measured by attention weight w ij This indicates that the higher the relevance, the better. ij The higher the value, the better the calculation of the i-th output vector. At that time, more attention will be paid to The attention weight matrix W is obtained by the following formula:

[0064]

[0065] Where d kThis represents the feature dimension of each head in the multi-head self-attention mechanism. The i-th output vector. Calculated by the following formula:

[0066]

[0067] The final output is obtained Since j in equation (11) takes the value j = 1, 2, ... N, each output vector contains global perception within the window. The outputs B1, B2, ... B of the K windows are then... K Arranging the windows in their original positions yields the three-dimensional feature F. s1 Due to the input A of the attention operation within the window. k and output B k The dimensions are consistent, therefore features F1 and F s1 The dimensions are consistent.

[0068] In the tile fusion module, for the 3D feature F s1 Divide it into 2×2×2 blocks, the number of blocks is Embedding vectors at the same positions in each patch are extracted and combined to obtain 8 new patches. These patches are then concatenated along the channel dimension and linearly mapped to obtain the 3D feature F2 with a size of [size missing]. Compared to F s1 The size of feature F2 is halved while the number of channels is doubled, further expanding the receptive field. The process of obtaining feature F2 from 3D feature F1 through the second downsampling block is as follows:

[0069]

[0070] in This indicates the tile fusion module in the second downsampling block.

[0071] Similarly, for the i-th downsampling block, the three-dimensional feature F i-1 Output three-dimensional features F i The process is as follows:

[0072]

[0073] in and This represents the self-attention operation within the window and within the shifted window in the i-th downsampling block. This represents the tile fusion module in the i-th downsampling block.

[0074] All upsampling blocks include: a 3D deconvolution layer, a LayerNorm normalization layer, and a LeakyReLU activation function. The upsampling output undergoes a skip connection operation and is concatenated with the corresponding size features from the downsampling process in the channel dimension to obtain features F of different sizes. j as follows:

[0075]

[0076] Three-dimensional feature F 2ld-1 Represents the downsampling layer features with the same size as the current upsampling layer; 3D feature F j The size is F j-1 The number of channels is determined by the number of convolution kernels; depending on the actual hardware and task requirements, some upsampling layers are selected as feature layers, and the output features of these feature layers will jointly form the three-dimensional bottleneck features F of the MFDet network. The symbol ⊕ represents the transpose convolution operation; ⊕ represents the channel dimension concatenation operation. This represents a 3D convolution operation with a kernel size of 1×1×1, used to recover the number of channels; such as Figure 2 In this embodiment, the outputs of two upsampling layers are selected to jointly form the three-dimensional bottleneck feature F of the MFDet network.

[0077] b) Multi-frame target detection module;

[0078] like Figure 2 As shown, the multi-frame detection head takes the three-dimensional spatiotemporal bottleneck feature F as input and performs upsampling in the time dimension to recover the number of frames N. f Then, the 3D features are transformed by the number of channels, and the RA prediction box offset coefficient P of each frame is output. RA , by offset coefficient P RA The information D of the prediction box is calculated. d The process is as follows:

[0079] P RA =f Conv1×1 (f upSam_Det (F)) (15)

[0080]

[0081] Where f upSam_Det (·) indicates that upsampling is performed only on the time dimension to recover the number of frames N. f ;f Conv1×1 (·) represents a two-dimensional convolution operation with a kernel size of 1×1, used to change the number of channels; Indicates the offset coefficient P RA To the prediction box information D d The mapping. Predicted box offset coefficient PRA Includes: In each grid cell of the feature map, in N anc The center point coordinate offset coefficient (p) under each anchor template x ,p y Width and height offset coefficient (p) w ,p h ), confidence coefficient p obj Information D of the prediction box d Includes bounding box location and size information B and confidence level O. d , by P RA To D d The mapping operations are as follows:

[0082]

[0083] Among them, (c x ,c y (a) represents the coordinates of the top-left corner of the grid to which the prediction box belongs. w ,a h ) represents the width and height values ​​of the anchor template; s represents the downsampling rate of the feature map; σ(·) represents the sigmoid function. Considering a feature map of size [h, w], the output P RA The size is [N f N anc [,h,w,5] indicates that (N f ×N anc (×h×w) bounding boxes were generated, and for each bounding box, offset coefficients for five pieces of information were predicted: offset coefficients for the center point coordinates, width and height values, and confidence level.

[0084] The multi-frame target detection module jointly senses intra-frame spatial features and inter-frame temporal features, extending two-dimensional detection to spatiotemporal three-dimensional detection, and realizing the parallel output of multi-frame RA plane detection results.

[0085] c) Semantic segmentation module;

[0086] like Figure 2 As shown, the semantic segmentation module takes the three-dimensional bottleneck feature F as input and outputs N. f The frame size is N r ×N a Confidence plot M s M s One pixel value corresponds to the input RA Figure 1 The confidence level of each distance-azimuth cell is calculated. The process is as follows:

[0087] M s =f upSam_Seg (F) (18)

[0088] Where f upSam_Seg(·) indicates that transposed convolution is applied to the time, distance, and orientation dimensions for upsampling to recover the number of frames N. f Number of distance-oriented units N r And the number of azimuth units N a .

[0089] Compared to grid-level bounding box detection, pixel-level segmentation can predict the confidence level of each pixel more precisely. Therefore, the segmentation results can provide a reasonable prior for bounding box confidence and have the potential to assist in correcting bounding box confidence.

[0090] d) Segmentation-aided refinement module

[0091] like Figure 2 As shown, the segmentation-aided refinement module uses the confidence map M s And prediction box information D d As input, the output is refined prediction box information D. r Specifically, by M s The segmentation confidence level O is obtained by sampling the center point coordinates of the predicted bounding box and using bilinear interpolation. s ; Fusion segmentation confidence level O s And the confidence level of the prediction box O d Obtain refined confidence level O r ; use O r Replace the original confidence information O d This yields refined prediction box information D. r The process is as follows:

[0092]

[0093]

[0094] in This represents sampling and bilinear interpolation operations. This indicates the integration of refined operations. The specific process is as follows:

[0095]

[0096] Among them, c r This is the corrected bounding box category, with values ​​either target (tg) or background (bg); s o d and o r These are individual bounding boxes in O s O d and O r The value in represents the confidence level of the bounding box segmentation result, the confidence level of the detection result, and the corrected confidence level, and the value range is [0,1]. Specifically, in equation (21), the prior probability term is calculated as follows:

[0097]

[0098] The remaining terms in equation (21) are likelihood probabilities, estimated using a three-dimensional convolutional layer. Specifically, the segmentation confidence O... s and bounding box confidence O d As two channels, a 3D convolutional kernel is used to fuse the segmentation and detection confidence scores, and a sigmoid function is used to output the corrected confidence score O. r .

[0099] S32. Construct the loss function for multi-frame parallel detection and multi-task training;

[0100] For multi-frame parallel 2D detection and segmentation tasks, the following multi-task loss function is adopted:

[0101] L=λ1L bbox +λ2L obj +λ3L seg (twenty three)

[0102] Among them, L bbox The loss function representing the bounding box position and size is the CioU loss; L obj The bounding box confidence loss function uses binary cross-entropy loss; L seg The loss for the segmentation task is represented by MSE loss; λ1, λ2, and λ3 represent the weights of the three losses. Specifically, L bbox L obj and L seg The calculation formula is as follows:

[0103]

[0104]

[0105]

[0106] Where m sij and m ij M respectively s The values ​​of M in the i-th distance cell and the j-th orientation cell; N bbox Indicates the total number of bounding boxes; o i and o gti Representing O r and O gt The i-th value, i.e., the confidence score and confidence label of the i-th bounding box; b i and b gti They represent B and B respectively. gtThe i-th vector, i.e., the center point size and label of the i-th bounding box; CIoU(·) represents the calculation of the Complete Intersection over Union (CIoU) for two bounding boxes.

[0107] S33. Conduct network training;

[0108] The training process includes forward propagation and back propagation. Forward propagation obtains the prediction results through the neural network, while back propagation continuously optimizes the parameters of each network layer by calculating the gradient of the loss function in reverse. During training, training stops when the difference between the loss function values ​​of two adjacent iterations is less than or equal to 1e-2.

[0109] In this embodiment, step S4 is specifically as follows:

[0110] Input multi-frame RA amplitude data with unknown target quantity and location, and output multi-frame detection results in parallel, i.e. target prediction boxes on the RA two-dimensional view. Figure 3 The bounding box labels and segmentation mask labels of two simulated samples and two measured samples are shown. Figure 4 The output of the four samples after processing with MFDet is shown, including the segmentation confidence map and bounding box detection results.

[0111] like Figure 5 As shown, existing CFAR detection methods... Figure 3 The detection results of four samples. Figure 6 For comparison, the T-RODNet network is used. Figure 4 The detection results of four samples. Figure 7 The results of the simulation dataset comparison experiment in this embodiment are used to compare the detection performance curves of the MFDet network of the present invention with those of the existing CFAR detection and T-RODNet network in different SCR simulation scenarios, using performance indicators recall, accuracy and F1 score.

[0112] The results of the comparative experiment on the actual test dataset in this embodiment, that is, the detection results of the method of the present invention and the two comparative methods in the actual test scenario, are presented using performance indicators recall rate, accuracy and F1 score, as shown in Table 1.

[0113] Table 1. Detection performance metrics of the MFDet network of the present invention and existing technologies on the CARRADA test dataset.

[0114]

[0115] In summary, the method of this invention targets multi-frame range-azimuth two-dimensional detection of radar targets. Based on the intra-frame and inter-frame spatiotemporal correlation features in multi-frame RA data, it utilizes a three-dimensional Swing Transformer architecture to jointly extract features. A region-level detection head is used to predict bounding boxes for each target instance in the multi-frame data, and a segmentation task is introduced to refine the confidence of the bounding boxes. This enables the network to effectively extract the differences in global spatiotemporal correlation features between targets and clutter, improving the network's detection performance in clutter environments, especially for nearby targets. Compared to existing CFAR detection and T-RODNet networks, the method of this invention has significant performance advantages in recall, accuracy, and F1 score, achieving certain performance improvements in clutter and nearby target scenarios.

[0116] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A segmentation-assisted radar multi-frame target detection method, characterized in that, include: S1. Perform fast Fourier transform on the measured multi-frame raw radar echo data in the range and azimuth dimensions respectively to obtain multi-frame range-azimuth two-dimensional data. S2. Based on the multi-frame range-azimuth 2D data from step S1, a dataset is established. The dataset includes: range-azimuth 2D data, corresponding range-azimuth view bounding box labels and their mask labels. Specifically, the range-azimuth view bounding box labels and their mask labels are obtained from the range-azimuth 2D data to obtain semantically separated mask labels and bounding box labels required for target detection, based on the target location. The dataset is divided into a training set and a test set. S3. Construct the object detection network MFDet, and perform multi-task training on the object detection network MFDet based on the training set of step S2; the object detection network MFDet specifically includes: a feature extraction module, a semantic segmentation module, a multi-frame object detection module, and a segmentation auxiliary refinement module; The feature extraction module includes several downsampling modules and several upsampling modules. The output of the upsampling modules, after a skip connection operation, is concatenated with the output features of the corresponding-sized downsampling modules in the channel dimension to obtain features of different sizes. The outputs of the upsampling modules are then combined to form a three-dimensional bottleneck feature. This serves as the output of the feature extraction module. The semantic segmentation module uses three-dimensional bottleneck features As input, output Frame size is Confidence plot , A pixel value corresponds to the confidence level of a distance-orientation cell in the input mask label; It is the number of sampling points in the distance direction. It is the number of sampling points in the azimuth direction; The multi-frame target detection module uses 3D bottleneck features As input, upsampling is performed in the time dimension to recover the number of frames. Then, the 3D features are transformed by the number of channels, and the output is the distance-orientation prediction box offset coefficient for each frame. , by offset coefficient Calculate the predicted bounding box information ; The segmentation-aided refinement module uses confidence maps and prediction box information As input, the output is refined prediction box information. Specifically: from the confidence plot The segmentation confidence score is obtained by sampling the center point coordinates of the predicted bounding box and using bilinear interpolation. ; Fusion segmentation confidence and prediction box confidence Obtain refined confidence level ;use Replace the original confidence information To obtain refined prediction box information ; S4. Input the test set data from step S2 into the target detection network MFDet trained in step S3 to obtain the detection results, namely the target prediction boxes on the multi-frame range-orientation two-dimensional view.

2. The segmentation-assisted radar multi-frame target detection method according to claim 1, characterized in that, The bounding box label includes a location size label and a confidence level label.

3. The segmentation-assisted radar multi-frame target detection method according to claim 2, characterized in that, The downsampling module is specifically a downsampling module based on 3D Transformer. The first downsampling module among several downsampling modules based on 3D Transformer includes: image segmentation operation and linear embedding operation. The image segmentation operation divides the input image into multiple image blocks, and the linear embedding operation performs linear mapping on the image blocks to obtain the embedding vector. The remaining downsampling modules have the same structure, including: a 3D shifted window self-attention module and a tile fusion module. In the 3D shifted window self-attention module, the 3D features output by the first downsampling module are first divided using multiple non-overlapping 3D windows, and self-attention operations are performed within the windows. Then, the windows are moved, and self-attention operations are performed again within the shifted windows. The tile fusion module divides the 3D features output by the 3D shifted window self-attention module into... The embedding vectors at the same position in each small block are extracted and combined to obtain 8 new blocks. These blocks are then spliced ​​along the channel dimension and linearly mapped.

4. The segmentation-assisted radar multi-frame target detection method according to claim 3, characterized in that, The upsampling module is an upsampling module based on a three-dimensional deconvolution layer; Each upsampling block has the same structure, including: a 3D deconvolution layer, a LayerNorm normalization layer, and a LeakyReLU activation function.

5. The segmentation-assisted radar multi-frame target detection method according to claim 4, characterized in that, The multi-frame object detection module is implemented using a 3D convolutional layer to perceive spatiotemporal features and output bounding boxes of multiple frames in parallel.

6. The segmentation-assisted radar multi-frame target detection method according to claim 5, characterized in that, The segmentation-assisted refinement module, based on Bayes' theorem, uses a 3D convolutional layer to estimate the likelihood probability in Bayes' theorem, thereby refining the segmentation results for the confidence of the bounding box.

7. The segmentation-assisted radar multi-frame target detection method according to claim 6, characterized in that, The loss function for sampling during the training of the object detection network MFDet is: ； in, The loss function representing the bounding box position and size is the CioU loss. The bounding box confidence loss function is represented by a binary cross-entropy loss. The loss for task splitting is represented by MSE loss; , and This represents the weights of the three losses.

Images