Radar distance-Doppler graph target feature extraction method and system based on bidirectional feature pyramid network

Abstract

The invention discloses a radar distance-Doppler graph target feature extraction method and system based on a bidirectional feature pyramid network, and relates to the technical field of artificial intelligence and radar echo target detection. According to the method, the calculation efficiency is improved by improving a multi-scale feature fusion network structure connected with a backbone network, and proper weights are added to feature maps with different scales through a rapid normalization fusion method with the weights, so that the contribution degree of the feature maps to detection is distinguished; therefore, the problem that noise points in a radar echo distance-Doppler map are misjudged as targets by the neural network due to insufficient network feature extraction capability in the prior art is solved.

Description

Technical Field

[0001] The invention relates to an echo detection technology, in particular to a radar range-Doppler image target feature extraction method and system based on a bidirectional feature pyramid network. Background Art

[0002] The radar range-Doppler (RD) two-dimensional image is usually obtained by coherently processing multiple pulse signals of the coherent radar. The image is composed of the target's scattered point echoes and various clutter and interference. Compared with general optical images, it is difficult to extract features, which is specifically reflected in:

[0003] (1) The amplitude of the echo from the target's strong scattering point varies greatly and may have irregular shapes.

[0004] (2) A large amount of clutter and interference blurs the boundary between the target and the background, and some prominent noise points may be mistaken for targets under traditional image recognition methods.

[0005] Therefore, the performance of general neural network image recognition methods in radar RD map target detection and recognition tasks still has a lot of room for improvement, which is mainly reflected in the inability to effectively extract the RD map features of the corresponding target, resulting in many misclassification phenomena.

[0006] Therefore, there is an urgent need for a radar range-Doppler map target feature extraction method and system based on a bidirectional feature pyramid network. Summary of the invention

[0007] The purpose of this specification is to provide a radar range-Doppler map target feature extraction method and system based on a bidirectional feature pyramid network. This method can solve the problem that noise points in the radar RD map are misjudged as targets by the neural network due to the insufficient feature extraction capability of the existing network, and improve the accuracy of target detection and recognition. The specific technical solution is as follows:

[0008] In a first aspect, the present invention provides a radar range-Doppler map target feature extraction method based on a bidirectional feature pyramid network, the method comprising:

[0009] The training sample set is fused with the multi-scale feature map to obtain a multi-scale fusion map; the multi-scale feature map is generated by the backbone neural network;

[0010] The multi-scale fusion graph is input into a bidirectional feature pyramid network for weighted fusion processing to generate a fused feature graph; the nodes in the bidirectional feature pyramid network use a weighted fast normalization method to add weights to feature graphs of different levels;

[0011] The fused feature map is input into the classification network and the border detection network for processing to generate image data with classification labels and a frame map of the target location.

[0012] Furthermore, before fusing the training sample set with the multi-scale feature map, the process further includes:

[0013] The radar range-Doppler map dataset is divided into a training set and a test set according to a specified ratio;

[0014] The training set is preprocessed to generate the training sample set, and the test set is preprocessed to generate the test sample set.

[0015] Furthermore, the preprocessing process is as follows:

[0016] Scaling the sample sizes in the radar range-Doppler graph data to the same size to obtain scaled graph data;

[0017] The zoom image data is normalized according to the maximum and minimum values ​​of the element amplitudes.

[0018] Furthermore, the normalization process is as follows:

[0019] Extract the maximum and minimum values ​​of the amplitude of the zoom image data elements, and normalize all elements according to the following formula:

[0020]

[0021] Among them, X train is the collection of matrices formed by the zoomed graph data, x is a matrix element and x∈X, where X is the zoomed graph data, min() represents the minimum value operation, and max() represents the maximum value operation.

[0022] Furthermore, the step of inputting the multi-scale fusion graph into a bidirectional feature pyramid network for weighted fusion processing includes:

[0023] Divide the multi-scale fusion graph into five layers and input them into the input layer of the bidirectional feature pyramid network, wherein the input layer is provided with five nodes, and the five nodes of the input layer correspond to the three nodes of the middle layer;

[0024] The multi-scale fusion graph is subjected to weighted feature fusion convolution processing in three nodes of the middle layer to obtain the middle feature graphs corresponding to the three nodes;

[0025] The intermediate feature map and the multi-scale fusion map are subjected to weighted feature fusion convolution processing to obtain a fused feature map.

[0026] Furthermore, the network includes: an input layer, an intermediate layer, and an output layer;

[0027] The input layer is provided with 5 nodes, the middle layer is provided with 3 nodes, and the output layer is provided with 5 nodes; the nodes of the input layer are connected to the nodes of the output layer at the same depth, the first node of the input layer is connected to the first node of the middle layer, the second node of the input layer is connected to the first node of the middle layer, the third node of the input layer is connected to the second node of the middle layer, the fourth node of the input layer is connected to the third node of the middle layer, the nodes of the middle layer are connected to the nodes of the output layer at the same depth, the first node of the middle layer is connected to the second node, the second node of the middle layer is connected to the third node of the middle layer, the third node of the middle layer is connected to the fifth node of the output layer, the fifth node of the output layer is connected to the fourth node of the output layer, the fourth node of the output layer is connected to the third node of the output layer, the third node of the output layer is connected to the second node of the output layer, and the second node of the output layer is connected to the first node of the output layer.

[0028] In a second aspect, the present invention also provides a radar range-Doppler map target feature extraction system based on a bidirectional feature pyramid network, the system comprising: a preliminary fusion module, a weighted fusion module, and a detection module;

[0029] The preliminary fusion module is used to fuse the training sample set with the multi-scale feature map to obtain a multi-scale fusion map; the multi-scale feature map is generated by the backbone neural network;

[0030] The weighted fusion module is used to input the multi-scale fusion image into the bidirectional feature pyramid network for weighted fusion processing to generate a fused feature image; the nodes in the bidirectional feature pyramid network use a weighted fast normalization method to add weights to feature images of different levels;

[0031] The detection module is used to input the fused feature map into the classification network and the border detection network for processing, and generate image data with classification labels and a frame map of the target location.

[0032] In another embodiment of this aspect, the system further comprises: a preprocessing module;

[0033] The preprocessing module is used to divide the radar range-Doppler map data set into a training set and a test set according to a specified ratio; preprocess the training set to generate the training sample set, and preprocess the test set to generate the test sample set.

[0034] In another embodiment of this aspect, the preprocessing process is as follows:

[0035] Scaling the sample sizes in the radar range-Doppler graph data to the same size to obtain scaled graph data;

[0036] The zoom image data is normalized according to the maximum and minimum values ​​of the element amplitudes.

[0037] In another embodiment of this aspect, the weighted fusion module is specifically used for:

[0038] Divide the multi-scale fusion graph into five layers and input them into the input layer of the bidirectional feature pyramid network, wherein the input layer is provided with five nodes, and the five nodes of the input layer correspond to the three nodes of the middle layer;

[0039] The multi-scale fusion graph is subjected to weighted feature fusion convolution processing in three nodes of the middle layer to obtain the middle feature graphs corresponding to the three nodes;

[0040] The intermediate feature map and the multi-scale fusion map are subjected to weighted feature fusion convolution processing to obtain a fused feature map.

[0041] The beneficial effects of the present invention are as follows:

[0042] The present invention discloses a radar range-Doppler map target feature extraction method and system based on a bidirectional feature pyramid network, and relates to the technical field of artificial intelligence and radar echo target detection. The method improves the computational efficiency by improving the multi-scale feature fusion network structure connected by the backbone network, and adds appropriate weights to feature maps of different scales through a weighted fast normalization fusion method to distinguish the contribution of feature maps to detection, thereby solving the problem that noise points in the radar echo range-Doppler map are misjudged as targets by the neural network due to the insufficient feature extraction capability of the existing network. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the flow of a radar RD image target feature extraction method based on a bidirectional feature pyramid network provided in this specification;

[0044] Figure 2 A schematic diagram of the bidirectional feature pyramid network structure for radar RD image target feature extraction provided in this specification. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and their corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this document.

[0046] The following combination Figure 1-2 , describes in detail the technical solutions provided by each embodiment of this specification. Specific embodiment 1:

[0048] A method for extracting target features from radar RD images based on a bidirectional feature pyramid network. The method improves the traditional bidirectional feature pyramid structure by removing the middle layer nodes with only a single input. Such nodes will not fuse the input feature map, so their contribution to feature fusion is low. Removing such nodes can reduce the amount of network calculation without affecting the feature fusion effect. The structure can fuse more features for the output layer without increasing too much calculation by directly adding the input layer feature map to the input of the output layer node. In addition, the bidirectional feature pyramid structure can be repeatedly connected, and the output of the previous bidirectional feature pyramid is used as the input feature map of the next bidirectional feature pyramid, so as to extract higher-level features. When fusing multiple feature maps of different resolutions, a fast normalization method with weights is used to add appropriate weights to feature maps of different levels, ultimately improving the target detection and recognition performance of the network.

[0049] Step 1: Divide the radar RD image dataset into a training set and a test set according to a specified ratio.

[0050] Step 2: preprocess the training data set to form a training sample set. Perform the same processing on the test data set according to the preprocessing parameters of the training data set to form a test sample set.

[0051] Step 2.1 Scale all RD graph samples to the same size

[0052] Step 2.2 Amplitude MinMax normalization processing.

[0053] Take the maximum and minimum values ​​of the amplitude of all elements in the training data set, and normalize all elements according to the following formula:

[0054]

[0055] Among them, X train is the collection of RD graph matrices of the training data set, x is a matrix element and x∈X, where X is any RD graph data in the training set and the test set, min() represents the minimum value operation, and max() represents the maximum value operation.

[0056] The above formula normalizes all matrix elements of any RD graph data in the training set and the test set to obtain the normalized matrix element x′ corresponding to the matrix element x. The above linear normalization can map the radar echo amplitude to [0,1]. At this time, the order of magnitude difference between different data samples can be eliminated, thereby reducing the number of iterations when using the gradient descent method to find the optimal solution.

[0057] Step 3: Input the training sample set into the backbone neural network training, fuse the multi-scale feature map generated by the backbone neural network, input the bidirectional feature pyramid network, and obtain the fused feature map output by the network. The training sample at this time is a normalized training set.

[0058] Use EfficientnetB0 as the backbone network to extract the feature maps output by the 3rd to 7th layers of the network. Figure 2 The bidirectional feature pyramid network structure consists of an input layer and a feature extraction layer. The input layer consists of 5 nodes, which correspond to the 3rd to 7th layers of the backbone network and receive the feature maps output by them, denoted as {P7, P6, P5, P4, P3}. The feature extraction layer consists of 1 intermediate layer and 1 output layer.

[0059] The middle layer consists of 3 nodes, each of which has multiple input feature maps, and one output feature map is obtained through feature fusion convolution calculation. The intermediate feature map output by the 3 nodes is recorded as The calculation method is:

[0060]

[0061] Among them, Conv() is the convolution calculation, f(P) is the fast normalized fusion function with weights, and the calculation method is:

[0062]

[0063] Among them, w i is the network training weight parameter, P i ∈P is the input feature map, and ε=0.0001 is the base value for increasing computational stability.

[0064] When fusing multiple feature maps of different resolutions, the above-mentioned weighted fast normalization method is used to add appropriate weights to feature maps of different levels, which can more accurately extract features at different levels and ultimately improve the network's target detection and recognition performance.

[0065] The output layer consists of 5 nodes. Each node performs a weighted fast fusion calculation on the input feature map and the intermediate layer feature map to obtain the fused output feature map. The output feature map of the 5 nodes is denoted as {P7 out ,P6out ,P5 out ,P4 out ,P3 out}, the calculation method is:

[0066]

[0067] Step 4: Input the fused feature map into the classification network and the bounding box detection network for training. Use the test sample set to input the network to obtain the classification label of the sample and the target location box. The performance of the neural network is evaluated by accuracy, recall rate, and average precision.

[0068] Precision, recall, and average precision are indicators for measuring the performance of neural networks. Average precision is the calculation result when the detection box threshold IoU = 0.5. The three indicators are recorded as {Prec, Rec, AP@.5}. The calculation method is:

[0069]

[0070] Among them, TP is the number of samples in a certain category that are correctly identified as this category, FP is the number of samples of other categories that are incorrectly identified as this category, FN is the number of samples of this category that are not correctly identified, and PR(r) is the precision-recall curve of this category. Specific embodiment 2:

[0072] The purpose of this specification is to provide a radar range-Doppler map target feature extraction method and system based on a bidirectional feature pyramid network. This method can solve the problem that noise points in the radar RD map are misjudged as targets by the neural network due to the insufficient feature extraction capability of the existing network, and improve the accuracy of target detection and recognition. The specific technical solution is as follows:

[0073] In a first aspect, the present invention provides a radar range-Doppler map target feature extraction method based on a bidirectional feature pyramid network, the method comprising:

[0074] The training sample set is fused with the multi-scale feature map to obtain a multi-scale fusion map; the multi-scale feature map is generated by the backbone neural network;

[0075] The multi-scale fusion graph is input into a bidirectional feature pyramid network for weighted fusion processing to generate a fused feature graph; the nodes in the bidirectional feature pyramid network use a weighted fast normalization method to add weights to feature graphs of different levels;

[0076] The fused feature map is input into the classification network and the border detection network for processing to generate image data with classification labels and a frame map of the target location.

[0077] Furthermore, before fusing the training sample set with the multi-scale feature map, the process further includes:

[0078] The radar range-Doppler map dataset is divided into a training set and a test set according to a specified ratio;

[0079] The training set is preprocessed to generate the training sample set, and the test set is preprocessed to generate the test sample set.

[0080] Furthermore, the preprocessing process is as follows:

[0081] Scaling the sample sizes in the radar range-Doppler graph data to the same size to obtain scaled graph data;

[0082] The zoom image data is normalized according to the maximum and minimum values ​​of the element amplitudes.

[0083] Furthermore, the normalization process is as follows:

[0084] Extract the maximum and minimum values ​​of the amplitude of the zoom image data elements, and normalize all elements according to the following formula:

[0085]

[0086] Among them, X train is the collection of matrices formed by the zoomed graph data, x is a matrix element and x∈X, where X is the zoomed graph data, min() represents the minimum value operation, and max() represents the maximum value operation.

[0087] Furthermore, the step of inputting the multi-scale fusion graph into a bidirectional feature pyramid network for weighted fusion processing includes:

[0088] Divide the multi-scale fusion graph into five layers and input them into the input layer of the bidirectional feature pyramid network, wherein the input layer is provided with five nodes, and the five nodes of the input layer correspond to the three nodes of the middle layer;

[0089] The multi-scale fusion graph is subjected to weighted feature fusion convolution processing in three nodes of the middle layer to obtain the middle feature graphs corresponding to the three nodes;

[0090] The intermediate feature map and the multi-scale fusion map are subjected to weighted feature fusion convolution processing to obtain a fused feature map.

[0091] In a second aspect, the present invention also provides a radar range-Doppler map target feature extraction system based on a bidirectional feature pyramid network, the system comprising: a preliminary fusion module, a weighted fusion module, and a detection module;

[0092] The preliminary fusion module is used to fuse the training sample set with the multi-scale feature map to obtain a multi-scale fusion map; the multi-scale feature map is generated by the backbone neural network;

[0093] The weighted fusion module is used to input the multi-scale fusion image into the bidirectional feature pyramid network for weighted fusion processing to generate a fused feature image; the nodes in the bidirectional feature pyramid network use a weighted fast normalization method to add weights to feature images of different levels;

[0094] The detection module is used to input the fused feature map into the classification network and the border detection network for processing, and generate image data with classification labels and a frame map of the target location.

[0095] In another embodiment of this aspect, the system further comprises: a preprocessing module;

[0096] The preprocessing module is used to divide the radar range-Doppler map data set into a training set and a test set according to a specified ratio; preprocess the training set to generate the training sample set, and preprocess the test set to generate the test sample set.

[0097] In another embodiment of this aspect, the preprocessing process is as follows:

[0098] Scaling the sample sizes in the radar range-Doppler graph data to the same size to obtain scaled graph data;

[0099] The zoom image data is normalized according to the maximum and minimum values ​​of the element amplitudes.

[0100] In another embodiment of this aspect, the weighted fusion module is specifically used for:

[0101] Divide the multi-scale fusion graph into five layers and input them into the input layer of the bidirectional feature pyramid network, wherein the input layer is provided with five nodes, and the five nodes of the input layer correspond to the three nodes of the middle layer;

[0102] The multi-scale fusion graph is subjected to weighted feature fusion convolution processing in three nodes of the middle layer to obtain the middle feature graphs corresponding to the three nodes;

[0103] The intermediate feature map and the multi-scale fusion map are subjected to weighted feature fusion convolution processing to obtain a fused feature map.

[0104] In a third aspect, the present invention further provides a bidirectional feature pyramid network, the network comprising: an input layer, an intermediate layer, and an output layer;

[0105] The input layer is provided with 5 nodes, the middle layer is provided with 3 nodes, and the output layer is provided with 5 nodes; the nodes of the input layer are connected to the nodes of the output layer at the same depth, the first node of the input layer is connected to the first node of the middle layer, the second node of the input layer is connected to the first node of the middle layer, the third node of the input layer is connected to the second node of the middle layer, the fourth node of the input layer is connected to the third node of the middle layer, the nodes of the middle layer are connected to the nodes of the output layer at the same depth, the first node of the middle layer is connected to the second node, the second node of the middle layer is connected to the third node of the middle layer, the third node of the middle layer is connected to the fifth node of the output layer, the fifth node of the output layer is connected to the fourth node of the output layer, the fourth node of the output layer is connected to the third node of the output layer, the third node of the output layer is connected to the second node of the output layer, and the second node of the output layer is connected to the first node of the output layer.

[0106] This method improves the traditional bidirectional feature pyramid structure by removing the middle layer nodes with only a single input. Such nodes will not fuse the input feature map, so their contribution to feature fusion is low. Removing such nodes can reduce the amount of network calculation without affecting the feature fusion effect. The structure can fuse more features for the output layer without increasing too much calculation by directly adding the input layer feature map to the input of the output layer node. In addition, the bidirectional feature pyramid structure can be repeatedly connected, and the output of the previous bidirectional feature pyramid is used as the input feature map of the next bidirectional feature pyramid, so as to extract higher-level features. When fusing multiple feature maps of different resolutions, a fast normalization method with weights is used to add appropriate weights to feature maps of different levels, ultimately improving the network's target detection and recognition performance.

[0107] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some embodiments, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A radar range-Doppler map target feature extraction method based on a bidirectional feature pyramid network, characterized in that: The method comprises: The training sample set is fused with the multi-scale feature map to obtain a multi-scale fusion map; the multi-scale feature map is generated by the backbone neural network; The multi-scale fusion graph is input into a bidirectional feature pyramid network for weighted fusion processing to generate a fused feature graph; the nodes in the bidirectional feature pyramid network use a weighted fast normalization method to add weights to feature graphs of different levels; The fused feature map is input into the classification network and the border detection network for processing to generate image data with classification labels and a frame map of the target location.

2. A radar range-Doppler image target feature extraction method based on a bidirectional feature pyramid network as claimed in claim 1, characterized in that: Before fusing the training sample set with the multi-scale feature map, the method further includes: The radar range-Doppler map dataset is divided into a training set and a test set according to a specified ratio; The training set is preprocessed to generate the training sample set, and the test set is preprocessed to generate the test sample set.

3. A radar range-Doppler map target feature extraction method based on a bidirectional feature pyramid network as claimed in claim 2, characterized in that: The pre-processing process is as follows: Scaling the sample sizes in the radar range-Doppler graph data to the same size to obtain scaled graph data; The zoom image data is normalized according to the maximum and minimum values ​​of the element amplitudes.

4. A radar range-Doppler map target feature extraction method based on a bidirectional feature pyramid network as claimed in claim 3, characterized in that: The normalization process is as follows: Extract the maximum and minimum values ​​of the amplitude of the zoom image data elements, and normalize all elements according to the following formula: Among them, X train is the collection of matrices formed by the zoomed graph data, x is a matrix element and x∈X, where X is the zoomed graph data, min() represents the minimum value operation, and max() represents the maximum value operation.

5. The method for extracting target features of radar range-Doppler images based on a bidirectional feature pyramid network as claimed in claim 1, characterized in that: The step of inputting the multi-scale fusion graph into a bidirectional feature pyramid network for weighted fusion processing comprises: Divide the multi-scale fusion graph into five layers and input them into the input layer of the bidirectional feature pyramid network, wherein the input layer is provided with five nodes, and the five nodes of the input layer correspond to the three nodes of the middle layer; The multi-scale fusion graph is subjected to weighted feature fusion convolution processing in three nodes of the middle layer to obtain the middle feature graphs corresponding to the three nodes; The intermediate feature map and the multi-scale fusion map are subjected to weighted feature fusion convolution processing to obtain a fused feature map.

6. A radar range-Doppler map target feature extraction method based on a bidirectional feature pyramid network as claimed in claim 5, characterized in that: The bidirectional feature pyramid network includes: an input layer, an intermediate layer, and an output layer; The input layer is provided with 5 nodes, the middle layer is provided with 3 nodes, and the output layer is provided with 5 nodes; the nodes of the input layer are connected to the nodes of the output layer at the same depth, the first node of the input layer is connected to the first node of the middle layer, the second node of the input layer is connected to the first node of the middle layer, the third node of the input layer is connected to the second node of the middle layer, the fourth node of the input layer is connected to the third node of the middle layer, the nodes of the middle layer are connected to the nodes of the output layer at the same depth, the first node of the middle layer is connected to the second node, the second node of the middle layer is connected to the third node of the middle layer, the third node of the middle layer is connected to the fifth node of the output layer, the fifth node of the output layer is connected to the fourth node of the output layer, the fourth node of the output layer is connected to the third node of the output layer, the third node of the output layer is connected to the second node of the output layer, and the second node of the output layer is connected to the first node of the output layer.

7. A radar range-Doppler map target feature extraction system based on a bidirectional feature pyramid network, characterized in that: The system comprises: a preliminary fusion module, a weighted fusion module, and a detection module; The preliminary fusion module is used to fuse the training sample set with the multi-scale feature map to obtain a multi-scale fusion map; the multi-scale feature map is generated by the backbone neural network; The weighted fusion module is used to input the multi-scale fusion image into the bidirectional feature pyramid network for weighted fusion processing to generate a fused feature image; the nodes in the bidirectional feature pyramid network use a weighted fast normalization method to add weights to feature images of different levels; The detection module is used to input the fused feature map into the classification network and the border detection network for processing, and generate image data with classification labels and a frame map of the target location.

8. A radar range-Doppler map target feature extraction system based on a bidirectional feature pyramid network as claimed in claim 7, characterized in that: The system also includes: a pre-processing module; The preprocessing module is used to divide the radar range-Doppler map data set into a training set and a test set according to a specified ratio; preprocess the training set to generate the training sample set, and preprocess the test set to generate the test sample set.

9. A radar range-Doppler map target feature extraction system based on a bidirectional feature pyramid network as claimed in claim 8, characterized in that: The pre-processing process is as follows: Scaling the sample sizes in the radar range-Doppler graph data to the same size to obtain scaled graph data; The zoom image data is normalized according to the maximum and minimum values ​​of the element amplitudes.

10. The radar range-Doppler map target feature extraction system based on a bidirectional feature pyramid network as claimed in claim 7, characterized in that: The weighted fusion module is specifically used for: Divide the multi-scale fusion graph into five layers and input them into the input layer of the bidirectional feature pyramid network, wherein the input layer is provided with five nodes, and the five nodes of the input layer correspond to the three nodes of the middle layer; The multi-scale fusion graph is subjected to weighted feature fusion convolution processing in three nodes of the middle layer to obtain the middle feature graphs corresponding to the three nodes; The intermediate feature map and the multi-scale fusion map are subjected to weighted feature fusion convolution processing to obtain a fused feature map.

Images