Radar video-based track-before-detect method and device

Abstract

The invention relates to a track-before-detect method and device based on a radar video, and belongs to the technical field of target tracking. Comprising the following steps: in a radar scanning mode, obtaining and storing radar video data of a current frame; constructing a rotating frame detection network model based on an attention mechanism; reading historical frame and current frame data to construct a detection sample, performing joint detection on continuous multi-frame data, outputting a detected central point heat map and a rotating frame parameter, and storing the detected central point heat map and the rotating frame parameter into a historical detection information base; and calculating the motion state of the target in the corresponding continuous frame according to the center point heat map and the rotating frame parameter, and updating the target tracking information. According to the invention, target detection is carried out through continuous multi-frame radar video data to obtain the motion information of the target, and the target is tracked.

Description

Technical Field

[0001] The invention relates to a pre-detection tracking method and device based on radar video, which is suitable for radar target tracking and belongs to the technical field of target tracking. Background Art

[0002] Traditional target tracking methods need to first condense the target echo information and use the obtained point coordinate information for tracking. The step-by-step processing flow only retains the location information of clutter and actual measurement, but loses the target's echo intensity and morphology information, affecting the tracking effect. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention provides a pre-detection tracking method and device based on radar video. The radar video target tracking method and device of the present invention perform target detection through continuous multi-frame radar video data, obtain the target's motion information, and track the target.

[0004] One of the purposes of the present invention is to provide a radar video-based pre-detection tracking method, which is characterized in that it comprises the following steps:

[0005] Step 1: In radar scanning mode, obtain and store the current frame radar video data;

[0006] Step 2: Build a rotation box detection network model based on attention mechanism;

[0007] Step 3: Read historical frames and current frame data to construct detection samples, perform joint detection on multiple consecutive frames of data, output the center point heat map and rotation box parameters after detection, and store them in the historical detection information library;

[0008] Step 4: According to the center point heat map and the rotation box parameters, calculate the motion state of the target in the corresponding continuous frames and update the target tracking information.

[0009] Furthermore, the step 2 specifically includes:

[0010] Step 2.1: Construct a convolution module to extract features from multiple frames of data and generate feature maps;

[0011] Step 2.2: Construct an attention module, including spatial attention and channel attention, and use the attention module to process the feature map generated in step 2.1. The channel attention extracts the change trend between target frames and captures the motion characteristics of the target. The spatial attention is used to extract the spatial position characteristics of the target in a single frame.

[0012] Step 2.3: Construct a prediction module to predict the center point heat map and rotation box parameters;

[0013] Step 2.4: Build a historical detection information database;

[0014] Step 2.4.1: Create an m×n all-one matrix;

[0015] Step 2.4.2: Align the center point heat map and detection box information with the element positions of the all-one matrix in step 2.4.1;

[0016] Step 2.4.3: Expand the long side and short side of the detection frame outward by half of the original length and width to obtain a new detection frame;

[0017] Step 2.4.4: Assign the matrix elements outside the new detection frame to 0.8 to obtain the detection information matrix at the current moment;

[0018] Step 2.4.5: Store the detection information matrix obtained in step 2.4.4 into the historical detection information library, which only stores the detection information matrices of the two most recent frames.

[0019] Furthermore, the step 3 specifically includes:

[0020] Step 3.1: Starting from the rth frame, read multiple frames of historical data and current frame data to construct a detection sample. The detection sample is a three-dimensional matrix with three dimensions: frame number dimension, orientation dimension, and distance dimension.

[0021] Step 3.2: Determine whether the current frame is the rth frame. If it is the rth frame, execute step 3.2.1. If it is not the rth frame, execute step 3.2.2.

[0022] Step 3.2.1: Input the detection sample into the network model constructed in step 2, where the frame dimension corresponds to the channel dimension of the network, and store the output center point heat map and detection box information into the historical detection information library constructed in step 2.4;

[0023] Step 3.2.2: Perform point multiplication on the detection sample and the detection information matrix at time t-1 in the historical detection information library to obtain a new sample, which is then input into the network model constructed in step 2, where the frame dimension corresponds to the channel dimension of the network, and the output center point heat map and detection box information are stored in the historical detection information library constructed in step 2.4;

[0024] Furthermore, the step 4 specifically includes:

[0025] Step 4.1: Calculate the motion state according to the center point heat map and rotation box parameters predicted in step 2, including the target's motion direction and speed;

[0026] Step 4.2: Update the target's position and motion status information, and output the target's tracking results.

[0027] On the other hand, the present invention provides a pre-detection tracking device based on radar video, which is characterized in that it includes:

[0028] The radar video information acquisition, storage and playback module acquires the video data of the current frame, stores and updates the video data of the adjacent multiple frames, and plays back the video data of the adjacent multiple frames;

[0029] The joint detection module reads the current frame and adjacent multi-frame video data, uses the rotation box detection network model based on the attention mechanism to perform joint detection on the continuous multi-frame data, and outputs the center point heat map and rotation box parameters;

[0030] The motion state calculation module includes the calculation of the target's motion direction and speed, as well as the update of the target's position and motion state information, and outputs the target tracking results.

[0031] The present invention discloses a pre-detection tracking method and device based on radar video, and the beneficial effects are as follows: the present invention utilizes radar video information to track the target, and can fully utilize the echo intensity and morphological information of the target during the tracking process to improve the tracking accuracy; the rotating frame detection network model proposed in the present invention can complete the target detection of the radar video frame, and based on this, complete the calculation of the target motion state, and has a strong engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic flow chart of a radar video-based pre-detection tracking method of the present invention;

[0033] Figure 2 This is a schematic diagram of continuous frames under a fixed field of view of the present invention;

[0034] Figure 3 The invention provides a rotating frame detection network model based on the attention mechanism;

[0035] Figure 4 Schematic diagram of the attention module of the present invention;

[0036] Figure 5 Schematic diagram of the detection results of the present invention DETAILED DESCRIPTION

[0037] In order to more clearly illustrate the technical solution of the present invention, the technical solution of the present invention is described in detail below in conjunction with the embodiments of the present invention and the accompanying drawings.

[0038] Figure 1 FIG. 1 is a flow chart of a radar video-based tracking method before detection according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0039] Step 1: In radar scanning mode, obtain and store multiple frames of radar video data. The video data format of a single frame is an m×n matrix, where m is the azimuth dimension, n is the distance dimension, and each element h ij represents the echo intensity of the i-th azimuth dimension and the j-th distance dimension, Figure 2 This is a schematic diagram of continuous frames under a fixed field of view in this embodiment;

[0040] Step 2: Build a rotation box detection network model based on attention mechanism. Figure 3 is the network model structure diagram, such as Figure 3 As shown, the construction of the network model includes:

[0041] Step 2.1: Construct a convolution module to extract features from multiple frames of data and generate feature maps. This embodiment provides a resnet network architecture;

[0042] Step 2.2: Construct an attention module, including spatial attention and channel attention, with the following structure: Figure 4 As shown, the feature map generated in step 2.1 is processed using the attention module, wherein the channel attention extracts the change trend between target frames and captures the motion characteristics of the target, and the spatial attention is used to extract the spatial position characteristics of the target single frame;

[0043] Step 2.2.1: In step 2.2, channel attention first performs maximum pooling and average pooling on each feature map

[0044]

[0045] After two fully connected layers, the channels are added together and the sigmoid function is used to get the channel attention matrix.

[0046]

[0047] Step 2.2.2: In step 2.2, spatial attention first performs maximum pooling and average pooling on each channel of the feature map.

[0048]

[0049] Then the two are concatenated in the channel dimension and convolved, and finally the sigmoid activation function is used to obtain the spatial attention matrix

[0050]

[0051] Step 2.2.3: In step 2.2, the channel attention matrix and spatial attention matrix obtained in step 2.2.1 are multiplied with the original feature map and then added to obtain the output of the attention module:

[0052] Fa =F×M c +F×M s (5)

[0053] Step 2.3: Construct a prediction module to predict the center point heat map and rotation box parameters;

[0054] Step 2.4: Build a historical detection information database;

[0055] Step 2.4.1: Create an m×n all-one matrix;

[0056] Step 2.4.2: Align the center point heat map and detection box information with the element positions of the all-one matrix in step 2.4.1;

[0057] Step 2.4.3: Expand the long side and short side of the detection frame outward by half of the original length and width to obtain a new detection frame;

[0058] Step 2.4.4: Assign the matrix elements outside the new detection frame to 0.8 to obtain the detection information matrix at the current moment;

[0059] Step 2.4.5: Store the detection information matrix obtained in step 2.4.4 into the historical detection information library, which only stores the detection information matrices of the two most recent frames.

[0060] Step 3: Read historical frames and current frame data to construct detection samples, and perform joint detection on multiple consecutive frames of data. Figure 5 It is a schematic diagram of the detection results, outputting the center point heat map and rotation box parameters after detection;

[0061] Step 3.1: Read multiple frames of historical data and current frame data, a total of r frames. This embodiment provides a choice of r as 4, and constructs a detection sample. The detection sample is three-dimensional data in the format of r×m×n. The three dimensions are frame number dimension, orientation dimension and distance dimension. Each element h ijk represents the echo intensity of the i-th azimuth, j-th range unit, and k-th frame;

[0062] Step 3.2: Determine whether the current frame is the rth frame. If it is the rth frame, execute step 3.2.1. If it is not the rth frame, execute step 3.2.2.

[0063] Step 3.2.1: Input the detection sample into the network model constructed in step 2, where the frame dimension corresponds to the channel dimension of the network, and store the output center point heat map and detection box information into the historical detection information library constructed in step 2.4;

[0064] Step 3.2.2: Perform point multiplication on the detection sample and the detection information matrix at time t-1 in the historical detection information library to obtain a new sample, which is then input into the network model constructed in step 2, where the frame dimension corresponds to the channel dimension of the network, and the output center point heat map and detection box information are stored in the historical detection information library constructed in step 2.4;

[0065] Step 4: Calculate the motion state of the target in the corresponding continuous frames based on the center point heat map and the rotation box parameters.

[0066] Step 4.1: Calculate the motion state based on the center point heat map and rotation box parameters predicted in step 2, including the target's motion direction and speed. In this example, r is 4, and the corresponding moments of consecutive frames are t, t-Δt, t-2Δt, and t-3Δt. θ is the target's motion direction, and the motion speed can be approximated as:

[0067]

[0068] Step 4.2: Update the target's position and motion status information, and output the target's tracking results.

Claims

1. A detection-before-tracking method based on radar video, characterized in that The following steps are involved: Step 1: In radar scanning mode, obtain and store the current frame radar video data; Step 2: Build a rotation box detection network model based on attention mechanism; Step 3: Read historical frames and current frame data to construct detection samples, perform joint detection on multiple consecutive frames of data, output the center point heat map and rotation box parameters after detection, and store them in the historical detection information library; Step 4: According to the center point heat map and the rotation box parameters, calculate the motion state of the target in the corresponding continuous frames and update the target tracking information.

2. The radar video-based tracking before detection method according to claim 1, characterized in that: The step 2 specifically includes: Step 2.1: Construct a convolution module to extract features from multiple frames of data and generate feature maps; Step 2.2: Construct an attention module, including spatial attention and channel attention, and use the attention module to process the feature map generated in step 2.

1. The channel attention extracts the change trend between target frames and captures the motion characteristics of the target. The spatial attention is used to extract the spatial position characteristics of the target in a single frame. Step 2.3: Construct a prediction module to predict the center point heat map and rotation box parameters; Step 2.4: Build a historical detection information database.

3. The radar video-based tracking before detection method according to claim 2, characterized in that: The specific steps of step 2.4 are: Step 2.4.1: Create an m×n all-one matrix; Step 2.4.2: Align the center point heat map and detection box information with the element positions of the all-one matrix in step 2.4.1; Step 2.4.3: Expand the long side and short side of the detection frame outward by half of the original length and width to obtain a new detection frame; Step 2.4.4: Assign the matrix elements outside the new detection frame range to 0.8 to obtain the detection information matrix at the current moment; Step 2.4.5: Store the detection information matrix obtained in step 2.4.4 into the historical detection information library, which only stores the detection information matrices of the two most recent frames.

4. The radar video-based tracking before detection method according to claim 1, characterized in that The step 3 specifically includes: Step 3.1: Starting from the rth frame, read multiple frames of historical data and current frame data to construct a detection sample. The detection sample is a three-dimensional matrix with three dimensions: frame number dimension, orientation dimension, and distance dimension. Step 3.2: Determine whether the current frame is the rth frame. If it is the rth frame, execute step 3.2.

1. If it is not the rth frame, execute step 3.2.

2. Step 3.2.1: Input the detection sample into the network model constructed in step 2, where the frame dimension corresponds to the channel dimension of the network, and store the output center point heat map and detection box information into the historical detection information library constructed in step 2.4; Step 3.2.2: Perform point multiplication on the detection sample and the detection information matrix at time t-1 in the historical detection information library. After obtaining the new sample, input it into the network model constructed in step 2, where the frame number dimension corresponds to the channel dimension of the network, and store the output center point heat map and detection box information in the historical detection information library constructed in step 2.

4.

5. The radar video-based tracking before detection method according to claim 1, characterized in that The step 4 specifically includes: Step 4.1: Calculate the motion state, including the target's motion direction and speed, based on the center point heat map and rotation box parameters predicted in step 2; Step 4.2: Update the target's position and motion status information, and output the target's tracking results.

6. A detection-pre-tracking device based on radar video, characterized in that include: The radar video information acquisition, storage and playback module acquires the video data of the current frame, stores and updates the video data of the adjacent multiple frames, and plays back the video data of the adjacent multiple frames; The joint detection module reads the current frame and adjacent multi-frame video data, uses the rotation box detection network model based on the attention mechanism to perform joint detection on the continuous multi-frame data, and outputs the center point heat map and rotation box parameters; The motion state calculation module includes the calculation of the target's motion direction and speed, as well as the update of the target's position and motion state information, and outputs the target tracking results.

Images