Tracking and recognition network integrated fusion system for SAR (Synthetic Aperture Radar) image

Abstract

The invention relates to a tracking and identification network integrated fusion system of an SAR (Synthetic Aperture Radar) image, which aims at solving the technical problems of poor single-frame identification effect, non-ideal detection generalization, easy loss of tracking and independent identification and tracking, adopts a technical scheme of weighting fusion tracking and identification outside a detection network, and utilizes the advantage of accurately identifying a target position by using a neural network. A detection position is used as an anchor point, a classical tracking method is embedded into neural network reasoning, a tracker does not depend on a scene, feature layer tracking is introduced on a small-scale feature layer, a cross-shaped position coding layer and double-path feedback are designed, and a large amount of data set training is not needed. The technical effects of improving the tracking and recognition robustness, improving the reasoning speed, improving the multi-target tracking and recognition effect, reducing the influence caused by translation and rotation of front and back frames of the image and improving recognition when the signal-to-noise of the SAR image is relatively low are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a technology for tracking and identifying targets using coding. Background Art

[0002] SAR imaging processing focuses on the scene. Due to the Doppler effect, the radar echo energy of moving targets is defocused to other Doppler units, appearing as shadow areas in the image. Various methods are used to detect and identify shadow features, such as traditional threshold detection, classifier cascades, and convolutional neural networks.

[0003] If the target's signal-to-noise ratio is poor, shadow features are less distinct, outlines are unclear, and it is difficult to distinguish it from the surrounding scene. Single-frame recognition performance is reduced, and the target detection rate is low over several consecutive frames. Due to the limitations of the signal-to-noise ratio, single-frame recognition is limited in effectiveness.

[0004] SAR datasets are not as rich as infrared datasets, the available targeted sample scenes are limited, and the detection generalization effect of different scenes is not ideal.

[0005] SAR image processing tracking methods, such as centroid and template matching filtering, suffer from poor signal-to-noise ratios, resulting in significant noise fluctuations in the same target pixel between frames. Even with correlation matching, target position extraction is difficult, tracking robustness is low, and targets are easily lost, especially when tracking multiple targets.

[0006] Recognition models based on convolutional neural networks typically target a single frame, are limited to image information, and do not exploit the motion correlation of targets across frames. Tracking is based on specific threshold features or target image templates, lacking global information. If the target is lost, it is difficult to recapture it.

[0007] In classic image processing, detection, recognition, and tracking are independent of each other. If the image signal-to-noise ratio is poor, the brightness, texture, and contour features of shadows are blurred, and the noise fluctuations of pixel values ​​between frames are large, which limits the effectiveness of tracking and recognition processing independently. Summary of the Invention

[0008] When the signal-to-noise ratio (SNR) of SAR imaging is low, a classic tracking algorithm alone may lose track of the target due to noise fluctuations within a few frames, rendering it unable to continue tracking. Independent neural networks are also susceptible to image noise in low SNR conditions. This can cause significant fluctuations in target detection confidence in frames where the human eye cannot distinguish them, leading to target loss below the detection threshold.

[0009] In order to solve the technical problems of poor single-frame recognition effect, unsatisfactory detection generalization, easy tracking loss, and independent recognition and tracking, a technical solution of weighted fusion tracking and recognition outside the detection network was adopted. The advantage of neural network in accurately identifying target position was utilized, and the detection position was used as an anchor point to embed the classic tracking method into neural network reasoning. The tracker does not depend on the scene, introduces feature layer tracking on the small-scale feature layer, designs a cross-type position encoding layer and dual-path feedback, and does not require a large amount of data set training. This has the technical effects of improving the robustness of tracking and recognition, improving the inference speed, improving the multi-target tracking effect, reducing the impact of translation and rotation of the previous and next frames of the image, and improving recognition when the SAR image signal-noise is low.

[0010] The system includes a replication unit, a backbone network, a feature layer, a task head unit, a post-processing unit, a position encoding layer, and a decoder, which constitute a neural network processing module. The system also includes a small-size tracker group, an encoder, and an image tracker group, which constitute a classic tracking processing module.

[0011] After preprocessing, the SAR image is input into the replication unit and outputs two paths, one of which is input into the backbone network and the other is input into the image tracker group.

[0012] The backbone network extracts features from the preprocessed SAR image, the position encoding layer generates position codes, and the decoder generates target position information, which are then input into the feature layer.

[0013] The feature layer uses deep channel convolution for weight fusion and outputs two paths, one of which is input to the task head unit and the other is input to the small-size tracker group.

[0014] The task head unit uses classification detection and box position regression to generate predictions of various scales, which are input into the post-processing unit and the small-size tracker group.

[0015] The small-scale tracker group determines whether the new target has been tracked, updates the detection box, inputs the encoder, generates a mask of the small-scale feature layer, inputs the decoder, and enhances the weight flow of the small-scale target position information.

[0016] The post-processing unit uses non-maximum suppression and image coordinate size transformation to generate multi-target classification and frame position information, which is input into the image tracker group. Combined with the pre-processed SAR image, it determines whether the new target has been tracked and outputs the multi-target position information.

[0017] Furthermore, the feature layer includes a small-scale feature layer, a medium-scale feature layer, and a large-scale feature layer. The backbone network generates features of different scales, which are input into the small-scale feature layer, the medium-scale feature layer, and the large-scale feature layer respectively.

[0018] The position encoding layer uses cross floating point encoding and inputs the small-scale feature layer, medium-scale feature layer, and large-scale feature layer respectively. The decoder inputs the small-scale feature layer, medium-scale feature layer, and large-scale feature layer respectively. The small-scale weights generated by the small-scale feature layer are input into the small-size tracker group.

[0019] Specifically, assuming that the pixel value input by the backbone network to the feature layer is h×w×c1, the feature layer to be encoded has a width of w, a height of h, and a depth of c1, and the position encoding layer has a width of w, a height of h, and a depth of 2. The first layer is encoded in the width direction, and the second layer is encoded in the height direction.

[0020] Assume that the w direction is the x-axis and the h direction is the y-axis, use [ ] to represent rounding, take x=[w / 2] and y=[h / 2] as the origin, and set the step size of the x-axis ws=1 / [w / 2] and the step size of the y-axis hs=1 / [h / 2]. Then the pixel value encoded in the width direction of the first layer is Wxy=x*ws-1, and the pixel value in the height direction of the second layer is Wxy=y*hs-1.

[0021] The pixel value input by the decoder to the feature layer is h×w×1, and the pixel value input by the position encoding layer to the feature layer is h×w×2, and the generated pixel value is h×w×(c1+3).

[0022] The small-size tracker group determines the new target detection box BoxD. If the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is greater than the preset threshold, the new target and the tracked target are determined to be the same target, and the new target fusion box is set to the weighted sum of BoxD and BoxT, that is, BoxR=sqrt(eta)*BoxD+(1-sqrt(eta))*BoxT, where BoxD, BoxT, and BoxR are functions of the box center coordinates x, y and the box width and length w, h; if the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is not greater than the preset threshold, it is determined to be a new target, the tracker is initialized, that is, BoxR=BoxD, and the encoder is input.

[0023] The channel dimension of the encoder is 1. According to the center coordinates x, y and width and length w, h of BoxR, BoxR is filled with a rectangular area of ​​1.0.

[0024] The image tracker group determines the new target detection box BoxD. If the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is greater than the preset threshold, the new target and the tracked target are determined to be the same target, and the new target fusion box is set to the weighted sum of BoxD and BoxT, that is, BoxR=sqrt(eta)*BoxD+(1-sqrt(eta))*BoxT, where BoxD, BoxT, and BoxR are functions of the box center coordinates x, y and the box width and length w, h; if the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is not greater than the preset threshold, it is determined to be a new target, the tracker is initialized, that is, BoxR=BoxD, and the multi-target position information is output. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the schematic diagram of the neural network processing module. Figure 2 This is the schematic diagram of the classic tracking processing module. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0027] Neural network processing modules such as Figure 1 As shown, it includes a replication unit 1, a backbone network 2, a small-scale feature layer 3s, a medium-scale feature layer 3m, a large-scale feature layer 3l, a task head unit 4, a post-processing unit 6, a position encoding layer 8, and a decoder 10.

[0028] The preprocessed SAR image is input into the image tracker group 7 and the backbone network 2 through the replication unit 1 for feature extraction, generating features of different scales, which are input into the small-scale feature layer 3s, the medium-scale feature layer 3m, and the large-scale feature layer 3l respectively.

[0029] The decoder 10 generates target position information and inputs it into the small-scale feature layer 3s, the medium-scale feature layer 3m, and the large-scale feature layer 3l respectively.

[0030] The position coding layer 8 uses a cross-type floating-point coding to generate position codes, which are input into the small-scale feature layer 3s, the medium-scale feature layer 3m, and the large-scale feature layer 3l respectively.

[0031] Assume that the pixel value input by backbone network 2 to the feature layer is h×w×c1, then the feature layer to be encoded has a width of w, a height of h, and a depth of c1, and the position encoding layer 8 has a width of w, a height of h, and a depth of 2. The first layer performs width encoding, and the second layer performs height encoding.

[0032] Assume that the w direction is the x-axis and the h direction is the y-axis, use [ ] to represent rounding, take x=[w / 2] and y=[h / 2] as the origin, and set the step size of the x-axis ws=1 / [w / 2] and the step size of the y-axis hs=1 / [h / 2]. Then the pixel value encoded in the width direction of the first layer is Wxy=x*ws-1, and the pixel value in the height direction of the second layer is Wxy=y*hs-1.

[0033] The pixel value input by the decoder 10 to the feature layer is h×w×1, and the pixel value input by the position encoding layer 8 to the feature layer is h×w×2. The feature layer adopts deep channel convolution and weight fusion, and the generated pixel value is h×w×(c1+3), which is input to the task head unit 4. The small-scale weight generated by the small-scale feature layer 3s is input to the small-size tracker group 15.

[0034] The task head unit 4 uses classification detection and box position regression to generate predictions 5 of each scale, which are input into the post-processing unit 6 and the small-size tracker group 15.

[0035] The post-processing unit 6 uses non-maximum suppression and image coordinate size transformation to generate multi-target classification frame position information 12, which is input into the image tracker group 7.

[0036] Classic tracking processing modules such as Figure 2 As shown, it includes a small-size tracker group 15, an encoder 13, and an image tracker group 7.

[0037] The small-size tracker group 15 determines the new target detection box BoxD. If the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is greater than a preset threshold, the new target and the tracked target are determined to be the same target, and the new target fusion box is set to the weighted sum of BoxD and BoxT, that is, BoxR=sqrt(eta)*BoxD+(1-sqrt(eta))*BoxT, where BoxD, BoxT, and BoxR are functions of the box center coordinates x, y and the box width and length w, h; if the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is not greater than a preset threshold, it is determined to be a new target, the tracker is initialized, that is, BoxR=BoxD, and the encoder 13 is input.

[0038] The channel dimension of the encoder 13 is 1. According to the box center coordinates x, y and the box width and length w, h of BoxR, BoxR is filled with a rectangular area of ​​1.0 to generate a mask 9 of the small-scale feature layer 3s, which is input into the decoder 10 of the neural network processing module to enhance the weight flow of small-scale target position information.

[0039] Image tracker group 7 determines the new target detection box BoxD. If the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is greater than a preset threshold, the new target and the tracked target are determined to be the same target, and the new target fusion box is set to the weighted sum of BoxD and BoxT, that is, BoxR=sqrt(eta)*BoxD+(1-sqrt(eta))*BoxT, where BoxD, BoxT, and BoxR are functions of the box center coordinates x, y and the box width and length w, h; if the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is not greater than a preset threshold, it is determined to be a new target, the tracker is initialized, that is, BoxR=BoxD, and the multi-target position information is output.

[0040] The above are embodiments of the present invention and do not limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims

1. A SAR image tracking and recognition network integrated fusion system, characterized by: include: The system includes a replication unit, a backbone network, a feature layer, a task head unit, a post-processing unit, a position encoding layer, and a decoder, forming a neural network processing module. The system also includes a small-size tracker group, an encoder, and an image tracker group, forming a classic tracking processing module. The SAR image is preprocessed and input into the replication unit, which outputs two paths: one input into the backbone network and the other input into the image tracker group. The backbone network extracts features from the preprocessed SAR image, the position encoding layer generates position codes, and the decoder generates target position information, all of which are input into the feature layer. The feature layer uses deep channel convolution for weight fusion and outputs two paths, one of which is input to the task head unit and the other to the small-size tracker group. The task head unit uses classification detection and box position regression to generate predictions at various scales, which are input to the post-processing unit and the small-size tracker group. The small-scale tracker group determines whether the new target has been tracked, updates the detection box, inputs the encoder, generates a mask of the small-scale feature layer, inputs the decoder, and enhances the weight flow of the small-scale target position information; The post-processing unit uses non-maximum suppression and image coordinate size transformation to generate multi-target classification and frame position information, which is input into the image tracker group. Combined with the pre-processed SAR image, it determines whether the new target has been tracked and outputs the multi-target position information.

2. The SAR image tracking and recognition network integrated fusion system according to claim 1, characterized in that: The feature layer includes a small-scale feature layer, a medium-scale feature layer, and a large-scale feature layer. The backbone network generates features of different scales, which are input into the small-scale feature layer, the medium-scale feature layer, and the large-scale feature layer respectively.

3. The SAR image tracking and recognition network integrated fusion system according to claim 2, characterized in that: The position encoding layer adopts cross floating point encoding and inputs the small-scale feature layer, the medium-scale feature layer and the large-scale feature layer respectively. The decoder inputs the small-scale feature layer, the medium-scale feature layer and the large-scale feature layer respectively. The small-scale weight generated by the small-scale feature layer is input into the small-size tracker group.

4. The SAR image tracking and recognition network integrated fusion system according to claim 1, characterized in that: Also includes: The pixel value input by the backbone network to the feature layer is h×w×c1. The feature layer to be encoded has a width of w, a height of h, and a depth of c1. The position encoding layer has a width of w, a height of h, and a depth of 2. The first layer is encoded in the width direction, and the second layer is encoded in the height direction.

5. The SAR image tracking and recognition network integrated fusion system according to claim 4 is characterized in that: It also includes: setting the w direction as the x-axis and the h direction as the y-axis, using [ ] to represent rounding, taking x=[w / 2] and y=[h / 2] as the origin, setting the x-axis step size ws=1 / [w / 2] and the y-axis step size hs=1 / [h / 2], then the pixel value encoded in the width direction of the first layer is Wxy=x*ws-1, and the pixel value in the height direction of the second layer is Wxy=y*hs-1.

6. The SAR image tracking and recognition network integrated fusion system according to claim 4, characterized in that: It also includes: the pixel value input by the decoder to the feature layer is h×w×1, the pixel value input by the position encoding layer to the feature layer is h×w×2, and the generated pixel value is h×w×(c1+3).

7. The SAR image tracking and recognition network integrated fusion system according to claim 4, characterized in that: Also includes: The channel dimension of the encoder is 1. According to the center coordinates x, y and width and length w, h of BoxR, BoxR is filled with a rectangular area of ​​1.

0.

8. The SAR image tracking and recognition network integrated fusion system according to claim 1, characterized in that: The small-size tracker group determines whether the new target has been tracked, including: if the intersection-over-union (IoU) of the new target detection box BoxD and the tracked target box BoxT is greater than a preset threshold, then the new target and the tracked target are determined to be the same target, and the new target fusion box is set to the weighted sum of BoxD and BoxT, that is, BoxR=sqrt(eta)*BoxD+(1-sqrt(eta))*BoxT, where BoxD, BoxT, BoxR are functions of the box center coordinates x, y and the box width and length w, h; if the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is not greater than the preset threshold, then it is determined to be a new target, the tracker is initialized, that is, BoxR=BoxD, and the encoder 13 is input.

9. The SAR image tracking and recognition network integrated fusion system according to claim 1, characterized in that: The image tracker group determines whether the new target has been tracked, including: if the intersection-over-union (IoU) of the new target detection box BoxD and the tracked target box BoxT is greater than a preset threshold, then the new target and the tracked target are determined to be the same target, and the new target fusion box is set to the weighted sum of BoxD and BoxT, that is, BoxR=sqrt(eta)*BoxD+(1-sqrt(eta))*BoxT, where BoxD, BoxT, and BoxR are functions of the box center coordinates x, y and the box width and length w, h; if the intersection-over-union (IoU) of BoxD and the tracked target box BoxT is not greater than a preset threshold, then it is determined to be a new target, the tracker is initialized, that is, BoxR=BoxD, and the multi-target position information is output.

Images