Cross-sensor-based SAR (Synthetic Aperture Radar) image target detection method, device, equipment and medium

Abstract

The invention relates to an SAR (Synthetic Aperture Radar) image target detection method, device, equipment and medium, and the method comprises the steps: extracting an SAR sample image from a source domain sample data set and a target domain sample data set which are obtained from targets of the same type through employing different sensors, so as to form a migration learning sample group, and inputting the migration learning sample group into a target detection neural network; and training the target domain SAR image through a dynamic feature adversarial discriminator and a perception center calibrator, and carrying out target detection on the target domain SAR image by using the trained target detection neural network so as to realize cross-sensor target detection. By adopting the method, the accuracy of identification and calibration can be enhanced, and the generalization performance of target detection is improved.

Description

Technical Field

[0001] The present application relates to the technical field of SAR image processing, and in particular to a method, device, equipment and medium for SAR image target detection based on cross-sensor. Background Art

[0002] Synthetic aperture radar has the characteristics of all-day and all-weather operation, and can acquire targets by actively emitting electromagnetic waves. It has been widely used in the field of earth observation and plays an important role in civilian fields such as ocean monitoring, disaster assessment, and resource management.

[0003] Traditional SAR target detection methods rely on the physical characteristics of the target, including geometric characteristics and polarization characteristics. Usually, the CFAR algorithm based on the constant false alarm rate is mainly used to detect SAR targets. This method uses the distribution of the target and background clutter to distinguish the target from the background. However, the detection accuracy of this method is not high and it is prone to high false alarms. In recent years, as deep learning has made great achievements in the field of remote sensing image interpretation, deep networks have also been gradually applied to the field of SAR target detection and recognition, and have made certain breakthroughs in the fields of aircraft, ships and vehicles, achieving excellent performance.

[0004] However, most of the existing research on SAR target detection focuses on the situation where the training and test data are distributed in the same way, that is, the training and test data come from the same sensor. With the increasing number of SAR imaging platforms in recent years, the actual operating conditions are often characterized by different distributions of training and test data and large differences in sensor conditions, which brings great challenges to SAR target detection and recognition. Different SAR imaging platforms have different technical indicators and system parameters, such as radar frequency, beam width, orbit altitude, observation angle, etc. In addition, SAR data under different imaging conditions present different probability distributions, and the image style and target features of the obtained data are quite different. Therefore, when the model trained under one imaging condition is applied to data under other imaging conditions, the performance will be significantly reduced, which leads to poor cross-sensor generalization of SAR target detection. A common method is to annotate the data under new imaging conditions and then retrain. However, since the interpretation and judgment of SAR images is difficult, it requires certain expert knowledge, and the annotation is time-consuming and labor-intensive, and it is difficult to perform accurate and efficient annotation. Therefore, how to generalize the detection model trained under existing imaging conditions to data under other imaging conditions is an urgent problem to be solved. Summary of the invention

[0005] Based on this, it is necessary to provide a cross-sensor SAR image target detection method, device, equipment and medium that can significantly improve the generalization capability in response to the above technical problems.

[0006] A cross-sensor SAR image target detection method, the method comprising:

[0007] Acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels;

[0008] Extracting a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and constructing a corresponding transfer learning training set;

[0009] Using each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the target detection capability for the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0010] The trained target detection neural network is used to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

[0011] In one embodiment, when training the target detection neural network:

[0012] A cross-domain shared feature extractor in the target detection neural network is used to perform multi-level feature extraction on a source domain SAR sample image and a target domain SAR sample image in a group of transfer learning samples to obtain a multi-level source domain feature image and a target domain feature image;

[0013] In the dynamic feature adversarial discriminator, adaptive dynamic weights are respectively assigned to the multi-level source domain feature images and the target domain feature images, and after multi-scale enhancement is performed on the feature images after the dynamic weight assignment using the first feature pyramid, the dynamic adversarial discrimination loss is calculated using the first feature discriminator;

[0014] In the perceptual center calibrator, after performing multi-scale enhancement according to the multi-level source domain feature image and the target domain feature image respectively using the second feature pyramid, a domain classification output image and a center output image are obtained using a full convolutional network, and a perceptual center calibration loss is calculated using the second feature discriminator, and the category output loss, the center loss and the position regression loss are calculated at the same time;

[0015] A total loss function is calculated based on the dynamic adversarial discrimination loss, perceptual center calibration loss, category output loss, center loss and position regression loss. The adjustable parameters in the target detection network are adjusted according to the total loss function until convergence, thereby obtaining the trained target detection neural network.

[0016] In one embodiment, in the dynamic feature adversarial discriminator, when adaptive dynamic weight allocation is performed on the multi-level source domain feature image and the target domain feature image respectively:

[0017] For a certain level of source domain or target domain feature image, encode along two directions of the feature map to obtain a pair of direction-aware feature maps, and then concatenate the pair of direction-aware feature maps to obtain a concatenated feature map;

[0018] A convolutional layer is used to perform dimension reduction and spatial segmentation operations on the concatenated feature map to obtain feature map weights in two directions;

[0019] The feature map weights obtained at the current level are adaptively weighted to the feature map at the previous level to obtain the feature map after dynamic weight allocation at the previous level.

[0020] In one embodiment, the method of performing multi-scale enhancement on the feature image after dynamic weight assignment by using the first feature pyramid and then using the first feature discriminator to calculate the dynamic adversarial discrimination loss includes:

[0021] After performing multi-scale enhancement on the feature images after dynamic weight allocation in the source domain and the target domain using the first feature pyramid, multi-scale enhanced feature images of the source domain and the target domain are obtained;

[0022] A feature detector is used to output domain classification maps of multi-scale source domain and target domain enhanced feature images respectively;

[0023] For the same scale, the domain classification maps obtained from the enhanced feature images of the source and target domains are used as a group to calculate the dynamic adversarial discrimination loss;

[0024] According to the dynamic adversarial discrimination losses corresponding to all scales, the total dynamic adversarial discrimination loss is obtained.

[0025] In one embodiment, the centrality output image represents a distance score from a network attention center on the image.

[0026] In one embodiment, when the second feature discriminator is used to calculate the perceptual center calibration loss:

[0027] For the same scale, for the source domain and the target domain respectively, the center classification map is calculated based on the domain classification output image and the center output image;

[0028] Multiplying the central classification map with the corresponding enhanced feature image to obtain a perceptual central feature map;

[0029] For the same scale, the perceptual center feature maps obtained from the source domain and the target domain are taken as a group to calculate the perceptual center calibration loss;

[0030] According to the perceptual center calibration losses corresponding to all scales, the total perceptual center calibration loss is obtained.

[0031] In one embodiment, the trained object detection neural network includes the cross-domain shared feature extractor, a second feature pyramid and a detection head.

[0032] The present application also provides a SAR image target detection device based on cross-sensor, the device comprising:

[0033] A migration training sample acquisition module is used to acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels;

[0034] A sample training group construction module is used to extract a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and to construct a corresponding transfer learning training set;

[0035] A transfer learning training module, used to use each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, and perform transfer learning training on the target detection neural network from the source domain to the target domain, so that the trained target detection neural network has the target detection capability of the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0036] The cross-sensor target detection module is used to perform target detection on the target domain SAR image using the trained target detection neural network to achieve cross-sensor target detection.

[0037] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels;

[0039] Extracting a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and constructing a corresponding transfer learning training set;

[0040] Using each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the target detection capability for the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0041] The trained target detection neural network is used to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0043] Acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels;

[0044] Extracting a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and constructing a corresponding transfer learning training set;

[0045] Using each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the target detection capability for the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0046] The trained target detection neural network is used to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

[0047] The above-mentioned cross-sensor-based SAR image target detection method, device, equipment and medium respectively extract a SAR sample image from the source domain sample data set and the target domain sample data set obtained by using different sensors for the same category of targets to form a group of transfer learning sample groups, and input them into the target detection neural network, and train them through a dynamic feature adversarial discriminator and a perception center calibrator, and use the trained target detection neural network to perform target detection on the target domain SAR image to achieve cross-sensor target detection. The use of this method can enhance the accuracy of identification and calibration and improve the generalization performance of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 1 is a flow chart of a method for detecting a target in a SAR image based on a cross-sensor sensor in one embodiment;

[0049] Figure 2 is a schematic diagram of an adaptive dynamic weight allocation mechanism in one embodiment;

[0050] Figure 3 A schematic diagram of the structure of a dynamic feature adversarial discriminator in one embodiment;

[0051] Figure 4 is a schematic diagram of calculating a centrality image according to an embodiment;

[0052] Figure 5 A schematic diagram of a portion of the structure of a sensing center calibrator in one embodiment;

[0053] Figure 6 is a structural block diagram of a cross-sensor SAR image target detection device in one embodiment;

[0054] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] In the existing methods of SAR target detection using neural networks, the characteristics of SAR targets and the characteristics of SAR images are not considered, and they cannot be directly applied to cross-sensor SAR target detection. On the one hand, SAR data is difficult to interpret, which appears as coherent speckle noise on the image, and the feature transfer in the network appears as the transfer of scattering information. Single-layer feature alignment will cause the loss of scattering information, resulting in inaccurate feature alignment. The relative weights of the network feature maps of the existing domain adaptation feature alignment methods are fixed, and weights cannot be dynamically allocated according to the style characteristics and key features of the SAR image. They are easily affected by irrelevant noise, resulting in inaccurate feature identification. On the other hand, the SAR target appears as a combination of target scattering centers, and its target structure presents alternating light and dark scattering blocks. The instance-level feature alignment of the existing methods pays more attention to the features within the candidate box. However, within the candidate box, there are certain background samples around the target. Aligning the features within the candidate box is easily affected by background negative samples, which will generate a large amount of redundant information and misaligned features. This method cannot focus on the strong scattering center of the SAR target itself, resulting in inaccurate positioning. In this application, such as Figure 1 As shown, a cross-sensor SAR image target detection method is provided, which specifically includes the following steps:

[0057] Step S100, obtaining a source domain sample dataset and a target domain sample dataset, wherein the source domain sample dataset and the target domain sample dataset are SAR images obtained by using different sensors for targets of the same category, and the source domain sample dataset includes SAR sample images and corresponding labels.

[0058] Step S110: extract a SAR sample image from the source domain sample data set and the target domain sample data set respectively to form a group of transfer learning sample groups, and construct a corresponding transfer learning training set.

[0059] Step S120, using each group of transfer learning sample groups in the transfer learning training set to be input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the ability to detect targets in the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator.

[0060] Step S130, using the trained target detection neural network to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

[0061] In this application, a SAR target detection method with dynamic adversarial identification and center-aware calibration across imaging conditions is proposed. By designing a dynamic feature adversarial discriminator and a perceptual center calibrator, feature map weights are dynamically selected and allocated, and the focus is on the alignment of target scattering centers. This reduces the misalignment caused by negative samples, enhances the accuracy of identification and calibration, and improves the generalization performance of target detection.

[0062] In step S100, for the SAR target detection task across imaging sensors, the target detection model trained in the source domain is generalized to the target domain. Here, it is assumed that the source domain sample dataset with complete annotations is represented as: The source domain sample data set includes N s samples, represents the source domain samples, Represents the category label. The unlabeled target domain data is represented as: The purpose of this method is to obtain the target domain data D by aligning the feature migration between the source domain and the target domain. t Tags and candidate boxes

[0063] In this embodiment, when training the target detection neural network for transfer learning, a dynamic feature adversarial discriminator and a perception center calibrator are emphasized. Among them, the dynamic feature adversarial discriminator adaptively and dynamically allocates weights to feature maps of different scales to identify key features at high and low levels, reduce the representation of other redundant features, and make up for the loss of scattered information caused by the increase in network dimension. Secondly, the perception center calibrator is used to focus on the perception center of the target, enhance the representation of the strong scattering structure of the target, and realize feature calibration at the center of the receptive field. It can avoid the influence of background negative samples in the candidate frame, reduce the misalignment caused by negative samples, and improve the accuracy of feature calibration.

[0064] Specifically, when training the target detection neural network: a cross-domain shared feature extractor in the target detection neural network is used to perform multi-level feature extraction on a source domain SAR sample image and a target domain SAR sample image in a group of transfer learning samples, respectively, to obtain a multi-level source domain feature image and a target domain feature image. In the dynamic feature adversarial discriminator, adaptive dynamic weights are assigned to the multi-level source domain feature image and the target domain feature image, and the first feature pyramid is used to perform multi-scale enhancement on the feature image after dynamic weight assignment, and then the first feature discriminator is used to calculate the dynamic adversarial identification loss. In the perception center calibrator, the second feature pyramid is used to perform multi-scale enhancement on the multi-level source domain feature image and the target domain feature image, and then the full convolution network is used to obtain the domain classification output image and the center output image, and the second feature discriminator is used to calculate the perception center calibration loss, and the category output loss, center loss and position regression loss are calculated at the same time. A total loss function is calculated based on dynamic adversarial discrimination loss, perceptual center calibration loss, category output loss, center loss and position regression loss. The adjustable parameters in the target detection network are adjusted according to the total loss function until convergence to obtain a trained target detection neural network.

[0065] In step S110, a SAR sample image is extracted from each of the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and a transfer learning training set is constructed by multiple groups of such transfer learning sample groups. During the iterative training of the target detection neural network, a group of transfer learning sample groups is output each time, and multiple losses are calculated based on the outputs of the dynamic feature adversarial discriminator and the perception center calibrator. These different loss functions are used to collaboratively train the network, so that the neural network has the ability to perform accurate target detection in the target domain under the guidance of various loss functions.

[0066] Next, we introduce the dynamic feature adversarial discriminator and the perceptual center calibrator respectively.

[0067] In order to reduce the difference in image styles under different sensor conditions, compensate for the loss of target scattering information and insufficient feature alignment caused by the increase in the number of network layers, and adaptively adjust the focus on key areas, a dynamic feature adversarial identification module is proposed. By designing a multi-scale identification method, a connection is established between feature maps of different scales, and multi-scale information is better utilized. Features are aligned at high and low levels respectively, which enhances the identification of SAR target detail features and grayscale style and scattering style, reduces the loss of SAR target feature information caused by the increase in network dimensions, and improves the accuracy of feature alignment. In addition, a dynamic attention mechanism is added to different feature layers to adaptively adjust the weights of different scales, focus on key area features in different feature maps, and suppress information interference caused by redundant information and areas with unclear feature responses.

[0068] First, the fully annotated source domain data and target domain data Enter a cross-domain shared feature extractor G, and the extracted features can be expressed as:

[0069]

[0070] Assume that G i The characteristics of the layer are In order to improve the efficiency of feature alignment, reduce the impact of redundant features, and improve the accuracy of feature adaptation and migration, at the feature layer G i and P i That is, an adaptive dynamic weight allocation mechanism is introduced between the cross-domain shared feature extractor G and the first feature pyramid, so that the network pays more attention to the key regional features of the feature map and dynamically selects appropriate weights according to the importance of different regions in the image. This is conducive to selecting more critical and more discriminative features, and avoids the problems of poor adaptability and feature alignment redundancy caused by the traditional convolutional layer's indiscriminate weighting of the entire image.

[0071] In this embodiment, in the dynamic feature adversarial discriminator, when adaptive dynamic weight allocation is performed on multi-level source domain feature images and target domain feature images respectively: for a certain level of source domain or target domain feature images, encoding is performed along two directions of the feature map to obtain a pair of direction-aware feature maps, and the pair of direction-aware feature maps are spliced ​​to obtain a spliced ​​feature map, and the spliced ​​feature map is subjected to dimensionality reduction and spatial segmentation operations using a convolutional layer to obtain feature map weights in two directions, and the feature map weights obtained at the current level are adaptively weighted to the feature map of the previous level to obtain the feature map of the previous level after dynamic weight allocation.

[0072] Specifically, the adaptive dynamic weight allocation mechanism decomposes the global pooling into two-directional sub-pooling, encoding the channels along the X and Y directions of the feature map respectively. Therefore, the feature map output of the c channel at h can be expressed as Where h and i represent the height and width. The feature map output of the c channel at w can be expressed as Among them, j and w represent the height and width. The feature map output in two directions can be expressed as:

[0073]

[0074] Next, the two transformations are aggregated along two spatial directions to generate a pair of direction-aware feature maps. These two transformations help the network capture long-range dependencies along different spatial directions and better preserve precise location information, which helps the network locate the target more accurately and adaptively and dynamically assign weights to different regions of the feature map.

[0075] Then, the two pooled feature maps in different directions are spliced ​​to obtain the spliced ​​feature map z h+w , expressed as:

[0076] z h+w =concat(z h ,z w ')

[0077] Furthermore, a 1x1 convolution kernel is used to transform z h+w Then, perform a split operation on f and divide it into f along the x and y directions. h and f w , and use a 1x1 convolution operation F h and F w The feature maps are dimensioned separately to obtain the final feature map weight g h and g w , respectively expressed as:

[0078] g h =σ(F h (f h ))

[0079] g w =σ(F w (f w ))

[0080] In the above formula, σ is the sigmoid activation function.

[0081] Finally, the feature map weights in the x-direction and y-direction obtained at the current level are multiplied by the feature map output by the previous level to obtain the feature map after dynamic weight allocation. The process can be expressed as:

[0082]

[0083] The adaptive dynamic weight allocation mechanism aggregates features in the horizontal and vertical directions through one-dimensional pooling operations to generate two direction-aware feature maps containing position information. These two directional feature maps are combined and embedded with global information to generate dynamic weights in two directions, which act on the horizontal and vertical directions of the input feature map respectively. Finally, the weights of the feature maps are dynamically adjusted according to the different feature information in the two directions to achieve adaptive channel information weighting.

[0084] like Figure 2 As shown, the entire adaptive dynamic weight allocation mechanism is shown.

[0085] like Figure 3 As shown in the figure, it is the structure of the dynamic adversarial discriminator. It can be seen from the figure that the feature images output by the cross-domain shared feature extractor G3 layer, G4 layer and G5 layer are respectively weighted by using the ADW, i.e. adaptive dynamic weight allocation mechanism, and then the feature image output by the previous layer is weighted, and then the feature image after dynamic weight allocation is input into the first feature pyramid P 1 -P 5 Layer, feature enhancement is performed, that is, G3, G4, and G5 are respectively subjected to ADW to obtain G3', G4', and G5', P3 is G5', P2 is equal to P3 upsampling plus G4', and P1 is equal to P2 upsampling plus G3', so as to enhance the target scattering feature information in the network, and obtain feature maps of different scales:

[0086] Furthermore, after using the first feature pyramid to perform multi-scale enhancement on the feature image after dynamic weight allocation, the first feature discriminator is used to calculate the dynamic adversarial identification loss, including: using the first feature pyramid to perform multi-scale enhancement on the feature images after dynamic weight allocation in the source domain and the target domain respectively, to obtain multi-scale source domain and target domain enhanced feature images, using a feature detector to output domain classification maps of the multi-scale source domain and target domain enhanced feature images respectively, for the same scale, the domain classification maps obtained according to the source domain and target domain enhanced feature images are used as a group to calculate the dynamic adversarial identification loss, and the total dynamic adversarial identification loss is obtained according to the dynamic adversarial identification losses corresponding to all scales.

[0087] Specifically, the enhanced feature maps of different scales are Input the first feature discriminator D 1 , realizing domain classification and adaptive dynamic feature alignment of enhanced feature maps. The first feature discriminator D 1 The output is the enhanced feature map The domain classification map of the same size, each position represents the domain label corresponding to the same position on the enhanced feature map. The domain labels y of the source domain and the target domain are s and t are set to 1 and 0 respectively, expressed as the following formula, and the discrimination is optimized by minimizing the binary cross entropy loss.

[0088]

[0089] In this embodiment, the first feature identifier D 1 Including 4 convolutional layers and 1 GRL gradient reverse layer, the specific composition structure of the discriminator is shown in Table 1.

[0090] Table 1 Discriminator structure

[0091] Layer Specific setting 1 Gradient Reversal Layer(GRL) 2 Conv 256x3x3, stride 1 3 GroupNorm,ReLU 4 Conv 256x3x3, stride 1 5 GroupNorm,ReLU 6 Conv 256x3x3, stride 1 7 GroupNorm,ReLU 8 Conv 256x3x3, stride 1 9 GroupNorm,ReLU

[0092] Since the first feature pyramid downsampling of the dynamic feature adversarial discriminator will lose a lot of semantic feature information of the SAR target, it will lead to inaccurate adversarial identification. Therefore, in order to compensate for the lost scattering information, a multi-scale feature identification method is adopted in this method. Feature maps of different scales are input into the discriminator to reduce the loss of semantic information, thereby obtaining a multi-level identification loss, which more comprehensively characterizes the feature differences between the source domain and the target domain. Among them, y represents the domain label of the source domain, and Represent the enhanced feature maps of the source domain and the target domain at multiple scales, and (m,n) represents different positions on the enhanced feature map. i The adversarial discrimination loss obtained in the first feature discriminator corresponding to the layer input is expressed as Then the total dynamic feature adversarial identification loss is obtained as L DFAD The loss of the discriminator is minimized by the formula, and the feature extractor is optimized by maximizing the above formula to achieve the purpose of adversarial identification. Among them, the adversarial identification loss is calculated using the following formula:

[0093]

[0094] The total dynamic adversarial discrimination loss L is obtained DFAD It can be expressed as:

[0095]

[0096] In the above formula, K is the number of different scales.

[0097] Furthermore, in order to avoid the influence of background negative samples in the target frame in instance-level feature alignment, reduce the misalignment caused by negative samples around the target, make the network pay more attention to the strong scattering structure and strong response center of the SAR target, and enhance the accuracy of feature calibration, a perception center feature calibrator is designed. Because the anchor frame contains negative samples and background clutter information, a certain amount of redundant information is generated in the network, which makes the source domain and target domain feature alignment inaccurate and there is a large error in positioning. By enhancing the saliency of the strong scattering area of ​​the SAR target, highlighting the perception center of the target, separating the target area from the background area, making the strong scattering structure information of the target more obvious, focusing more on the distinctive target area, and suppressing the generation of false alarms. The perception center feature calibrator focuses on the local perception center feature of the target, highlights the saliency of the target, and the feature vector is closer to the target class center. The target center perception feature alignment is realized on the feature map, and the features of the strongest area of ​​the target scattering structure can be learned. In addition, calibrating the perception center at different scales can more effectively calibrate the target features under different sensor conditions.

[0098] In this embodiment, similar to the dynamic feature adversarial discriminator, the perception center feature calibrator includes a second feature pyramid and a second feature discriminator. After the multi-level feature image is extracted by the cross-domain shared feature extractor, in order to enhance the representation of the scattered information in the network, it is directly input into the second feature pyramid to extract enhanced feature images of different scales. And these enhanced feature maps Passed to two different 4-layer fully convolutional networks, the domain classification output graphs are obtained respectively. The output is [0,1], and the centrality output graph Where C is the number of categories. Each position on the domain classification output map and the centrality output map represents the corresponding domain classification result and centrality score, respectively. In addition, the target category output loss L is also obtained. cls , center loss L cts and position regression loss L reg Among them, L cls Focal loss is used for optimization, L cts Use binary cross entropy loss to optimize the centrality, L reg The IOU loss was used for optimization.

[0099] L cls =-α t (1-p t ) r log(p t )

[0100] In the above formula, p t is the predicted probability of the sample, α tIt is a parameter to adjust the weights of positive and negative samples.

[0101] L cts =-[ylog(p cts )+(1-y)log(1-p cts )]

[0102] In the above formula, y is the true label (0 or 1), p cts is the centralness value predicted by the model.

[0103]

[0104] In the above formula, S gt and S pre are the areas enclosed by the true value coordinates and the predicted coordinates respectively.

[0105] In this embodiment, the centerness output image (centerness map) represents the distance score from the network attention center on the image. Figure 4 As shown in the figure, assuming that the pixel position is (m, n), L, r, t, b represent the distance to the left, right, top and bottom of the target anchor point respectively. When the anchor point is farther away from the annotation center, the lower the centerness score is, and when the anchor point is closer to the annotation center, the higher the centerness score is, so as to determine the weight of each part of the target from the center, and to distinguish and judge the target and background areas. The score value of the target area will be very high, and the score value of the background area will be very low, which reduces the influence of the background area to a certain extent. This operation can effectively reduce the influence of background noise, distinguish the background and target parts, and improve the effectiveness of target alignment.

[0106] Specifically, the center degree of each position (m, n) in the center degree output image is calculated using the following formula:

[0107]

[0108] In this embodiment, M cls (class map) can show the domain classification results at different positions on the image, classify different pixel areas on the image, ignore areas with domain classification errors, remove false alarms and false targets such as trees, buildings, and metals, so as to obtain the image area more accurately and efficiently, and then align the target area. The value range of the domain classification result is [0,1].

[0109] In this embodiment, when the second feature identifier is used to calculate the perceptual center calibration loss: for the same scale, for the source domain and the target domain respectively, the center classification map is calculated according to the domain classification output image and the center output image, and the center classification map is multiplied with the corresponding enhanced feature image to obtain the perceptual center feature map. For the same scale, the perceptual center feature maps obtained in the source domain and the target domain are taken as a group to calculate the perceptual center calibration loss, and the total dynamic perceptual center calibration loss is obtained according to the perceptual center calibration losses corresponding to all scales.

[0110] Specifically, the center degree output image is multiplied and combined with the domain classification output image to obtain the center classification image M cc , where the center output image M ces It can highlight the center position of the target, highlight the strong response area of ​​the target in the SAR image, and model the strength of the target structure according to the response center. The potential position of the target, according to the domain classification centrality weight, can obtain a better representation of the area with domain invariant characteristics, highlight the key feature area, make the network more focused on the strong scattering area of ​​the target and the area with shared features, calibrate the aligned features, and focus on the center of the target.

[0111] Furthermore, when the center output image is merged with the domain classification output image, the process is expressed as:

[0112] M c ' ls =maxpool(M cls )

[0113] M cc =σ(σM cls ⊙σM ces )

[0114] In the above formula, σ represents the sigmoid layer.

[0115] Since the weights on the feature map of the network can highlight the key areas that the network focuses on, the central classification map M of the source domain and the target domain is cc And the feature map F of the corresponding domain s and F t Multiply to get the perceptual center feature map of the source domain and the target domain and It is expressed as:

[0116]

[0117] The perceptual center feature map result represents the classification weight scores of different areas on the map from the target center. The higher the weight score, the more critical the feature, the greater the scattering intensity, and the richer the scattering information contained. It can better characterize the scattering structure and texture characteristics of the target, focus on the local changes and distribution inside the target, highlight the central response area of ​​the target, and further characterize the domain-invariant features of the source domain and the target domain, weaken the influence of the background area of ​​the negative sample, and enhance the significance of the target with weaker scattering intensity.

[0118] The perceptual center feature maps of different scales in the source domain and the target domain are and After passing through the GRL layer, the input feature discriminator realizes the perceptual center calibration, and the perceptual center calibration loss of different scales is obtained. It is expressed as:

[0119]

[0120] Furthermore, the perceptual center calibration losses of feature maps of different scales are combined, and the sum of the perceptual center calibration losses can be expressed as:

[0121]

[0122] like Figure 5 The figure shows a schematic diagram of data processing in the perception center calibrator.

[0123] Considering the different sizes of different targets, the image features obtained under different sensor conditions are different. Therefore, the discrimination loss under a single scale cannot fully realize feature discrimination and it is difficult to truly reflect the difference between the target domain and the source domain. The dynamic feature discrimination loss and the center perception loss obtained at different scales are combined to capture targets of different sizes and improve the adaptability and generalization of feature discrimination and calibration. In addition, the classification loss L obtained by classifying and regressing source domain data is cls , position regression loss L reg and the center loss L ces It is also calculated in the total loss. Among them, λ and μ are the weights of dynamic feature adversarial identification loss and perceptual center calibration loss respectively. The total loss function is expressed as:

[0124] L total =L cls +L reg +L ces +λL DFAD +μL CAFC

[0125] In the above formula, λ and μ are both set to 0.5.

[0126] In this embodiment, the trained object detection neural network includes the cross-domain shared feature extractor, the second feature pyramid and the detection head.

[0127] In this embodiment, an algorithm for implementing the above training process is also provided, as shown below:

[0128]

[0129] In order to verify the performance of the proposed method, the cross-sensor domain adaptation SAR image target detection method proposed in this paper is compared with the traditional Faster RCNN and 5 SOTA DA target detection methods DAF, EPM, HTCN, I3Net, SFA. For the Faster RCNN baseline network, in order to have the same initial settings as other domain adaptation target detection methods, the labeled source domain dataset is used as the training set, and the trained model is tested on the target domain dataset. The five SOTA methods compared are currently the most classic and well-performing methods in the SAR field and computer vision field. When performing experimental performance comparisons, the abbreviations of the comparison methods are used for ease of presentation.

[0130] The cross-sensor detection tasks are divided into four types, namely, the adaptation of FARAD Ka data to MiniSAR data (Table 2), the adaptation of MiniSAR data to FARAD Ka data (Table 3), the adaptation of FARAD X data to MiniSAR data (Table 4), and the adaptation of MiniSAR data to FARAD X data (Table 5). Taking FARAD Ka data to MiniSAR data as an example, the training data is labeled FARAD Ka data and unlabeled MiniSAR data, and the test data is MiniSAR data. This task is recorded as Ka->MiniSAR. By analogy, four tasks are obtained: Ka->MiniSAR, MiniSAR->Ka, X->MiniSAR, and MiniSAR->X. The generalization performance of the detection model is evaluated from the four indicators of Precision, Recall, F1, and mAP. The detection results of the four tasks are shown in Tables 2 to 5.

[0131] In these four tasks, the detection performance of Faster-RCNN is less than 35%, and the generalization performance is poor. In the cross-sensor detection task from Ka to miniSAR, as shown in Table 2, the mAP of the proposed method can reach 68.71%, which is about 7% higher than the second EPM algorithm and more than 10% higher than other algorithms. In addition, the Precision and Recall results of the proposed method are 76.45% and 62.92% respectively. The F1 result is 69.03%. These three indicators also achieved the best results among all algorithms.

[0132] In the cross-sensor detection task from X to miniSAR, as shown in Table 4, the mAP and F1 of the proposed method can reach 68.51% and 73.16% respectively. The precision and recall of the proposed algorithm also have obvious advantages over other algorithms. The HTCN algorithm is a method of image migration using CycleGAN, but it is only about 44% in the cross-sensor detection task from X to miniSAR. The reason for this may be that the HTCN algorithm only migrates the style of the entire image, while ignoring the alignment of the scattering characteristics and details of the target itself, resulting in low performance. The SFA algorithm is an algorithm that combines transformers for cross-domain migration, and its mAP is also reduced by nearly 20% compared with the proposed algorithm. The reason for this may be that the performance of the transformer network depends on the amount of data, and the more and richer the data, the better its performance. However, the SAR data in this article itself has a small amount of data, and the interpretation of SAR data is difficult, which leads to poor performance of the mAP algorithm.

[0133] In addition, in the MiniSAR to Ka and MiniSAR to X tasks, the proposed method also achieved good results, which is more advantageous than other methods. In the MiniSAR to Ka and MiniSAR to X tasks, the detection performance of various algorithms is much lower than that of the forward adaptation task. The reason for this may be that the number of targets in the MiniSAR data in the dataset is less than that in the FARAD data, resulting in insufficient migration. In addition, most of the vehicle targets in the FARAD data are in urban areas, with more target shapes, different postures, more complex scenes, and greater impact from background clutter. In the MiniSAR data, the vehicle targets are in more open areas in the suburbs and are less affected by background clutter such as buildings and trees. Therefore, the test results of the MiniSAR data are better than those of the FARAD data.

[0134] From the analysis of the results, the mAP of the cross-sensor detection task between Ka data and MiniSAR data is 68.71% and 55.12% respectively. The mAP of the cross-sensor detection between X data and MiniSAR data is 68.51% and 62.56% respectively. From the results, it can be seen that the migration adaptation effect of X data and MiniSAR data is better, which also indirectly shows that compared with Ka data, X data and MiniSAR data are more similar at the feature level, the image style is more similar, and the feature migration is more sufficient. Overall, the proposed method can achieve better performance advantages than other algorithms, which also verifies the effectiveness of the method in this paper.

[0135] Table 2

[0136]

[0137] Table 3

[0138]

[0139] Table 4

[0140]

[0141] Table 5

[0142]

[0143] In the above-mentioned cross-sensor SAR image target detection method, a cross-sensor SAR target detection method based on dynamic adversarial discrimination and perception center calibration is proposed to solve the problem of poor generalization and adaptability of SAR target detection models under the condition that training data and test data come from different sensors. On the one hand, a dynamic adversarial discriminator is designed to dynamically select and assign weights according to the correlation of pixels in the feature map to avoid the influence of irrelevant noise. At the same time, a multi-scale structure is designed to integrate high-level and low-level features to make up for the lost scattering information in the network. And enhance the identification of SAR target texture details and SAR image abstract semantic features at low-dimensional and high-dimensional levels. On the other hand, a perception center calibrator is designed to enhance the representation of the strong scattering center of the target in the network and accurately capture the center characteristics of the target. The perception center calibrator combines the center map with the domain classification map to highlight the perception center area of ​​the SAR target, ignore the background negative samples of the candidate box, and avoid the misalignment caused by negative samples. It can enhance the representation of the invariant features of the target domain and realize the calibration of the perception center.

[0144] It should be understood that although Figure 1The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0145] In one embodiment, Figure 6 As shown, a cross-sensor SAR image target detection device is provided, comprising: a migration training sample acquisition module 200, a sample training group construction module 210, a migration learning training module 220 and a cross-sensor target detection module 230, wherein:

[0146] The migration training sample acquisition module 200 is used to acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for the same category of targets, and the source domain sample data set includes SAR sample images and corresponding labels;

[0147] A sample training group construction module 210 is used to extract a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and to construct a corresponding transfer learning training set;

[0148] A transfer learning training module 220 is used to use each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, and perform transfer learning training on the target detection neural network from the source domain to the target domain, so that the trained target detection neural network has the target detection capability of the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0149] The cross-sensor target detection module 230 is used to perform target detection on the target domain SAR image using the trained target detection neural network to achieve cross-sensor target detection.

[0150] For the specific definition of the cross-sensor based SAR image target detection device, please refer to the definition of the cross-sensor based SAR image target detection method above, which will not be repeated here. Each module in the above-mentioned cross-sensor based SAR image target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0151] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a SAR image target detection method based on cross-sensor is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0152] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0154] Acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels;

[0155] Extracting a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and constructing a corresponding transfer learning training set;

[0156] Using each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the target detection capability for the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0157] The trained target detection neural network is used to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

[0158] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0159] Acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels;

[0160] Extracting a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and constructing a corresponding transfer learning training set;

[0161] Using each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the target detection capability for the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator;

[0162] The trained target detection neural network is used to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

[0163] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0164] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A cross-sensor SAR image target detection method, characterized in that: The method comprises: Acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by using different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels; Extracting a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and constructing a corresponding transfer learning training set; Using each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, the target detection neural network is trained for transfer learning from the source domain to the target domain, so that the trained target detection neural network has the target detection capability for the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator; The trained target detection neural network is used to perform target detection on the target domain SAR image to achieve cross-sensor target detection.

2. The SAR image target detection method according to claim 1, characterized in that: When training the target detection neural network: A cross-domain shared feature extractor in the target detection neural network is used to perform multi-level feature extraction on a source domain SAR sample image and a target domain SAR sample image in a group of transfer learning samples to obtain a multi-level source domain feature image and a target domain feature image; In the dynamic feature adversarial discriminator, adaptive dynamic weights are respectively assigned to the multi-level source domain feature images and the target domain feature images, and after multi-scale enhancement is performed on the feature images after the dynamic weight assignment using the first feature pyramid, the dynamic adversarial discrimination loss is calculated using the first feature discriminator; In the perceptual center calibrator, after performing multi-scale enhancement according to the multi-level source domain feature image and the target domain feature image respectively using the second feature pyramid, a domain classification output image and a center output image are obtained using a full convolutional network, and a perceptual center calibration loss is calculated using the second feature discriminator, and the category output loss, the center loss and the position regression loss are calculated at the same time; A total loss function is calculated based on the dynamic adversarial discrimination loss, perceptual center calibration loss, category output loss, center loss and position regression loss. The adjustable parameters in the target detection network are adjusted according to the total loss function until convergence, thereby obtaining the trained target detection neural network.

3. The SAR image target detection method according to claim 2, characterized in that: In the dynamic feature adversarial discriminator, when adaptive dynamic weight allocation is performed on the multi-level source domain feature image and the target domain feature image respectively: For a certain level of source domain or target domain feature image, encode along two directions of the feature map to obtain a pair of direction-aware feature maps, and then concatenate the pair of direction-aware feature maps to obtain a concatenated feature map; A convolutional layer is used to perform dimension reduction and spatial segmentation operations on the concatenated feature map to obtain feature map weights in two directions; The feature map weights obtained at the current level are adaptively weighted to the feature map at the previous level to obtain the feature map after dynamic weight allocation at the previous level.

4. The SAR image target detection method according to claim 3, characterized in that: After performing multi-scale enhancement on the feature image after dynamic weight allocation by using the first feature pyramid, and then using the first feature discriminator to calculate the dynamic adversarial discrimination loss includes: After performing multi-scale enhancement on the feature images after dynamic weight allocation in the source domain and the target domain using the first feature pyramid, multi-scale enhanced feature images of the source domain and the target domain are obtained; A feature detector is used to output domain classification maps of multi-scale source domain and target domain enhanced feature images respectively; For the same scale, the domain classification maps obtained from the enhanced feature images of the source and target domains are used as a group to calculate the dynamic adversarial discrimination loss; According to the dynamic adversarial discrimination losses corresponding to all scales, the total dynamic adversarial discrimination loss is obtained.

5. The SAR image target detection method according to claim 2, characterized in that: The center output image represents the distance score from the network's attention center on the image.

6. The SAR image target detection method according to claim 5, characterized in that: When calculating the perceptual center calibration loss using the second feature discriminator: For the same scale, for the source domain and the target domain respectively, the center classification map is calculated based on the domain classification output image and the center output image; Multiplying the central classification map with the corresponding enhanced feature image to obtain a perceptual central feature map; For the same scale, the perceptual center feature maps obtained from the source domain and the target domain are taken as a group to calculate the perceptual center calibration loss; According to the perceptual center calibration losses corresponding to all scales, the total perceptual center calibration loss is obtained.

7. The SAR image target detection method according to any one of claims 2 to 6, characterized in that: The trained target detection neural network includes the cross-domain shared feature extractor, a second feature pyramid and a detection head.

8. A cross-sensor SAR image target detection device, characterized in that: The device comprises: A migration training sample acquisition module is used to acquire a source domain sample data set and a target domain sample data set, wherein the source domain sample data set and the target domain sample data set are SAR images acquired by different sensors for targets of the same category, and the source domain sample data set includes SAR sample images and corresponding labels; A sample training group construction module is used to extract a SAR sample image from the source domain sample data set and the target domain sample data set to form a group of transfer learning sample groups, and to construct a corresponding transfer learning training set; A transfer learning training module, used to use each group of transfer learning sample groups in the transfer learning training set to input into the target detection neural network, and perform transfer learning training on the target detection neural network from the source domain to the target domain, so that the trained target detection neural network has the target detection capability of the target domain SAR image, wherein the target detection neural network is trained by a dynamic feature adversarial discriminator and a perception center calibrator; The cross-sensor target detection module is used to perform target detection on the target domain SAR image using the trained target detection neural network to achieve cross-sensor target detection.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to claim 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 7 are implemented.

Images