Cross-Sensor SAR Image Target Detection Method, Device, Equipment and Medium

Abstract

The present application relates to a cross-sensor SAR image target detection method, device, equipment and medium. By inputting the SAR source domain image with true value labels and the SAR target domain image without true value labels into a multi-layer shared feature extractor, corresponding semantic feature maps are obtained respectively, and corresponding graph structures are constructed based on these feature maps. By constructing a loss function to constrain the consistency of the source domain semantic graph structure and the target domain semantic graph structure during training, the trained network has good generalization ability on the target domain. At the same time, combining target detection constraints and global feature constraints, accurate detection of targets in the target domain SAR image can be achieved under the training of the target domain image without true value labels.

Description

Technical Field

[0001] This application relates to the technical field of SAR target detection, and particularly to a cross-sensor SAR image target detection method, device, equipment, and medium. Background Art

[0002] Synthetic Aperture Radar (SAR) has the ability to work day and night and is not affected by weather. It captures target information by actively emitting electromagnetic waves and has been widely used in the field of earth observation. In civilian fields such as marine monitoring, disaster assessment, and resource management, SAR plays an indispensable role.

[0003] Traditional SAR target detection methods mainly rely on the physical characteristics of targets, such as geometric shapes and polarization characteristics. Among them, the Constant False Alarm Rate (CFAR) - based algorithm is a commonly used target detection method, which identifies targets by analyzing the distribution differences between targets and background clutter. However, the detection accuracy of this method is limited and it is prone to generating a high false alarm rate. In recent years, deep learning technology has made remarkable progress in the field of remote sensing image analysis. Deep neural networks have gradually been applied to SAR target detection and recognition tasks and have achieved breakthrough results in fields such as airplanes, ships, and vehicles, demonstrating excellent performance.

[0004] However, most current SAR target detection studies assume that the training data and test data come from the same distribution, that is, data obtained from the same sensor. With the continuous increase in SAR imaging platforms, in practical applications, the training data and test data often come from different sensors, resulting in differences in data distribution. This difference mainly stems from the differences in technical indicators and system parameters of different SAR platforms, such as radar frequency, beam width, orbital altitude, and observation angle. In addition, SAR data under different imaging conditions have different probability distributions, and there are also significant differences in image styles and target features. Therefore, when a model trained under a specific imaging condition is applied to data under other conditions, its performance will drop significantly, indicating that the generalization ability of SAR target detection under cross - sensor conditions is poor.

[0005] A common solution is to annotate the data under the new imaging condition and then retrain the model. However, since the interpretation of SAR images requires professional knowledge, the annotation process is both time - consuming and laborious, making it difficult to achieve efficient and accurate annotation. Therefore, how to enable a detection model trained under one imaging condition to successfully generalize to data under other conditions has become an urgent problem to be solved. Summary of the Invention

[0006] Based on this, it is necessary to provide a cross-sensor SAR image target detection method, device, equipment and medium that can achieve cross-domain accurate detection for the above technical problems.

[0007] A cross-sensor SAR image target detection method, the method comprising:

[0008] Obtain a source domain dataset and a target domain dataset, where the source domain dataset includes multiple SAR source domain images with ground truth labels, and the target domain dataset includes multiple SAR target domain images without ground truth labels;

[0009] Extract one SAR image from each of the source domain dataset and the target domain dataset as a training pair, input the training pair into a multi-layer shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and construct corresponding semantic scattering map structures according to the source domain semantic feature map and the target domain semantic feature map respectively, to obtain a source domain semantic scattering map structure and a target domain semantic scattering map structure;

[0010] Construct a scattering structure map alignment loss function according to the source domain semantic scattering map structure and the target domain semantic scattering map structure;

[0011] Use a discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map;

[0012] Use a detection head to perform target detection according to the source domain semantic feature map and the target domain semantic feature map respectively, and construct a detection loss function;

[0013] Use the scattering structure map alignment loss function, the global feature alignment loss function and the detection loss function to adjust the adjustable parameters in the multi-layer shared feature extractor and the detection head until convergence, and construct a cross-domain target detection network according to the trained multi-layer shared feature extractor and the detection head;

[0014] Obtain a SAR image to be detected in the target domain, and use the cross-domain target detection network to perform target detection on the SAR image to be detected to obtain the target category and the target location.

[0015] In one embodiment, the constructing corresponding semantic scattering map structures according to the source domain semantic feature map and the target domain semantic feature map includes:

[0016] For the source domain semantic feature map, use the ground truth boxes in the corresponding ground truth labels to perform spatially uniform sampling of pixel points at different feature layers to obtain a source domain semantic scattering point set;

[0017] For the semantic feature map of the target domain, use the detection head to generate the class scores of each pixel point, and based on the class scores, obtain the set of semantic scattering points of the target source domain;

[0018] Map the feature points in the source domain semantic scattering point set and the target source domain semantic scattering point set to a low-dimensional vector space respectively, convert them into graph nodes, and construct the corresponding semantic scattering graph structure.

[0019] In one embodiment, the construction of the scattering graph structure alignment loss function according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure includes:

[0020] Calculate the semantic feature point alignment loss function according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure;

[0021] For each graph node in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure respectively, use the feature distribution of the graph node and the nearest neighbor algorithm to perform node enhancement, and obtain the source domain node enhanced graph structure and the target domain node enhanced graph structure;

[0022] After interacting the source domain node enhanced graph structure and the target domain node enhanced graph structure, calculate the semantic scattering point classification loss function;

[0023] Construct a semantic similarity graph structure matching loss function according to the maximum node similarity, minimum node similarity and edge similarity of the matching node pairs in the source domain node enhanced graph structure and the target domain node enhanced graph structure;

[0024] Obtain the scattering graph structure alignment loss function according to the semantic feature point alignment loss function, the semantic scattering point classification loss function and the semantic similarity graph structure matching loss function.

[0025] In one embodiment, the semantic feature point alignment loss function is expressed as:

[0026] ;

[0027] In the above formula, represents the node alignment discriminator network, represents the th graph node in the source domain semantic scattering graph structure, represents the th graph node in the target domain semantic scattering graph structure, represents the domain classification label, 、 respectively represent the number of graph nodes in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure.

[0028] In one embodiment, when enhancing each graph node in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure:

[0029] Model the distribution of graph nodes in the semantic scattering graph structure as a Gaussian distribution, sample feature points from this Gaussian distribution, and then obtain enhanced semantic scattering nodes according to the linear projection formula, and construct a first set of enhanced semantic scattering graph nodes;

[0030] By taking the union of the graph nodes in the original semantic scattering graph node set and the first set of enhanced semantic scattering graph nodes, obtain a second set of enhanced semantic scattering graph nodes;

[0031] Based on the original semantic scattering graph node set, obtain the corresponding adjacency matrix through inner product multiplication, and use a graph convolutional network combined with the adjacency matrix to enhance each graph node in the second set of enhanced semantic scattering graph nodes according to the features of neighboring nodes, obtaining a third set of enhanced semantic scattering graph nodes;

[0032] Perform a clustering operation on the graph nodes in the third set of enhanced semantic scattering graph nodes using the K-nearest neighbor algorithm to obtain the final node-enhanced graph structure.

[0033] In one embodiment, before constructing the first set of enhanced semantic scattering graph nodes, for the target domain semantic scattering graph structure, also enhance the features of the graph nodes in the target domain semantic scattering graph structure using the distribution variance of the graph structure in the target domain semantic scattering graph structure and the mean value of the source domain semantic scattering graph structure.

[0034] The present application also provides a cross-sensor SAR image target detection device, and the device includes:

[0035] A data set acquisition module, configured to acquire a source domain data set and a target domain data set, where the source domain data set includes multiple SAR source domain images with ground truth labels, and the target domain data set includes multiple SAR target domain images without ground truth labels;

[0036] A graph structure construction module, configured to extract one SAR image from the source domain data set and the target domain data set respectively as a training pair, input the training pair into a multi-layer shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and construct corresponding semantic scattering graph structures according to the source domain semantic feature map and the target domain semantic feature map respectively, obtaining a source domain semantic scattering graph structure and a target domain semantic scattering graph structure;

[0037] A graph structure alignment constraint module, configured to construct a scattering graph structure alignment loss function according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure;

[0038] A global feature alignment constraint module for constructing a multi-level global feature alignment loss function by using a discriminator based on the source domain semantic feature map and the target domain semantic feature map;

[0039] A target detection constraint module for performing target detection respectively based on the source domain semantic feature map and the target domain semantic feature map by using a detection head and constructing a detection loss function;

[0040] A training module for adjusting the tunable parameters in the multi-level shared feature extractor and the detection head by using the scattering structure diagram for its loss function, the global feature alignment loss function and the detection loss function until convergence, and constructing a cross-domain target detection network based on the trained multi-level shared feature extractor and the detection head;

[0041] A cross-domain detection module for obtaining a SAR image to be detected in the target domain, and performing target detection on the SAR image to be detected by using the cross-domain target detection network to obtain the target category and the target location.

[0042] A computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0043] Obtain a source domain data set and a target domain data set, the source domain data set includes multiple SAR source domain images with ground truth labels, and the target domain data set includes multiple SAR target domain images without ground truth labels;

[0044] Extract a SAR image from the source domain data set and the target domain data set respectively as a training pair, input the training pair into the multi-level shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and construct corresponding semantic scattering diagram structures according to the source domain semantic feature map and the target domain semantic feature map to obtain a source domain semantic scattering diagram structure and a target domain semantic scattering diagram structure;

[0045] Construct a scattering structure diagram alignment loss function according to the source domain semantic scattering diagram structure and the target domain semantic scattering diagram structure;

[0046] Use a discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map;

[0047] Use a detection head to perform target detection respectively based on the source domain semantic feature map and the target domain semantic feature map and construct a detection loss function;

[0048] Adjust the adjustable parameters in the multi - layer shared feature extractor and the detection head using the loss function of the scattering structure diagram, the global feature alignment loss function, and the detection loss function until convergence, and construct a cross - domain object detection network based on the trained multi - layer shared feature extractor and the detection head;

[0049] Obtain the SAR image to be detected in the target domain, and perform object detection on the SAR image to be detected using the cross - domain object detection network to obtain the object category and the object location.

[0050] A computer - readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0051] Obtain a source - domain dataset and a target - domain dataset. The source - domain dataset includes multiple SAR source - domain images with ground - truth labels, and the target - domain dataset includes multiple SAR target - domain images without ground - truth labels;

[0052] Extract one SAR image from the source - domain dataset and the target - domain dataset respectively as a training pair, input the training pair into the multi - layer shared feature extractor to obtain the source - domain semantic feature map and the target - domain semantic feature map respectively, and construct corresponding semantic scattering diagram structures according to the source - domain semantic feature map and the target - domain semantic feature map to obtain the source - domain semantic scattering diagram structure and the target - domain semantic scattering diagram structure;

[0053] Construct a scattering structure diagram alignment loss function according to the source - domain semantic scattering diagram structure and the target - domain semantic scattering diagram structure;

[0054] Use a discriminator to construct a multi - level global feature alignment loss function according to the source - domain semantic feature map and the target - domain semantic feature map;

[0055] Use a detection head to perform object detection according to the source - domain semantic feature map and the target - domain semantic feature map respectively, and construct a detection loss function;

[0056] Adjust the adjustable parameters in the multi - layer shared feature extractor and the detection head using the loss function of the scattering structure diagram, the global feature alignment loss function, and the detection loss function until convergence, and construct a cross - domain object detection network based on the trained multi - layer shared feature extractor and the detection head;

[0057] Obtain the SAR image to be detected in the target domain, and perform object detection on the SAR image to be detected using the cross - domain object detection network to obtain the object category and the object location.

[0058] The above cross-sensor SAR image target detection method, device, equipment, and medium input the SAR source domain image with ground truth labels and the SAR target domain image without ground truth labels into a multi-layer shared feature extractor to obtain corresponding semantic feature maps respectively, and construct corresponding graph structures based on these feature maps. By constructing a loss function to constrain the consistency of the source domain semantic graph structure and the target domain semantic graph structure during training, the trained network has good generalization ability on the target domain. At the same time, combining target detection constraints and global feature constraints, accurate detection of targets in the target domain SAR image can be achieved under the training of the target domain image without ground truth labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a schematic flowchart of the cross-sensor SAR image target detection method in one embodiment;

[0060] Figure 2 It is a schematic flowchart of constructing a semantic scattering graph in one embodiment;

[0061] Figure 3 It is a schematic diagram of enhancing nodes of a semantic scattering graph in one embodiment;

[0062] Figure 4 It is a schematic diagram of perceptual association of a semantic scattering graph in another embodiment;

[0063] Figure 5 It is a schematic diagram of aligning a semantic scattering graph structure in another embodiment;

[0064] Figure 6 It is a structural block diagram of the cross-sensor SAR image target detection device in one embodiment;

[0065] Figure 7 It is an internal structure diagram of a computer device in one embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] Regarding the existing cross-sensor SAR image detection methods, data obtained under new imaging conditions (i.e., another sensor) need to be labeled and then the model needs to be retrained. However, due to the fact that the interpretation of SAR images requires professional knowledge, the labeling process is both time-consuming and laborious, and it is difficult to achieve efficient and accurate labeling. In the present application, as Figure 1 shown, a cross-sensor SAR image target detection method is provided, which specifically includes the following steps:

[0068] Step S100: Obtain the source domain dataset and the target domain dataset. The source domain dataset includes multiple SAR source domain images with ground truth labels, and the target domain dataset includes multiple SAR target domain images without ground truth labels.

[0069] Step S110: Extract one SAR image from the source domain dataset and the target domain dataset respectively as a training pair, input the training pair into the multi-layer shared feature extractor to obtain the source domain semantic feature map and the target domain semantic feature map respectively, and construct corresponding semantic scattering map structures according to the source domain semantic feature map and the target domain semantic feature map, so as to obtain the source domain semantic scattering map structure and the target domain semantic scattering map structure.

[0070] Step S120: Construct a scattering structure map alignment loss function according to the source domain semantic scattering map structure and the target domain semantic scattering map structure.

[0071] Step S130: Use the discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map.

[0072] Step S140: Use the detection head to perform object detection according to the source domain semantic feature map and the target domain semantic feature map respectively, and construct a detection loss function.

[0073] Step S150: Use the scattering structure map alignment loss function, the global feature alignment loss function and the detection loss function to adjust the tunable parameters in the multi-layer shared feature extractor and the detection head until convergence, and construct a cross-domain object detection network according to the trained multi-layer shared feature extractor and the detection head;

[0074] Step S160: Obtain the SAR image to be detected in the target domain, and use the cross-domain object detection network to perform object detection on the SAR image to be detected to obtain the object category and the object location.

[0075] For the SAR object detection task across imaging sensors, the detection model trained on the source domain needs to be generalized to the target domain. Assume that the source domain data with complete ground truth labels is , where the source domain contains samples, represents the source domain samples, represents the class labels. The target domain data without ground truth labels is , which contains samples. The purpose of the proposed method is to obtain the labels and the candidate boxes of the target domain data through feature transfer alignment between the source domain and the target domain.

[0076] In step S100, a source domain dataset and a target domain dataset are respectively constructed, and part of the data in the target domain dataset is used for training and part for testing. The source domain dataset and the target domain dataset are respectively obtained by two different sensors detecting the same target scene. Due to the differences in sensors, the detection angles and poses are different, so the contents of the source domain SAR images and the target domain SAR images obtained are not the same.

[0077] In step S110, a set of training pairs is input into the multi-layer shared feature extractor to extract multi-level SAR feature maps, and constraints of different dimensions are respectively constructed during the training process according to the SAR feature maps, so that the detection model trained on the source domain can be generalized to the target domain.

[0078] In this embodiment, a training method based on graph structure alignment is mainly proposed. First, corresponding semantic scattering graph structures are respectively constructed according to the source domain semantic feature map and the target domain semantic feature map. The process is as Figure 2 shown, including: for the source domain semantic feature map, spatial uniform sampling of pixel points is performed at different feature layers using the ground truth boxes in the corresponding ground truth labels to obtain the source domain semantic scattering point set. For the target domain semantic feature map, the detection head is used to generate the class score for each pixel point, and based on the class score, the target source domain semantic scattering point set is obtained. Then, the feature points in the source domain semantic scattering point set and the target source domain semantic scattering point set are respectively mapped into the low-dimensional vector space, converted into graph nodes, and the corresponding semantic scattering graph structures are constructed.

[0079] In this embodiment, a bidirectional spatial aggregation mechanism is added to the multi-layer shared feature extractor and the multi-layer shared feature extractor is used to extract the high-dimensional semantic features of the target in the source domain and the target domain to obtain the feature maps of the source domain and the target domain;

[0080] .

[0081] Specifically, for the source domain semantic feature map, spatial uniform sampling is performed at different feature layers, the pixels within the source domain ground truth box are sampled to obtain the target semantic scattering nodes, and the pixels outside the target box are collected as background semantic scattering nodes at a ratio of 1:2, and then the source domain semantic scattering point set obtained is , expressed as: .

[0082] Specifically, for the target domain image, the detection head is used to generate the class score for each feature scattering pixel point, the points greater than the threshold are used as the target feature points, and from the pixel points with lower scores, background semantic scattering nodes are sampled at a sampling rate of 1:2, and target semantic scattering nodes are dynamically sampled from the pixel points with higher scores, and the One target domain semantic scattering point is used to construct the set of target source domain semantic scattering points , which is expressed as: .

[0083] Furthermore, in order to achieve fine alignment of the target in domain adaptation, the high-dimensional semantic scattering feature points are converted into graph nodes, and edges are constructed between each node. The target domain and the source domain are modeled into graph structures, and the source domain semantic scattering graph structure and the target domain semantic scattering graph structure are obtained. The two graph structures are respectively expressed as:

[0084] ;

[0085] ;

[0086] In the above formula, , , where , respectively represent the set of nodes in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure. The edge connection between nodes can be expressed as , , where the superscripts i and j respectively represent different nodes.

[0087] In this embodiment, the semantic scattering nodes are mapped to a low-dimensional vector space to retain the graph structure information and semantic node information, improve the effectiveness of subsequent feature alignment, and achieve adaptation and generalization of the target scattering information.

[0088] In step S120, according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure, the scattering graph structure alignment loss function is constructed as follows: First, according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure, the semantic feature point alignment loss function is calculated. Then, for each graph node in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure, the node enhancement is performed using the feature distribution of the graph node and the nearest neighbor algorithm to obtain the source domain node enhanced graph structure and the target domain node enhanced graph structure. After the interaction between the source domain node enhanced graph structure and the target domain node enhanced graph structure, the semantic scattering point classification loss function is calculated. Finally, according to the maximum node similarity, minimum node similarity, and edge similarity of the matching node pairs in the source domain node enhanced graph structure and the target domain node enhanced graph structure, the semantic similarity graph structure matching loss function is constructed. According to the semantic feature point alignment loss function, the semantic scattering point classification loss function, and the semantic similarity graph structure matching loss function, the scattering graph structure alignment loss function is obtained.

[0089] Specifically, in order to align the nodes in the source domain and the target domain, a semantic feature point alignment loss function is designed, expressed as:

[0090] ;

[0091] In the above formula, represents the node alignment discriminator network, represents the th graph node in the source domain semantic scattering graph structure, represents the th graph node in the target domain semantic scattering graph structure, represents the domain classification label, , respectively represent the number of graph nodes in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure.

[0092] Furthermore, since the semantic scattering points obtained by randomly sampling the image will miss some key scattering point features, resulting in incomplete semantic information, thus affecting the effect of SAR target cross-sensor domain adaptation detection. In order to further enhance the semantic representation of the source domain and the target domain, node information of different domains is generated according to the sampled graph node distribution. Since there are significant differences in the variance information of different domains and it will change during the training process, the variance of the opposing domain is used to ensure the similarity of the node distribution in the semantic space and improve the effect of domain adaptation.

[0093] In this embodiment, when enhancing each graph node in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure: the distribution of the graph nodes in the semantic scattering graph structure is modeled as a Gaussian distribution, feature points are sampled from this Gaussian distribution, and then the enhanced semantic scattering nodes are obtained according to the linear projection formula, and the first enhanced semantic scattering graph node set is constructed. By taking the union of the graph nodes in the original semantic scattering graph node set and the first enhanced semantic scattering graph node set, the second enhanced semantic scattering graph node set is obtained. Based on the original semantic scattering graph node set, the corresponding adjacency matrix is obtained by inner product multiplication. The graph convolution network is used to enhance each graph node in the second enhanced semantic scattering graph node set according to the features of the neighboring nodes, and the third enhanced semantic scattering graph node set is obtained. The K-nearest neighbor algorithm is used for clustering operation on the graph nodes in the third enhanced semantic scattering graph node set to obtain the final node-enhanced graph structure.

[0094] Among them, before constructing the first enhanced semantic scattering graph node set, for the target domain semantic scattering graph structure, the distribution variance of the graph structure in the target domain semantic scattering graph structure and the mean value Enhance the features of the graph nodes in the semantic scattering graph structure of the target domain to reduce the distribution difference between the target domain and the source domain.

[0095] Specifically, model the distributions of the graph nodes in the semantic scattering graph structures of the source domain and the target domain as Gaussian distributions, where and represent feature points sampled from the Gaussian distribution, and through linear projection obtain the enhanced semantic scattering nodes, generate the enhanced scattering point structure graph, thereby further enhancing the representation ability of the semantic scattering points and reducing the problem of uneven distribution of scattering points caused by differences in sampling methods. The enhanced semantic scattering node sets of the source domain and the target domain, namely the first enhanced semantic scattering graph node sets, can be expressed as:

[0096] ;

[0097] ;

[0098] In the above formula, and respectively follow Gaussian distributions and , , respectively represent the first enhanced semantic scattering graph node sets of the source domain and the target domain.

[0099] Then, take the union of the original semantic scattering graph node sets and the first enhanced semantic scattering graph node sets to obtain the second enhanced semantic scattering graph node sets, where and represent the original semantic scattering graph node sets of the source domain and the target domain, and the second enhanced semantic scattering graph node sets are respectively expressed as:

[0100] ;

[0101] ;

[0102] Among them, the linear projection operation is used to map the sampled feature points to the target space and can be expressed as:

[0103] ;

[0104] In the above formula, W and b are weight coefficients.

[0105] Furthermore, construct adjacency matrices for the second enhanced semantic scattering graph node sets of the source domain and the target domain respectively, and obtain the adjacency matrices that can represent the internal node association information through inner product multiplication, expressed as:

[0106] ;

[0107] ;

[0108] In the above formula, and respectively represent the adjacent matrices in a single graph structure of the source domain and the target domain, which are used to represent the encoded structure information. and is a learnable weight matrix.

[0109] In this embodiment, graph convolution is used for message propagation, and a single-layer GCN layer is used for long-distance semantic transmission to aggregate cross-image semantic knowledge, thereby enhancing the representation of nodes. The graph convolution operation can be expressed as:

[0110] ;

[0111] In the above formula, is the node feature matrix of the th layer, is the node feature matrix of the th layer, is a learnable weight matrix, is the adjacency matrix, is 's degree matrix, is the activation function.

[0112] Furthermore, in the graph convolution operation, the network aggregates the information of neighboring nodes through the product of the adjacency matrix of the nodes and the node feature matrix, enhances the features of the current node with the neighboring nodes, and obtains the nodes that have been aggregated through graph convolution respectively, that is, the third enhanced semantic scattering graph node set and , which are respectively expressed as:

[0113] ;

[0114] ;

[0115] In the above formula, and represent and 's neighboring nodes, and represent the elements in the adjacency matrix, is a learnable parameter.

[0116] Further, map the semantic scatter point map with the completed semantic graph structure onto the established graph structure, and use the KNN clustering method to cluster different scatter points of the target, so that the locally connected semantic scatter points in the graph are similar, and then obtain different semantic scatter components of the target. Cluster the positive sample nodes of the target to obtain the semantic scatter components of the target. Semantic scatter component clustering can capture the structured information of the target in the SAR image, enhance the network's understanding of the target structure, thereby improving the accuracy of semantic alignment and capturing the hidden structure in the graph structure.

[0117] As Figure 3 shown, it is a schematic diagram of the process of enhancing the nodes of the semantic scatter graph.

[0118] In order to narrow the feature distance between the source domain and the target and reduce the feature distribution difference, establish a semantic interaction correlation relationship between the source domain and the target domain between the two constructed graphs, and achieve feature alignment by interacting and fusing the information between the two domains, so that each semantic scatter node has the ability of cross-domain perception, and then learn the semantic information of the other domain, as Figure 4 shown.

[0119] Interact the scatter information of the target domain with the semantic scatter information of the source domain itself, and the set of source domain semantic scatter nodes after interaction is:

[0120] ;

[0121] Interact the scatter information of the source domain with the semantic scatter information of the target domain itself, and the set of target domain semantic scatter nodes after interaction is:

[0122] ;

[0123] In the above two formulas, and represent the enhanced set of semantic scatter nodes, , and are the learned weight matrices.

[0124] Further, use linear transformation and normalization operations to update the node representation, update the obtained nodes, and make the scatter information of different domains interact to narrow the domain distribution difference. Finally, obtain the semantic scatter point classification loss , which is expressed as:

[0125] ;

[0126] In the above formula, and are the true classification labels of the source domain and target domain images respectively, Represents the classification branch of the detection head, the classifier.

[0127] Furthermore, by learning the inherent semantic relationship between two graph nodes in the source domain and the target domain, encoding it as an associated representation, and achieving domain adaptation through node-to-node matching, that is, constructing a semantic scattering structure graph alignment loss function, the process is as Figure 4 shown.

[0128] First, construct a similarity matrix to measure the semantic similarity between scattering nodes in the graph. Specifically, project the input node feature matrix, where the node feature matrix is the enhanced graph structure. Map the features to a new space, and then use the cosine similarity to calculate the semantic feature similarity between the feature vectors of two nodes and and encode it as an element of the node similarity matrix , expressed as:

[0129] ;

[0130] Traverse the node pairs between the two domains, combine the similarity values of all node pairs into a matrix form to measure the semantic similarity between nodes in the source domain and the target domain graph, and obtain the final similarity matrix , expressed as:

[0131] ;

[0132] Construct a connection indication matrix between the two graph structures in the source domain and the target domain, where the elements are If two nodes belong to the same category, assign the element of the connection indication matrix to 1, otherwise assign 0, expressed as:

[0133] ;

[0134] For each source domain node , find the node in the target domain that matches it, which can maximize the similarity between them, enhance the similarity of correctly matched node pairs, and ensure that the similarity of the node pairs is as close to 1 as possible, obtaining the maximum node similarity loss , expressed as:

[0135] ;

[0136] In the above formula, represents the Hadamard product.

[0137] Next, traverse each node in the source domain and each node , for node pairs that do not belong to the same category, minimize their similarity to avoid incorrect matching. Suppress the similarity of incorrectly matched node pairs to ensure that the similarity of incorrectly matched node pairs is as close to 0 as possible, obtaining the minimum node similarity loss. , which is expressed as:

[0138] ;

[0139] Furthermore, in order to ensure the consistency of the structural information of the source domain and target domain graphs, so that the matched node pairs have similar connection relationships in the graph, and to maintain the integrity of the graph structure and the consistency of component composition, an edge structure loss is designed. , which is expressed as:

[0140] ;

[0141] In the above formula, and , respectively represent the adjacency matrices of the source domain and target domain graphs, encoding the structural information of the graphs. If the source domain node and the target domain node belong to the same type of sample, it is assigned 1, otherwise 0.

[0142] Then, the semantic similarity graph structure matching loss is expressed as:

[0143] ;

[0144] In step S130, the source domain and target domain feature maps at different scales and are input into the discriminator , obtaining a multi-level global feature alignment loss, which is expressed as:

[0145] ;

[0146] In the above formula, represents the discriminator, which consists of 3 discriminators and 1 gradient backpropagation layer, represents the number of layers of the multi-level feature extractor, , represent the coordinates of the pixel points.

[0147] In step S140, the detection loss mainly includes the category output loss of the target, the centerness loss and the position regression loss . Among them, uses the focal loss for optimization, uses the binary cross-entropy loss to optimize the centerness of centerness, It is optimized using the IOU loss.

[0148] ;

[0149] In the above formula, represents the predicted probability of the sample, represents the parameter for adjusting the weights of positive and negative samples, and the superscript r represents the harmonic factor.

[0150] ;

[0151] In the above formula, is the true label (0 or 1), is the centerness value predicted by the model.

[0152] ;

[0153] In the above formula, and are the areas of the regions enclosed by the true value coordinates and the predicted coordinates respectively.

[0154] In step S150, in order to optimize the entire cross-sensor SAR target detection model, the losses in the network are integrated, and the final collaborative loss mainly includes the multi-level global alignment loss , the graph scattering node alignment loss , the graph scattering node classification loss , the graph structure matching loss and the detection loss. The total loss function is expressed as:

[0155] ;

[0156] In the above formula, and represent the loss weight coefficients.

[0157] After the training converges, the trained multi-level shared feature extractor and the detection head are used to construct a cross-domain target detection network.

[0158] In this paper, the effectiveness of the proposed method is also verified by experimental results. In the experiment, three datasets from different imaging sensors are used to experimentally verify the algorithm. The SAR measured data of MiniSAR and FARAD are used to verify the proposed method in this paper. The imaging sensor band of MiniSAR data is the Ku band, and the shooting location is in the suburbs. The sensor bands of FARAD data are the Ka band and the X band, and the shooting location is in the urban area with dense buildings. The two types of datasets cover three imaging bands, and FARAD and MiniSAR use completely different sensors. Among them, the image size is 512x512. The training set and the test set are divided according to the ratio of 7:3, and the metrics used are Precision (PR), Recall (RE), mAP, and F1.

[0159] Two different cross-sensor experimental tasks are designed, including from the FARAD Ka sensor to the MiniSAR sensor, and from the MiniSAR sensor to the FARAD X sensor. Experiments are carried out on the two types of cross-sensor tasks and compared with 9 classical deep learning and domain adaptation algorithms. The experimental results are shown in Tables 1 and 2. It can be seen that the algorithm proposed by the proposed method can achieve good experimental results.

[0160] As shown in Table 1, in the Ka—>M cross-sensor task, the proposed method obtained 82.71% and 68.54% in the Precision and Recall metrics respectively. It is about 10% higher than the second-best algorithm with better performance. Compared with the classical Faster-RCNN algorithm, the proposed algorithm is about 40% higher in the mAP metric and about 30% higher in the F1 metric, significantly improving the performance in the cross-sensor object detection task.

[0161] As shown in Table 2, in the M—>X cross-sensor task, the proposed method reached 71.84% and 75.10% in the Precision and Recall metrics respectively. Compared with the EPM method, the mAP value of the proposed algorithm reached 68.27%, which is about 15% higher than the EPM algorithm. In addition, in terms of the F1 value, the proposed algorithm exceeded other algorithms by about 13%-30%, achieving relatively superior performance.

[0162] Table 1 Results on the Ka—>M cross-sensor task

[0163]

[0164] Table 2 Results on the M—>X cross-sensor task

[0165]

[0166] In the above cross-sensor SAR image target detection method, the SAR source domain image with ground truth labels and the SAR target domain image without ground truth labels are input into a multi-layer shared feature extractor to obtain corresponding semantic feature maps respectively, and corresponding graph structures are constructed based on these feature maps. By constructing a loss function to constrain the consistency of the source domain semantic graph structure and the target domain semantic graph structure during training, the trained network has good generalization ability on the target domain. At the same time, combined with target detection constraints and global feature constraints, accurate target detection of the target domain SAR image can be achieved under the training of the target domain image without ground truth labels.

[0167] It should be understood that although Figure 1 the steps in the flowchart of Figure 1 are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0168] In one embodiment, as Figure 6 shown, a cross-sensor SAR image target detection device is provided, including: a dataset acquisition module 200, a graph structure construction module 210, a graph structure alignment constraint module 220, a global feature alignment constraint module 230, a target detection constraint module 240, a training module 250, and a cross-domain detection module 260, where:

[0169] The dataset acquisition module 200 is configured to acquire a source domain dataset and a target domain dataset. The source domain dataset includes multiple SAR source domain images with ground truth labels, and the target domain dataset includes multiple SAR target domain images without ground truth labels;

[0170] The graph structure construction module 210 is configured to extract one SAR image from the source domain dataset and the target domain dataset respectively as a training pair, input the training pair into a multi-layer shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and construct corresponding semantic scattering graph structures based on the source domain semantic feature map and the target domain semantic feature map respectively, to obtain a source domain semantic scattering graph structure and a target domain semantic scattering graph structure;

[0171] The graph structure alignment constraint module 220 is used to construct a scattering graph structure alignment loss function according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure;

[0172] The global feature alignment constraint module 230 is used to utilize a discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map;

[0173] The target detection constraint module 240 is used to utilize a detection head to perform target detection according to the source domain semantic feature map and the target domain semantic feature map respectively, and construct a detection loss function;

[0174] The training module 250 is used to adjust the adjustable parameters in the multi-level shared feature extractor and the detection head by using the scattering graph structure alignment loss function, the global feature alignment loss function, and the detection loss function until convergence, and construct a cross-domain target detection network according to the trained multi-level shared feature extractor and the detection head;

[0175] The cross-domain detection module 260 is used to obtain a SAR image to be detected in the target domain, and perform target detection on the SAR image to be detected by using the cross-domain target detection network to obtain the target category and the target location.

[0176] For the specific limitations of the cross-sensor SAR image target detection device, reference can be made to the limitations of the cross-sensor SAR image target detection method in the above text, which will not be elaborated here. Each module in the above cross-sensor SAR image target detection device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0177] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 7As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a cross-sensor SAR image target detection method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, a touchpad, or a mouse, etc.

[0178] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0179] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0180] Obtain a source domain data set and a target domain data set. The source domain data set includes multiple SAR source domain images with ground truth labels, and the target domain data set includes multiple SAR target domain images without ground truth labels;

[0181] Extract one SAR image from the source domain data set and the target domain data set respectively as a training pair, input the training pair into a multi-layer shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and construct corresponding semantic scattering map structures according to the source domain semantic feature map and the target domain semantic feature map to obtain a source domain semantic scattering map structure and a target domain semantic scattering map structure;

[0182] Construct a scattering structure map alignment loss function according to the source domain semantic scattering map structure and the target domain semantic scattering map structure;

[0183] Use a discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map;

[0184] Using a detection head, perform object detection based on the source domain semantic feature map and the target domain semantic feature map respectively, and construct a detection loss function;

[0185] Using the scattering structure diagram, adjust the adjustable parameters in the multi-layer shared feature extractor and the detection head according to its loss function, the global feature alignment loss function, and the detection loss function until convergence, and construct a cross-domain object detection network according to the trained multi-layer shared feature extractor and the detection head;

[0186] Obtain the SAR image to be detected in the target domain, and use the cross-domain object detection network to perform object detection on the SAR image to be detected, and obtain the object category and the object location.

[0187] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0188] Obtain a source domain dataset and a target domain dataset. The source domain dataset includes multiple SAR source domain images with ground truth labels, and the target domain dataset includes multiple SAR target domain images without ground truth labels;

[0189] Extract one SAR image from the source domain dataset and the target domain dataset respectively as a training pair, input the training pair into the multi-layer shared feature extractor to obtain the source domain semantic feature map and the target domain semantic feature map respectively, and construct corresponding semantic scattering map structures according to the source domain semantic feature map and the target domain semantic feature map respectively, to obtain the source domain semantic scattering map structure and the target domain semantic scattering map structure;

[0190] According to the source domain semantic scattering map structure and the target domain semantic scattering map structure, construct a scattering structure diagram alignment loss function;

[0191] Using a discriminator, construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map;

[0192] Using a detection head, perform object detection based on the source domain semantic feature map and the target domain semantic feature map respectively, and construct a detection loss function;

[0193] Using the scattering structure diagram, adjust the adjustable parameters in the multi-layer shared feature extractor and the detection head according to its loss function, the global feature alignment loss function, and the detection loss function until convergence, and construct a cross-domain object detection network according to the trained multi-layer shared feature extractor and the detection head;

[0194] Obtain the SAR image to be detected in the target domain, and use the cross-domain object detection network to perform object detection on the SAR image to be detected to obtain the object category and object location.

[0195] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0196] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0197] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A cross-sensor SAR image target detection method, characterized in that, The method includes: Obtaining a source domain dataset and a target domain dataset, where the source domain dataset includes multiple SAR source domain images with ground truth labels, and the target domain dataset includes multiple SAR target domain images without ground truth labels; Extracting one SAR image from the source domain dataset and the target domain dataset respectively as a training pair, inputting the training pair into a multi-layer shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and constructing corresponding semantic scattering map structures according to the source domain semantic feature map and the target domain semantic feature map, obtaining a source domain semantic scattering map structure and a target domain semantic scattering map structure; Constructing a scattering structure map alignment loss function according to the source domain semantic scattering map structure and the target domain semantic scattering map structure; Using a discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map; Using a detection head to perform object detection according to the source domain semantic feature map and the target domain semantic feature map respectively, and constructing a detection loss function; Using the scattering structure map alignment loss function, the global feature alignment loss function, and the detection loss function to adjust the tunable parameters in the multi-layer shared feature extractor and the detection head until convergence, and constructing a cross-domain object detection network according to the trained multi-layer shared feature extractor and the detection head; Obtaining a SAR image to be detected in the target domain, and using the cross-domain object detection network to perform object detection on the SAR image to be detected to obtain the object category and the object location.

2. The cross-sensor SAR image target detection method according to claim 1, wherein The constructing of the corresponding semantic scattering map structure according to the source domain semantic feature map and the target domain semantic feature map includes: For the source domain semantic feature map, using the ground truth boxes in the corresponding ground truth labels to perform spatial uniform sampling of pixel points at different feature layers to obtain a source domain semantic scattering point set; For the target domain semantic feature map, using the detection head to generate the category scores of each pixel point, and obtaining a target source domain semantic scattering point set according to the category scores; Mapping the feature points in the source domain semantic scattering point set and the target source domain semantic scattering point set into a low-dimensional vector space respectively, converting them into graph nodes, and constructing the corresponding semantic scattering map structure.

3. The cross-sensor SAR image target detection method according to claim 2, characterized in that The constructing of the scattering structure map alignment loss function according to the source domain semantic scattering map structure and the target domain semantic scattering map structure includes: Calculating a semantic feature point alignment loss function according to the source domain semantic scattering map structure and the target domain semantic scattering map structure; For each graph node in the source domain semantic scattering map structure and the target domain semantic scattering map structure respectively, using the feature distribution of the graph node and the nearest neighbor algorithm to perform node enhancement, obtaining a source domain node enhanced graph structure and a target domain node enhanced graph structure; After interacting the source domain node enhanced graph structure and the target domain node enhanced graph structure, calculating a semantic scattering point classification loss function; Constructing a semantic similarity graph structure matching loss function according to the maximum node similarity, the minimum node similarity, and the edge similarity of the matching node pairs in the source domain node enhanced graph structure and the target domain node enhanced graph structure; Based on the semantic feature point alignment loss function, the semantic scattering point classification loss function, and the semantic similarity graph structure matching loss function, the scattering structure graph alignment loss function is obtained.

4. The cross-sensor SAR image target detection method according to claim 3, characterized in that The semantic feature point alignment loss function is expressed as: ； In the above formula, represents the node alignment discriminator network, represents the th graph node in the source domain semantic scattering graph structure, represents the th graph node in the target domain semantic scattering graph structure, represents the domain classification label, and respectively represent the numbers of graph nodes in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure.

5. The cross-sensor SAR image target detection method according to claim 4, wherein When enhancing each graph node in the source domain semantic scattering graph structure and the target domain semantic scattering graph structure: Model the distribution of graph nodes in the semantic scattering graph structure as a Gaussian distribution, sample feature points from this Gaussian distribution, and then obtain the enhanced semantic scattering nodes according to the linear projection formula, and construct the first enhanced semantic scattering graph node set; Obtain the second enhanced semantic scattering graph node set by taking the union of the graph nodes in the original semantic scattering graph node set and the first enhanced semantic scattering graph node set; Based on the original semantic scattering graph node set, obtain the corresponding adjacency matrix through inner product multiplication, and use the graph convolutional network combined with the adjacency matrix to enhance each graph node in the second enhanced semantic scattering graph node set according to the features of neighboring nodes, and obtain the third enhanced semantic scattering graph node set; Perform clustering operation on the graph nodes in the third enhanced semantic scattering graph node set using the K-nearest neighbor algorithm to obtain the final node enhanced graph structure.

6. The cross-sensor SAR image target detection method according to claim 5, wherein Before constructing the first enhanced semantic scattering graph node set, for the target domain semantic scattering graph structure, the features of the graph nodes in the target domain semantic scattering graph structure are also enhanced using the distribution variance of the graph structure in the target domain semantic scattering graph structure and the mean value of the source domain semantic scattering graph structure.

7. A cross-sensor SAR image target detection device, characterized in that The device includes: A dataset acquisition module for acquiring a source domain dataset and a target domain dataset. The source domain dataset includes multiple SAR source domain images with ground truth labels, and the target domain dataset includes multiple SAR target domain images without ground truth labels; A graph structure construction module for respectively extracting a SAR image from the source domain dataset and the target domain dataset as a training pair, inputting the training pair into a multi-layer shared feature extractor to obtain a source domain semantic feature map and a target domain semantic feature map respectively, and respectively constructing corresponding semantic scattering graph structures according to the source domain semantic feature map and the target domain semantic feature map to obtain a source domain semantic scattering graph structure and a target domain semantic scattering graph structure; A graph structure alignment constraint module for constructing a scattering structure graph alignment loss function according to the source domain semantic scattering graph structure and the target domain semantic scattering graph structure; A global feature alignment constraint module for using a discriminator to construct a multi-level global feature alignment loss function according to the source domain semantic feature map and the target domain semantic feature map; A target detection constraint module for using a detection head to perform target detection according to the source domain semantic feature map and the target domain semantic feature map respectively, and constructing a detection loss function; A training module for using the scattering structure graph alignment loss function, the global feature alignment loss function, and the detection loss function to adjust the tunable parameters in the multi-layer shared feature extractor and the detection head until convergence, and constructing a cross-domain target detection network according to the trained multi-layer shared feature extractor and detection head; A cross-domain detection module, configured to obtain a SAR image to be detected in a target domain, and perform target detection on the SAR image to be detected by using the cross-domain target detection network, so as to obtain a target category and a target location.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images