A physical information guided SAR aircraft target detection method
By constructing a scattering key point prediction network and a feature reweighting network based on a Gaussian mixture model, the problems of difficulty in distinguishing targets from the background and structural discretization in SAR aircraft target detection are solved, achieving efficient and accurate aircraft target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2026-03-27
AI Technical Summary
Existing SAR aircraft target detection methods face problems in practical applications, such as difficulty in distinguishing the similarity between the target and the background, and inaccurate detection due to the discretization of the target structure.
A physical information-guided approach is adopted. By constructing an aircraft target slice dataset, scattering key points are extracted. A probability density image is generated using a Gaussian mixture model as a ground truth heatmap. A joint model is constructed by combining a target scattering key point prediction network and a feature reweighting network to perform feature extraction and detection.
It achieves complete and accurate detection of SAR aircraft targets, suppresses background interference, and improves the accuracy and robustness of detection results.
Smart Images
Figure CN119942323B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to a physical information-guided... SAR Aircraft target detection methods. Background Technology
[0002] Synthetic Aperture Radar (SAP) Synthetic Aperture Radar, SAR Synthetic aperture radar (SAR) technology is a common remote sensing technology with advantages such as all-weather, all-day operation and near-independence from weather conditions, making it a crucial means of Earth observation. Facing future combat scenarios, the pre-war phase requires real-time and rapid updates to the mapping and support information needed by the sensing system to adapt to the complex and constantly changing battlefield environment. SAR reconnaissance is unaffected by weather and lighting conditions. SAR Intelligent target recognition technology based on images has broad application prospects in modern warfare, such as conducting all-weather, 24 / 7 military target reconnaissance in cloudy and rainy areas, collecting intelligence on military targets of interest, and thus guiding operational decisions and tactical deployments.
[0003] For high-resolution remote sensing imagery SAR Aircraft target detection is widely used in traffic management, urban planning, and precision strikes in combat, demonstrating high application value. In recent years, with the advancement of high-resolution... SAR The data acquisition capabilities of remote sensing platforms are constantly improving, based on deep learning. SAR Target detection and recognition technology has developed rapidly, but currently... SAR Aircraft target detection methods still face the following two difficulties in practical applications:
[0004] 1. SAR The aircraft target is similar to the surrounding background.
[0005] First of all, SAR Aircraft targets in images are often located in scenes such as airports and terminals. The geometric distribution of the surrounding background has a high degree of inter-class similarity with the target, which makes it difficult for conventional target detection neural networks to distinguish between real aircraft targets and surrounding background features. The strong scattering key points of buildings distributed around the aircraft target can easily be confused with aircraft parts, resulting in inaccurate target localization.
[0006] 2. SAR The aircraft target structure in the image is discrete.
[0007] In addition, due to SAR The imaging mechanism of optical images differs significantly from that of optical images. SARThe aircraft target in the image is a set of a series of discrete scattering key points or scattering clusters. The discretization of the target structure causes the network to easily misclassify a complete aircraft target into multiple parts when extracting features, which reduces the detection performance of the network. SUMMARY
[0008] The purpose of the present application is to provide a physical information guided SAR aircraft target detection method to further improve SAR aircraft target detection efficiency and accuracy.
[0009] In order to achieve the above-mentioned task, the present application adopts the following technical scheme:
[0010] A physical information guided SAR aircraft target detection method, comprising:
[0011] Step 1, constructing an aircraft target slice data set; extracting scattering key points from the aircraft target slices in the aircraft target slice data set; based on a Gaussian mixture model, generating a probability density image corresponding to the aircraft target slice as a true value heat map using the scattering key points;
[0012] Step 2, constructing a target scattering key point prediction network, training the target scattering key point prediction network using the aircraft target slice data set, and calculating the network loss using the true value heat map corresponding to the aircraft target slice and the predicted heat map obtained by the aircraft target slice through the target scattering key point prediction network during the training process;
[0013] Step 3, obtaining SAR an aircraft detection image data set, and constructing a feature reweighting network; combining the feature reweighting network and the trained target scattering key point prediction network to form a joint model, and connecting the joint model to a deep learning backbone network and a target detection network to construct a physical information guided SAR aircraft target detection network; training the SAR aircraft target detection network using the SAR aircraft detection image data set, saving the trained SAR aircraft target detection network model for use in identifying SAR aircraft target detection images of unknown target classes; wherein the feature reweighting network comprises a compression module and an iterative feature enhancement module.
[0014] SARThe aircraft detection image is input into the deep learning backbone network and the target scattering key point prediction network for feature extraction, respectively, to obtain the corresponding depth feature map and prediction heatmap. The depth feature map and prediction heatmap are compressed by the compression module and then fed into the iterative feature enhancement module for feature enhancement. The enhanced features are then fused with the depth feature map to obtain the detection features, which are then input into the target detection network to obtain the classification and detection results of the aircraft target.
[0015] Furthermore, the extraction of scattering key points from aircraft target slices in the aircraft target slice dataset involves first convolving the aircraft target slices with a Gaussian kernel function to generate a multi-level scale space, and then searching for candidate corner points in each scale space.
[0016] After obtaining the candidate corner points, use LOG The operator filters candidate corner points at different scales and uses an iterative method to verify the candidate corner points at each scale. LOG If the calculated value is an extreme point in all scale spaces, then the candidate corner point is considered the final corner point; the set of all corner points retained in the final aircraft target slice is the scattering key point of the aircraft target slice.
[0017] Furthermore, the step of generating a probability density image corresponding to an aircraft target slice based on a Gaussian mixture model and using scattering key points as a ground truth heatmap includes:
[0018] use K - means Clustering methods generate initial sub-distribution parameters for Gaussian mixture models, including the mean, covariance matrix, and weight coefficients of the clustered sub-distributions;
[0019] Calculate each scattering key point i Belongs to the k Posterior probability of a Gaussian distribution The calculation formula is as follows:
[0020]
[0021] in, K This represents the sub-distributions included in the Gaussian mixture model. Quantity, Indicates the first k The mean of the sub-distribution, and This represents the first k The covariance matrix corresponding to each sub-distribution It is the first in the Gaussian mixture model k The weight coefficients corresponding to each sub-distribution are determined by the number of samples assigned to that sub-distribution. j Indicates the first i Sub-distribution, iIndicates the first i One key scattering point, Indicates the first i Coordinates of the key scattering points;
[0022] Update the parameters in the posterior probability formula. The update formula for each parameter is as follows:
[0023]
[0024] in N Indicates the number of scattering keypoints, superscript T Indicates the transpose operation;
[0025] Iteratively update the parameters until the parameters are updated. , and The updated values are all less than the corresponding preset values, or the preset iteration stop cycle has been reached;
[0026] Then according to K Parameters of the sub-distribution , and Probability density function for constructing a Gaussian mixture model for:
[0027]
[0028] in, X Indicates the key points of scattering. and These are the mean vector and covariance matrix of the Gaussian mixture model;
[0029] Using the probability density function, the probability density image corresponding to each aircraft target slice is obtained, which is also known as the ground truth heatmap.
[0030] Furthermore, the construction of the target scattering key point prediction network, and the training of the target scattering key point prediction network using an aircraft target slice dataset, includes:
[0031] First, the aircraft target slices are preprocessed by adjusting their size. The preprocessed aircraft target slices and the corresponding ground truth heatmaps are then input into the target scattering keypoint prediction network. The target scattering keypoint prediction network includes a backbone and a keypoint feature learning part, where:
[0032] The preprocessed aircraft target slices first pass through the backbone section. The backbone section first uses two convolutional layers to reduce the image resolution by one-quarter, then uses four... BottleNeck The first layer performs preliminary feature extraction, and the extracted feature map is input into the key point feature learning part.
[0033] The key point feature learning part includes four sub-networks, the first layer sub-network to the fourth layer sub-network respectively include 10, 8, 5, 2 convolution layers connected in turn; the feature map in the second layer sub-network is reduced by half in resolution but doubled in channel number compared with the feature map in the first layer sub-network;
[0034] The feature map extracted by the backbone part A The input is input into the first layer sub-network, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map A 1; the feature map extracted by the backbone part A After down-sampling, the input is input into the second layer sub-network, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map A 2; the feature map output by the second convolution layer of the second layer sub-network is input into the third layer sub-network after down-sampling, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map A 3; the feature map output by the second convolution layer of the third layer sub-network is input into the fourth layer sub-network after down-sampling, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map A 4;
[0035] Among them, the feature map output by the first convolution layer of the i-th layer sub-network is denoted as i The feature map output by the j-th convolution layer of the i-th layer sub-network is denoted as j a ij The following relationship exists:
[0036] a 12 After down-sampling and adding with a 22 , the third convolution layer of the second layer sub-network is entered; and a 21 After up-sampling and adding with a 13 , the fourth convolution layer of the first layer sub-network is entered;
[0037] a 24 , a 31 Respectively after up-sampling and then adding with a 16 , the seventh convolution layer of the first layer sub-network is entered; a 15 After down-sampling, a 31 After up-sampling, the result is added with a 25 , and then the sixth convolution layer of the second layer sub-network is entered; a 15 ,a 24 respectively after down-sampling and a 32 are added, and then enter the third convolutional layer of the third sub-network;
[0038] a 27 , a 34 , a 41 respectively after up-sampling and a 19 are added, and then enter the tenth convolutional layer of the first sub-network; a 18 after down-sampling, a 34 after up-sampling, a 41 respectively after up-sampling and a 28 are added, and the result is A 2; a 18 after down-sampling, a 27 after down-sampling, a 41 respectively after up-sampling and a 35 are added, and the result is A 3; a 18 after down-sampling, a 27 after down-sampling, a 34 respectively after down-sampling and a 42 are added, and the result is A 4;
[0039] The final generated prediction heat map is A 2, A 3, A 4 respectively after up-sampling and A 1 added to the feature map.
[0040] Further, the network loss is calculated by using the true value heat map corresponding to the aircraft target slice and the prediction heat map predicted by the target scattering key point prediction network.
[0041] A loss function based on the prediction heat map and the true value heat map is constructed, which is used to calculate the network loss during training; wherein the loss function is as follows:
[0042]
[0043] wherein represents the network loss, N represents the total number of pixels in the ground truth heat map, , respectively represent the values of the pixels with coordinates ( w , h ) in the ground truth heat map corresponding to the aircraft target slice input into the network and the predicted heat map obtained through network prediction, W and H represent the total number of pixels in the length direction and the width direction.
[0044] Further, the deep learning backbone network adopts ResNet 18 or VGG 16; and the target detection network adopts YOLOv 5.
[0045] Further, the compression module comprises a mean pooling layer and a max pooling layer, SAR the deep feature map extracted by the deep learning backbone network and the predicted heat map extracted by the target scattering key point prediction network are respectively processed by the mean pooling layer and the max pooling layer, and the two pooled images obtained through the mean pooling layer and the max pooling layer are respectively weighted and fused according to a pre-set weight, the fusion process is to add each pixel element of the pooled images, so as to obtain the corresponding backbone compressed feature map and key point compressed feature map, and input them into the iterative feature enhancement module.
[0046] Further, the processing process of the iterative feature enhancement module is as follows:
[0047] The backbone compressed feature map is projected into the key-value matrix, while the key point compressed feature map is projected into the query matrix; cross-attention calculation is performed using the query matrix of the key point compressed feature map and the key-value matrix of the backbone compressed feature map to obtain a feature Z representing the correlation between the two; then the feature Z is re-projected through a linear layer, and the projected result is connected with the backbone compressed feature map according to weights , so as to fuse the complementary information from different features and obtain a complementary feature ; the result of the complementary feature processed by the feedforward network is connected with the complementary feature according to weights , so as to obtain an enhanced feature ; the enhanced feature is element-wise added and fused with the deep feature map extracted by the deep learning backbone network to obtain the final detection feature , In the input target detection network, a detection result of the aircraft target is obtained.
[0048] A terminal device comprises a processor, a memory and a computer program stored in the memory; when the processor executes the computer program, the physical information guided SAR Aircraft target detection method.
[0049] A computer readable storage medium, the medium stores a computer program; when the computer program is executed by the processor, the physical information guided SAR Aircraft target detection method.
[0050] Compared with the prior art, the present application has the following technical features:
[0051] 1. For SAR The strong scattering key points of the aircraft target are usually distributed in the nose, wings and other key parts, and contain a large amount of structural feature information, so the present application first extracts the scattering key points of the target, and in order to further represent the extracted scattering key points as SAR The key cluster perceptual features of the aircraft target structure, the Gaussian mixture model is used to cluster the scattering key points; the aircraft target slice is first subjected to a backbone network to reduce the resolution of the image by one fourth to obtain basic features, and then the basic features are subjected to a subsequent multi-scale feature pyramid network HRNet After extraction, the predicted heat map of the aircraft target slice is output; the loss function of the predicted heat map and the true value heat map is calculated to evaluate.
[0052] 2. For the predicted heat map of the target scattering key point prediction network, the present application designs a feature reweighting network based on the physical information guided cross attention mechanism; first, the two features input are subjected to information dimension pooling compression through a spatial feature contraction module, and the compressed features are subjected to interactive fusion through an iterative cross-modal feature enhancement module; finally, a feedforward network is used to further refine the feature representation, improve the robustness and accuracy of the model.
[0053] Based on the above design, the present method can on the one hand detect SAR The aircraft target completely and accurately, and on the other hand can effectively suppress SAR The background interference similar to the target in the image, and has the characteristics of good detection target completeness effect and high detection result accuracy. BRIEF DESCRIPTION OF DRAWINGS
[0054] Figure 1 The flowchart of the aircraft target detection algorithm based on the physical information guided SAR of the present application;
[0055] Figure 2 The flowchart of the aircraft target detection algorithm based on the physical information guided SARFlowchart of training the aircraft target slice dataset;
[0056] Figure 3 This is a schematic diagram of the target scattering key point prediction network of the present invention;
[0057] Figure 4 This is a schematic diagram of the structure of the upsampling / downsampling network fusion method of the present invention.
[0058] Figure 5 This is a schematic diagram of the feature reweighting network of the present invention. Detailed Implementation
[0059] To address the problems existing in the prior art, this invention aims to provide a physical information-guided... SAR Aircraft target detection methods employ physical knowledge such as target scattering cluster models to... SAR A unified structured description and quantitative expression of aircraft target characteristics; based on SAR This research focuses on the physical characteristics of targets, constructing a self-supervised learning framework for predicting high-resolution feature pyramid-shaped scattering key points guided by physical characteristics; and developing a hybrid neural network model for SAR aircraft detection and recognition coupled with physical knowledge. These methods aim to complete the research on interpretable SAR target intelligent interpretation technology based on physical characteristics. SAR Experiments on the scattering keypoint prediction network algorithm based on aircraft target slices as training data and SAR Aircraft detection images form the training data SAR Aircraft target detection and identification experiments, forming SAR Target physical property extraction algorithm SAR Self-supervised feature learning algorithms for targets, intelligent recognition algorithms combining physical features, SAR The system integrates aircraft detection and recognition algorithms and other unit modules into a complete physical feature combination. SAR Aircraft intelligent detection and recognition algorithm software. Achieves robust detection of aircraft targets in complex environments, improving... SAR Improve aircraft target detection and recognition performance, while also enhancing algorithm transparency and reliability.
[0060] This invention provides a physical information guided SAR The aircraft target detection method includes the following steps:
[0061] Step 1: Construct an aircraft target slice dataset; extract scattering key points from the aircraft target slices in the dataset; based on a Gaussian mixture model (... GMM The probability density image corresponding to the aircraft target slice is generated using scattering key points as the ground truth heatmap.
[0062] Firstly, from the publicly available... SARMultiple clear aircraft target slices were selected from the aircraft dataset, and each aircraft target slice contains a complete aircraft target, thus constructing an aircraft target slice dataset.
[0063] In this embodiment, the aircraft target slices are respectively from... SADD , TerraSAR-X , Multiangle SAR Dataset and SAR - CAD After filtering 280, 131, 443 and 170 images from the dataset, a total of 1024 aircraft target slices were obtained.
[0064] Secondly, key scattering points were extracted from the aircraft target slices, see Appendix. Figure 2 The details are as follows:
[0065] (1) Extract the corner points of the aircraft target slice.
[0066] First, a scale space is generated by convolving a Gaussian kernel function with the aircraft target slice. Candidate corner points are then searched within each scale space. This method generates the scale space by convolving Gaussian kernel functions with different scale parameters with the aircraft target slice. The expression is as follows:
[0067]
[0068] In the formula, For scale space, This represents the convolution operation. Slice the input aircraft target. Gaussian kernel function , and The x and y coordinates of the pixel positions in the aircraft target slice. This is the scale parameter.
[0069] Inspired by the autocorrelation function in signal processing, the autocorrelation matrix... M The eigenvalues are the first-order curvatures of the autocorrelation function. If both first-order curvature values of a pixel are large, then it is a corner point. The autocorrelation matrix... M Extending this to scale space, it can be represented as follows:
[0070]
[0071] In the formula, It is a Gaussian function. Representing scale space respectively x , y direction and The differential obtained by multiplying the Gaussian convolution kernels, where For differential scale, to counteract the scaling of the derivative scale to the Gaussian kernel; For the integral scale, used to adjust the scale of the Gaussian kernel, the larger the value, the larger the corresponding scale.
[0072] Define the corner response function:
[0073]
[0074] where, R is the corner response value, det is the determinant of the matrix, trace is the trace of the matrix, is a constant, usually in the range [0.04, 0.06], in this embodiment 0.04.
[0075] Calculate the corner response value of each pixel point in a single scale space, and determine the candidate corner point when the corner response value exceeds the pre-set threshold. In this way, the candidate corner points in a single scale space are obtained. However, such candidate corner points do not have scale invariance, so multiple scale spaces need to be established to search for candidate corner points in different spaces.
[0076] A set of is predefined, is the n th scale parameter, and a set of scale spaces under different scale Gaussian kernel functions for the aircraft target slice is obtained. In this embodiment n 3 is set. The candidate corner points in each scale space are extracted according to the above method.
[0077] (2) Since there are a large number of redundancies or detection errors in the candidate corner points obtained in multiple scale spaces, a LOG operator ( LaplacianofGaussian ) is introduced to screen the candidate corner points under different scale spaces. The iterative method is used to check whether the operation value of the candidate corner point in each scale space is the extreme value point in all scale spaces. If not, it is discarded, and if so, it is considered as the final corner point. LOG
[0078] (3) Finally, the set of all corner points retained by the aircraft target slice is the scattering key point of the aircraft target slice.
[0079] Step 1.4, the scattering key points extracted from each aircraft target slice are clustered using a Gaussian mixture model to obtain a probability density image of the scattering key point clustering of the aircraft target slice as a true value heat map.
[0080] (1) Parameter initialization setting:
[0081] UseK - means The clustering method generates initial parameters of the sub-distributions of the Gaussian mixture model, including the mean, covariance matrix and weight coefficient of the sub-distribution. In this embodiment, the number of clustering centers is 9.
[0082] For the scattering key points of the aircraft target slice, the mean of each sub-distribution is the coordinate mean of the scattering key points in the class, the covariance matrix is calculated by the covariance of the scattering key points, and the weight coefficient is determined by the number of scattering key points in each sub-distribution.
[0083] (2) Calculate the posterior probability:
[0084] Calculate the posterior probability of each scattering key point i belonging to the k th sub-Gaussian distribution , and the calculation formula is as follows:
[0085]
[0086] Wherein, K represents the number of sub-distributions contained in the Gaussian mixture model, represents the mean of the k th sub-distribution, and represents the covariance matrix corresponding to the k th sub-distribution, is the weight coefficient corresponding to the k th sub-distribution in the Gaussian mixture model, which is determined by the number of samples divided into the sub-distribution, j represents the i th sub-distribution, i represents the i th scattering key point, represents the coordinates of the i th scattering key point.
[0087] (3) Update the parameters in the posterior probability formula, and the update formula of each parameter is as follows:
[0088]
[0089] Wherein N represents the number of scattering key points, and the superscript T represents the transpose operation.
[0090] (4) Repeat iteration
[0091] Repeat steps (2) and (3) until the parameters , and The updated values are all less than the corresponding preset values, or the preset iteration stop number is reached; in this embodiment, the preset value is 0.01, and the iteration number is 50. The updated mean, covariance matrix, and weight coefficients of the sub-distribution are obtained.
[0092] The above method allows us to estimate the Gaussian sub-distribution to which each scattering key point belongs, as well as the mean of the sub-distribution corresponding to that Gaussian sub-distribution. Covariance matrix and weighting coefficients Then according to K Parameters of the sub-distribution , and Probability density function for constructing a Gaussian mixture model for:
[0093]
[0094] in, X Indicates the key points of scattering. and These are the mean vector and covariance matrix of the Gaussian mixture model.
[0095] Using the probability density function, a probability density image, or ground truth heatmap, is obtained for each aircraft target slice, as shown below. Figure 2 As shown, the true heatmap size is one-quarter the size of the aircraft target slice.
[0096] Step 2: Construct a target scattering key point prediction network. Train the target scattering key point prediction network using the aircraft target slice dataset. During the training process, calculate the network loss using the ground truth heatmap corresponding to the aircraft target slice and the predicted heatmap obtained by the target scattering key point prediction network for the aircraft target slice.
[0097] Step 2.1: In the network training stage using the aircraft target slice dataset, the aircraft target slices are first preprocessed to adjust their size. During preprocessing, the original aspect ratio of the image is kept unchanged, and empty parts are padded with zeros. In this embodiment, the adjusted size is (256, 192).
[0098] Step 2.2: Input the preprocessed aircraft target slice and the corresponding ground truth heatmap into the target scattering key point prediction network to learn the scattering key point feature distribution of the aircraft target slice.
[0099] The target scattering key point prediction network provided by this invention is as follows: Figure 3 As shown, it includes the main body and the key point feature learning part, where:
[0100] (1) Main section
[0101] The preprocessed aircraft target slice first passes through the backbone part, which first uses two 3*3 convolution layers to reduce the resolution of the image by one fourth, so as to match the resolution size of the real heat map. Then four BottleNeck layers are used for preliminary feature extraction, and the extracted feature map is input into the key point feature learning part. In this embodiment, the size of the feature map output by the backbone part and the real heat map is (32, 24), so as to facilitate subsequent loss calculation.
[0102] (2) Key point feature learning part
[0103] The key point feature learning part includes four sub-networks, and the first layer sub-network to the fourth layer sub-network respectively includes 10, 8, 5, and 2 convolution layers connected in turn; the resolution of the feature map in the second layer sub-network is reduced by half but the number of channels is doubled compared with that in the first layer sub-network, and the specific design is as follows:
[0104] The feature map extracted by the backbone part is input into the first layer sub-network, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map is A 1; the feature map extracted by the backbone part is input into the second layer sub-network after down-sampling, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map is A 2; the feature map output by the second convolution layer of the second layer sub-network is input into the third layer sub-network after down-sampling, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map is A 3; the feature map output by the second convolution layer of the third layer sub-network is input into the fourth layer sub-network after down-sampling, and the feature map is extracted through each convolution layer in the sub-network in turn, and the output feature map is A 4; A A Among them, the feature map output by the first convolution layer of the first layer sub-network is denoted as
[0105] The feature map output by the first convolution layer of the second layer sub-network is denoted as i The feature map output by the first convolution layer of the third layer sub-network is denoted as j The feature map output by the first convolution layer of the fourth layer sub-network is denoted as a ij The following relationship exists:
[0106] a 12 After being down-sampled and added to a 22 , the third convolution layer of the second layer sub-network is entered; and a 21 After being up-sampled and added to a 13 , the fourth convolution layer of the first layer sub-network is entered;
[0107] a 24 、 a 31 respectively up-sampled and then added to a 16 , and then enters the seventh convolutional layer of the first subnetwork; a 15 down-sampled, a 31 , and then enters the sixth convolutional layer of the second subnetwork; a 25 , a 15 , a 24 respectively down-sampled and then added to a 32 , and then enters the third convolutional layer of the third subnetwork;
[0108] a 27 , a 34 , a 41 respectively up-sampled and then added to a 19 , and then enters the tenth convolutional layer of the first subnetwork; a 18 down-sampled, a 34 up-sampled, a 41 , and then enters the ninth convolutional layer of the first subnetwork; a 28 , and the result is A 2; a 18 down-sampled, a 27 down-sampled, a 41 , and then enters the eighth convolutional layer of the first subnetwork; a 35 , and the result is A 3; a 18 down-sampled, a 27 down-sampled, a 34 , and then enters the seventh convolutional layer of the first subnetwork; a 42 , and the result is A 4.
[0109] The finally generated prediction heat map is A 2, A 3, A4 the results after upsampling respectively and A 1 the feature map after addition.
[0110] The above multi-layer sub-network structure designed in the scheme, each layer of sub-network is reduced by half than the resolution of the previous layer, but the number of channels is doubled; the sub-networks of different resolutions are connected in parallel, and are fused at different convolution layers; wherein the specific fusion mode is as shown in Figure 4 When the low-resolution feature map is fused into the high-resolution feature map, the resolution size is different by one half, and an upsampling module is added to enlarge the feature map, and the high-resolution feature map is added element by element; similarly, when the feature map is fused from high to low, the resolution size is different by one half, and a downsampling module is added, which is realized by a 3*3 convolution layer with a step of 2, and the reduced feature map is added element by element with the to-be-fused feature map; finally, the feature maps output by the last three sub-networks are added after upsampling, and the feature map of the first sub-network is added to obtain the final prediction heat map.
[0111] Step 2.3, a loss function based on the prediction heat map and the true value heat map is constructed, which is used to calculate the network loss during training; wherein the loss function is as follows:
[0112]
[0113] Wherein represents the network loss, N represents the total number of pixels in the true value heat map, , respectively represent the values of the pixel points with coordinates ( w , h ) in the true value heat map and the prediction heat map obtained by network prediction corresponding to the input network aircraft target slice, W and H represent the total number of pixels in the length direction and the width direction. The loss function aims to evaluate and constrain the learning effect of the network on the key point features by calculating the mean square difference of each pixel point between the true value heat map and the prediction heat map.
[0114] After network training, when the network loss converges, the trained target scattering key point prediction network is obtained, which can realize good prediction of the key point features in the aircraft target slice, and the prediction result is the corresponding prediction heat map.
[0115] Step 3, obtaining SAR aircraft detection image data set, and constructing feature re-weighting network; the feature re-weighting network and the trained target scattering key point prediction network are combined into a joint model, and the joint model is connected to the deep learning backbone network and the target detection network, so as to construct the physical information guided SARAn aircraft target detection network; utilizing SAR An aircraft detection image dataset SAR Training the aircraft target detection network, saving the trained SAR The aircraft target detection network model is used for identifying unknown target categories SAR The aircraft target detection image; wherein the feature reweighting network comprises a compression module and an iterative feature enhancement module.
[0116] SAR The aircraft detection image in the aircraft detection image dataset SAR The aircraft detection image is input into the backbone network of the target detection network and the target scattering key point prediction network for feature extraction, to obtain corresponding depth feature maps and prediction heat maps. The depth feature maps and prediction heat maps are compressed by the compression module and then jointly enter the iterative feature enhancement module for feature enhancement. The enhanced features are then input into the target detection network to obtain the classification and detection results of the aircraft target.
[0117] In this step, SAR The aircraft detection image in the aircraft detection image dataset SAR The aircraft detection image is a real image taken for the aircraft target SAR Different from the aircraft target slice in step 1; SAR The aircraft detection image may contain one or more aircraft targets. Although the target scattering key point prediction network is trained for the aircraft target slice, its network parameters also have good key point detection effect for the complete SAR Aircraft detection image. In this scheme, SAR The aircraft detection image dataset is selected from the public dataset SAR - Aircraft 1.0, the ratio of training set and test set is consistent with the original dataset.
[0118] (1) Deep learning backbone network
[0119] The deep learning backbone network in this scheme can adopt an existing network structure, for example ResNet 18 or VGG 16, etc. to process the input SAR Aircraft detection image to obtain a depth feature map. In this example, the deep learning backbone network is ResNet18.
[0120] (2) Compression module
[0121] In order to reduce the computational complexity of the subsequent module while minimizing the loss of key point feature information, the spatial feature compression module compresses the deep feature map extracted by the backbone network and the predicted heat map extracted by the target scattering key point prediction network, adaptively aggregates feature information by combining average pooling and maximum pooling; the average pooling retains background information, and the maximum pooling retains texture features, and the final compressed feature map is obtained by weighted summation; through the spatial feature compression module, the spatial dimension of the feature map can be significantly reduced, thereby reducing the computational complexity of the subsequent module.
[0122] Specifically, as shown in Figure 5 , the compression module includes a mean pooling layer and a maximum pooling layer, SAR The deep feature map extracted by the deep learning backbone network and the predicted heat map extracted by the target scattering key point prediction network of the aircraft detection image are processed by the mean pooling layer and the maximum pooling layer respectively, and the weights λ The two pooled images obtained by processing the mean pooling layer and the maximum pooling layer are fused by weighting, and the fusion process is to add each pixel element of the pooled image, thereby obtaining the corresponding backbone compressed feature map and key point compressed feature map, and inputting to the iterative feature enhancement module. In this example, λ The value is 0.5.
[0123] (3) Iterative feature enhancement module
[0124] The iterative feature enhancement module continuously strengthens the cross-modal feature information in an iterative manner, thereby improving the discriminability of the feature representation.
[0125] The processing process of the iterative feature enhancement module is as follows:
[0126] The backbone compressed feature map is projected into the key value K , V ) matrix, and the key point compressed feature map is projected into the query Q matrix; cross attention calculation is performed using the query matrix Q of the key point compressed feature map and the key value matrix K , V ) of the backbone compressed feature map, to obtain a feature Z representing the correlation between the two; then the feature Z is re-projected through a linear layer, and the projected result is connected with the backbone compressed feature map according to the weight , to fuse the complementary information from different features and obtain a complementary feature , thereby improving the robustness and accuracy of the model; the result after the complementary feature is processed by the feedforward network FNN , and then the complementary feature Respectively according to the weight Connect to get enhanced features , so as to improve the robustness and accuracy of the model; the enhanced features and the deep feature map extracted by the deep learning backbone network are element-wise added and fused to obtain the final detection feature .
[0127] Wherein, All are learnable parameters in the training process. In the present scheme The initialization value is 0.5.
[0128] (4) Target detection network
[0129] The target detection network in the present scheme is used for detecting targets in the feature map, which can adopt an existing network, for example YOLOv5 , etc. The final detection feature is input into the target detection network to obtain the classification and detection result of the aircraft target. The target detection network in the present scheme uses YOLOv5 .
[0130] Embodiment:
[0131] In an embodiment of the present application, the central processor is Intel ( R ) i 9-10900 X 3.7 GHzCPU , memory 256 G , 4 NvidiaGTX 3090 GPU , Ubuntu 18.04 operating system, using Pytorch framework for simulation; during the experiment, SAR The training set and test set ratio of the aircraft detection image is set according to SAR - Aircraft 1.0. The test results are shown in Table 1; from the test results, it can be seen that the detection method proposed in the present application has obviously improved the recognition rate of different targets compared with the traditional network.
[0132] Table 1 Physical information guided SAR aircraft detection method verification result
[0133]
[0134] The above examples are only used to illustrate the technical solutions of the present application, but not limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A physical information guided SAR Aircraft target detection method characterized in that, comprises: Step 1, constructing an aircraft target slice data set; extracting scattering key points of the aircraft target slice in the aircraft target slice data set; based on a Gaussian mixture model, generating a probability density image corresponding to the aircraft target slice as a true value heat map using the scattering key points; Step 2, constructing a target scattering key point prediction network, training the target scattering key point prediction network using the aircraft target slice data set, and calculating the network loss using the true value heat map corresponding to the aircraft target slice and the predicted heat map predicted by the target scattering key point prediction network during the training process; Step 3, obtaining SAR The aircraft detects the image data set and constructs a feature reweighting network; The feature reweighting network and the trained target scattering key point prediction network are combined into a joint model, and the joint model is connected to a deep learning backbone network and a target detection network, so as to construct a physical information guided SAR aircraft target detection network. Utilizing SAR Aircraft detection image dataset pair SAR Training an aircraft target detection network, saving the trained SAR The aircraft target detection network model is used for identifying the SAR Aircraft target detection image; wherein the feature The weighted network comprises a compression module and an iterative feature enhancement module. SAR The aircraft detection image is respectively input into a deep learning backbone network and a target scattering key point prediction network for feature extraction to obtain corresponding depth feature maps and prediction heat maps. The depth feature maps and the prediction heat maps are respectively compressed by a compression module and then jointly input into an iterative feature enhancement module for feature enhancement. The enhanced features are fused with the depth feature maps to obtain detection features and input into a target detection network to obtain classification and detection results of the aircraft target.
2. The physical information guide of claim 1, SAR A method of detecting an aircraft target, characterized by, The extraction of the scattering key points of the aircraft target slice in the aircraft target slice data set first generates a multi-scale space by convolution with a Gaussian kernel function and the aircraft target slice, and finds candidate corner points in each scale space. After obtaining the candidate corner points, use LOG The operator filters candidate corner points at different scales and uses an iterative method to verify the candidate corner points at each scale. LOG If the calculated value is an extreme point in all scale spaces, then the candidate corner point is considered the final corner point; the set of all corner points retained in the final aircraft target slice is the scattering key point of the aircraft target slice.
3. The physical information guide of claim 1, wherein SAR A method for detecting an aircraft target, characterized by The true value heat map is generated by the scattering key points based on the Gaussian mixture model, comprising: Use K - means The clustering method generates initial parameters of the sub-distributions of the mixture Gaussian model, including the mean, covariance matrix and weight coefficient of the clustered sub-distributions; Compute each scattering keypoint i belongs to the k posterior probability of the th sub-Gaussian distribution is computed as follows: wherein, K denotes the number of sub-distributions contained in the Gaussian mixture model, denotes the mean of the k th sub-distribution, and represents the covariance matrix corresponding to the k th sub-distribution, is the weight coefficient corresponding to the k th sub-distribution in the Gaussian mixture model, which is determined by the number of samples divided into the sub-distribution, j denotes the i th sub-distribution, i denotes the i th scattering key point, denotes the coordinate of the i th scattering key point; Update the parameters in the posterior probability formula, and the update formula of each parameter is as follows: wherein N denotes the number of scattering keypoints, the superscript T denotes the transpose operation; The parameters are iteratively updated until the updated values of the parameters , and are all less than the corresponding preset values, or a preset iteration stopping round is reached. Then the probability density function of the Gaussian mixture model is constructed according to the parameters of the individual sub-distributions K , , and as follows: f(x) = ååååååååååååååååååååååååååååååååååååååååååååååå wherein, X denotes a scattering key point, and is a mean vector and a covariance matrix of a Gaussian mixture model; Using the probability density function, the probability density image corresponding to each aircraft target slice, i.e. the true value heat map, is obtained.
4. The physical information guide of claim 1, wherein SAR A method of detecting an aircraft target, characterized by The construction of the target scattering key point prediction network and the training of the target scattering key point prediction network using the aircraft target slice data set comprises: First, the aircraft target slice is preprocessed by adjusting the size, and the preprocessed aircraft target slice and the corresponding true value heat map are input into the target scattering key point prediction network; the target scattering key point prediction network comprises a backbone part and a key point feature learning part, wherein: The preprocessed aircraft target slice first passes through a backbone part, the backbone part first uses two convolutional layers to reduce the resolution of the image by one fourth, then uses four BottleNeck layers to perform preliminary feature extraction, and the extracted feature map is input to a key point feature learning part; The key point feature learning part comprises four sub-networks, and the first layer sub-network to the fourth layer sub-network respectively comprise 10, 8, 5 and 2 convolution layers connected in turn; the resolution of the feature map in the second layer sub-network is reduced by half but the channel number is doubled compared with that in the first layer sub-network; feature maps extracted by the backbone part A input into the first layer subnetwork, sequentially pass through each convolutional layer in the subnetwork for feature map extraction, and output feature maps A 1; feature maps extracted by the backbone part A input into the second layer subnetwork after down-sampling, sequentially pass through each convolutional layer in the subnetwork for feature map extraction, and output feature maps A 2; feature maps output by the second convolutional layer of the second layer subnetwork are input into the third layer subnetwork after down-sampling, sequentially pass through each convolutional layer in the subnetwork for feature map extraction, and output feature maps A 3; feature maps output by the second convolutional layer of the third layer subnetwork are input into the fourth layer subnetwork after down-sampling, sequentially pass through each convolutional layer in the subnetwork for feature map extraction, and output feature maps A 4; Wherein, record the first i layer network of the first j convolutional layer output feature map is a ij Then there is the following relationship: a 12 down-sampled and added to a 22 the third convolutional layer of the second subnetwork; and a 21 up-sampled and added to a 13 the fourth convolutional layer of the first subnetwork; a 24 , a 31 respectively up-sampled and then added to a 16 a seventh convolutional layer of the first subnetwork; a 15 down-sampled, a 31 the up-sampled results are added to a 25 a sixth convolutional layer of the second subnetwork; a 15 , a 24 the down-sampled results are added to a 32 a third convolutional layer of the third subnetwork; a 27 , a 34 , a 41 respectively after upsampling and adding, and then entering the tenth convolutional layer of the first layer subnetwork; a 19 a 18 after downsampling, a 34 after upsampling, a 41 the result after upsampling and adding, a 28 A 2; a 18 after downsampling, a 27 after downsampling, a 41 the result after upsampling and adding, a 35 A 3; a 18 after downsampling, a 27 after downsampling, a 34 the result after downsampling and adding, a 42 A 4; The finally generated prediction heat map is A 2、 A 3、 A 4 respectively after up-sampling the results and A 1 after adding the feature map.
5. The physical information guide of claim 1, wherein SAR A method of detecting an aircraft target, characterized by The network loss is calculated using the true value heat map corresponding to the aircraft target slice and the predicted heat map predicted by the target scattering key point prediction network using the aircraft target slice, comprising: A loss function based on the predicted heat map and the true value heat map is constructed, which is used to calculate the network loss during training; wherein the loss function is as follows: wherein represents a network loss, N represents a total number of pixel points of a ground truth heat map, , respectively represent values of pixel points with coordinates ( w , h ) in a ground truth heat map corresponding to an aircraft target slice of an input network and a predicted heat map obtained by network prediction, W and H represent total numbers of pixels in a length direction and a width direction.
6. The physical information guide of claim 1, wherein SAR A method of detecting an aircraft target, characterized by The deep learning backbone network adopts ResNet 18 or VGG 16; the target detection network adopts YOLOv 5.
7. The physical information guide of claim 1, wherein SAR A method of detecting an aircraft target, characterized by The compression module comprises a mean pooling layer and a maximum pooling layer, SAR The deep feature map extracted by the deep learning backbone network and the predicted heat map extracted by the target scattering key point prediction network are processed by the mean pooling layer and the maximum pooling layer respectively, and the two pooled images obtained by processing the mean pooling layer and the maximum pooling layer are weighted and fused according to the pre-set weight. The fusion process is to add each pixel element of the pooled image, so as to obtain the corresponding backbone compression feature map and key point compression feature map, and input them into the iterative feature enhancement module.
8. The physical information guide of claim 1, wherein SAR A method of detecting an aircraft target, characterized by The processing process of the iterative feature enhancement module is: The backbone compressed feature map is projected into the key-value matrix, and the key point compressed feature map is projected into the query matrix. Cross-attention is calculated using the query matrix of the keypoint compressed feature map and the key-value matrix of the backbone compressed feature map to obtain features representing the correlation between the two. Z Next, the features Z The image is reprojected through a linear layer, and the projected result is compared with the backbone compressed feature map according to their respective weights. By connecting and fusing complementary information from different features, complementary features are obtained. Complementary characteristics The result after processing by the feedforward network is then compared with complementary features. According to weight Perform connections to obtain enhanced features. ; will enhance features The final detection features are obtained by element-wise addition and fusion of the deep feature maps extracted by the deep learning backbone network. ,Will The data is input into the target detection network to obtain the detection results of the aircraft target. 9.A terminal device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that, The processor, when executing the computer program, implements the physical information guiding according to any one of claims 1-8 SAR Aircraft target detection method.
10. A computer readable storage medium having stored therein a computer program; characterized in that, The computer program, when executed by a processor, implements the physical information guiding according to any one of claims 1-8 SAR Aircraft target detection method.