SAR target recognition method based on fusion of ASC features and multi-scale depth features

Abstract

The present invention discloses a SAR target recognition method based on the fusion of ASC features and multi-scale depth features, comprising: obtaining the original SAR complex image of the observed target and extracting the attribute scattering center corresponding to each SAR complex image; reconstructing the attribute scattering center to obtain global and local reconstruction images of different scales; constructing a deep neural network including a feature extraction module and a feature fusion module to perform feature fusion at different levels on the multi-scale depth feature map and the binarized image corresponding to the reconstructed image; inputting the original SAR complex image and the binarized image corresponding to the reconstructed image into a trained deep neural network for processing, and outputting the target recognition result. The method uses accurately estimated ASC parameters to perform multiple types of reconstructions on the target and fuses them with depth features of different scales, providing more information to the network, thereby improving recognition performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar target recognition, and in particular relates to a SAR target recognition method based on the fusion of ASC features and multi-scale depth features. Background Art

[0002] SAR (Synthetic Aperture Radar) is an active Earth observation system. Benefiting from its unique electromagnetic scattering imaging mechanism, SAR can operate around the clock and in all weather conditions and achieve long-range, high-resolution imaging. Therefore, it is widely used in both military and civilian fields. Compared with optical images, SAR images lack color information and are susceptible to speckle noise, making their interpretation more difficult. Automatic target recognition (ATR) is a key topic in the intelligent interpretation of SAR images and has attracted widespread attention from researchers.

[0003] In recent years, with the rapid development of deep learning, researchers have proposed a variety of deep learning-based SAR target recognition algorithms. Deep learning is a data-driven algorithm, and its excellent performance often relies on a large amount of training data. However, the acquisition of large amounts of labeled field-measured SAR data is costly, and compared to optical datasets, SAR image datasets are relatively few in number and smaller in size. The problem of insufficient samples has a significant impact on SAR ATR algorithms using deep learning. However, SAR images also have unique characteristics, and the problem of insufficient training samples can be alleviated by effectively using these features. Due to their unique electromagnetic scattering properties, the ASC (Attributed Scattering Center) model can be used to extract and estimate the parameters of SAR targets. The ASC model is an effective method for interpreting SAR measurement results. It provides physically relevant features of complex targets and has a certain degree of anti-interference ability. By using ASC features, the recognition performance of SAR ATR algorithms can be improved.

[0004] Several studies have been conducted to address these issues. For example, in their 2020 paper, "Efficient Attributed Scatter Center Extraction Based on Image-Domain Sparse Representation" (IEEE Transactions on Single Processing), Yang et al. proposed a fast attribute scattering center extraction algorithm based on image-domain sparse representation. This method proposed and proved that scattering centers have translational and superposition properties in the image domain, and used these properties to simplify the dictionary. When solving for the sparse coefficients, they used the Newton method to optimize the parameter estimation results for continuous values, thereby obtaining fast and highly accurate attribute scattering center parameter estimation results. In their 2021 paper, "Multiscale CNN Based on Component Analysis for SAR ATR" (IEEE Transactions on Geoscience and Remote Sensing), Li et al. used an algorithm for multi-scale fusion of deep learning features and attribute scattering center features. They fused the overall attribute scattering center reconstruction image and the binarized results of eight types of attribute scattering center reconstruction images with the same deep network features. The fused features were used as features for target recognition tasks, improving the network's recognition accuracy. In their 2021 paper "A Convolutional Neural Network Combined with Attributed Scattering Centers for SAR ATR" (MDPI Remote Sensing), Zhou et al. performed 3D imaging of the scattering center model using the attribute scattering center extraction results of SAR images, fed the imaging results into a convolutional neural network for training, fused the trained features with the deep features obtained from SAR image training, and used the fused features for target recognition tasks.In their 2021 paper "Integrating the Reconstructed Scattering CenterFeature Maps With Deep CNN Feature Maps for Automatic SAR Target Recognition" (IEEE Geoscience and Remote Sensing Letters), Zhang et al. imaged each attribute scattering center extracted from the target, directly spliced the imaging results of all individual attribute scattering centers with the deep network features, used the spliced features to complete the target recognition task, and used the parameter transfer method in transfer learning when training the deep network, thereby effectively improving training efficiency.

[0005] However, among the four methods mentioned above, the first method artificially sets zeros when estimating the target attribute scattering center parameters. That is, after extracting an attribute scattering center, all pixels in the area covered by this scattering center are set to zero to ensure that the same location is not extracted repeatedly. This step increases the inaccuracy of the attribute scattering center parameter estimation results. The second and third methods are both based on the attribute scattering center extraction method of the first method. They use their own designed convolutional neural network structure for deep feature extraction of SAR images. Fully training the network requires a high time cost. In addition, the multi-scale feature fusion of the second method actually only uses the last layer of features of the deep network, which underutilizes the multi-scale features of the deep network and also affects target recognition performance. The fourth method uses a frequency domain extraction algorithm to extract attribute scattering centers. To achieve the same target reconstruction performance, a large number of attribute scattering center points need to be extracted. Moreover, the frequency domain extraction method is not only computationally inefficient but also has high memory requirements.

[0006] In summary, the accuracy and extraction efficiency of the attribute scattering center features used in existing methods have always been affected by high memory and high computational complexity, which in turn affects the target recognition performance; and the deep network used in the existing algorithm requires a large number of iterations to train a deep network with good classification performance, and its training cost is high. Summary of the Invention

[0007] In order to solve the above problems existing in the prior art, the present invention provides a SAR target recognition method based on the fusion of ASC features and multi-scale depth features. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0008] A SAR target recognition method based on the fusion of ASC features and multi-scale depth features, comprising:

[0009] Step 1: Obtain the original SAR complex image of the observed target and extract the attribute scattering center corresponding to each SAR complex image based on the improved image domain sparse representation ASC parameter estimation algorithm;

[0010] Step 2: reconstruct the image globally and locally for the attribute scattering center, and obtain global and local reconstruction images of different scales by downsampling;

[0011] Step 3: Construct a deep neural network including a feature extraction module and a feature fusion module;

[0012] The feature extraction module is used to perform multi-scale feature extraction on the amplitude image corresponding to the original SAR complex image to obtain a multi-scale depth feature map;

[0013] The feature fusion module is used to perform feature fusion at different levels on the extracted multi-scale depth feature map and the binarized image corresponding to the reconstructed image;

[0014] Step 4: Input the original SAR complex image and the binarized image corresponding to the reconstructed image into the trained deep neural network for processing, and output the target recognition result.

[0015] Beneficial effects of the present invention:

[0016] 1. This invention fully utilizes the physical properties of the target reflected by the ASC model, uses a precisely estimated ASC parameter set in the image domain to perform various types of reconstruction of the observed target, and fuses it with deep features of different scales, providing more information for the network, thereby improving target recognition performance. In addition, this method requires fewer scattering points to be extracted, is more efficient, and saves memory.

[0017] 2. This invention improves and optimizes the shortcomings of the newer image-based fast ASC extraction algorithm. By changing the way the residual image is processed when solving the sparse coefficients and the corresponding method of selecting the initial dictionary atoms, the artificial zeroing operation is eliminated during the ASC extraction process of the target. As a result, the final ASC parameter estimation result of the target is more accurate and the reconstruction error of the target is smaller.

[0018] 3. The present invention uses a parameter migration method in the deep network training process. By using the parameter results of the first 13 layers of VGG16Net fully trained on the ImageNet dataset as the initial parameters of the first 13 layers of the deep network of the present invention, the network of the present invention can not only achieve better recognition performance through fewer rounds of training, but also improve training efficiency.

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 1 is a flow chart of a SAR target recognition method based on the fusion of ASC features and multi-scale depth features provided by an embodiment of the present invention;

[0021] Figure 2 This is the flow chart of the existing fast attribute scattering center extraction algorithm based on sparse representation in image domain;

[0022] Figure 3 This is a flow chart of a fast attribute scattering center extraction algorithm for improved image domain sparse representation provided by an embodiment of the present invention;

[0023] Figure 4 Schematic diagram of the structure and forward propagation of a deep neural network provided by an embodiment of the present invention;

[0024] Figure 5 is the original SAR image of T72 in the simulation test;

[0025] Figure 6 The algorithm of the present invention is used to extract the attribute scattering center and reconstruct the image of the T72 original SAR image;

[0026] Figure 7 yes Figure 6 The binarized image corresponding to the reconstruction result in . DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0028] Example 1

[0029] See Figure 1 , Figure 1 1 is a flow chart of a SAR target recognition method based on the fusion of ASC features and multi-scale depth features provided by an embodiment of the present invention, which includes:

[0030] Step 1: Obtain the original SAR complex image of the observed target and extract the attribute scattering center corresponding to each SAR complex image based on the improved image domain sparse representation ASC parameter estimation algorithm.

[0031] The existing fast attribute scattering center extraction algorithm based on image domain sparse representation is as follows: Figure 2As shown, when estimating the target attribute scattering center parameters, an artificial zeroing operation is performed. That is, after extracting an attribute scattering center, all pixels in the area covered by this scattering center are set to zero to ensure that the same location is not extracted repeatedly. This step increases the inaccuracy of the attribute scattering center parameter estimation results. To address this problem, this embodiment improves the more recently proposed fast attribute scattering center extraction algorithm based on image domain sparse representation. Since the artificial zeroing operation in the extraction algorithm makes the parameter estimation results inaccurate, this solution eliminates the artificial zeroing operation and improves the accuracy of the attribute scattering center parameter estimation results by optimizing the selection method of the initial atoms.

[0032] In this embodiment, the improved image domain sparse representation ASC parameter estimation algorithm is used to extract the scattering centers of the SAR image S. The number of extracted scattering center points Q is set to 25. Each scattering center point corresponds to a feature vector. The feature vector corresponding to the i-th scattering center can be expressed as Among them A i represents the complex amplitude, α i represents the frequency dependence factor, x i and y i Represents the position coordinates in the range and azimuth directions respectively, L i represents the length of the scattering center, and γ i A factor representing the angular and azimuth dependence of the scattering center.

[0033] For details, see Figure 3 , Figure 3 Flowchart of the improved fast attribute scattering center extraction algorithm for image domain sparse representation provided by an embodiment of the present invention, which includes:

[0034] 11) First, the SAR echo signal is converted to the image domain. The problem that needs to be solved by the improved fast attribute scattering center extraction algorithm based on sparse representation in the image domain is still to estimate the attribute scattering center parameters of the target from the backscattered echo, that is, the number Q of attribute scattering centers that constitute a complex target and the parameter set θ of each attribute scattering center. m ,m=1,2,…,Q. This problem can be described as follows:

[0035]

[0036] Where f represents the operating frequency of the radar, represents the synthetic aperture range, f k , represent discrete f, K and H represent the number of discrete points in frequency and azimuth respectively, x and y represent the pixel coordinates after conversion to the image domain, f0 represents the center frequency, c represents the speed of light, represents the backscattered echo data of the target, represents the echo data of the qth attribute scattering center, Q represents the total number of attribute scattering centers, σ q represents the sparse coefficient, θ q represents the parameter set of the qth scattering center, ε represents the error coefficient, and ε>0, S(x,y),D(x,y;θ q ) respectively represent and corresponding image domain representation; and Represent the discretized and

[0037] Formula (1) is still a sparse representation problem. and Applying the same linear imaging operator Β{·}, the specific expression is shown in formula (2), and formula (1) can be transformed into image domain representation, as shown in formula (3):

[0038]

[0039]

[0040] Among them, S(x,y),D(x,y;θ q ) represent and The corresponding image domain representation.

[0041] 12) Use the NOMP algorithm to solve the attribute scattering center parameters of S(x,y), and add a parameter fine correction process during the solution process to obtain several attribute scattering centers.

[0042] First, an initial dictionary is established and the residual image R(x,y) is initialized, with R(x,y) = S(x,y). Then, attribute scattering center extraction is performed using the optimized NOMP algorithm. Attribute scattering center extraction for the observed target is divided into four steps: atom selection, precise estimation of atomic parameters, least squares solution, and residual calculation.

[0043] Specifically, step 12) includes:

[0044] a) Establish the initial dictionary Φ, which is expressed as:

[0045]

[0046] Where,

[0047]

[0048]

[0049] in, represents the normalized attribute scattering center image; Θ loc and Θ dis Represent the parameter sets corresponding to the local attribute scattering center and the distributed attribute scattering center respectively; Θ A ,Θ αloc ,Θ Lloc , Θ γloc ,Θ x ,Θ y The parameters A, α, L, corresponding to the local attribute scattering center, γ,x,y, where A represents the complex amplitude, α represents the frequency dependence factor, x and y represent the position coordinates in the range and azimuth directions respectively, and L represents the length of the scattering center. and γ represent the directional angle and azimuth dependence factor of the scattering center, respectively; Θ A ,Θ αdis ,Θ Ldis , Θ γdis ,Θ x ,Θ y The parameters A, α, L, corresponding to the distributed attribute scattering center, The specific meaning of each parameter γ, x, y is the same as above, and "×" represents the Cartesian product.

[0050] When building the initial dictionary Φ, let Θ A ={1},Θ αloc ={0},Θ αdis ={0},Θ Lloc ={0},Θ Ldis ={2ΔL,4ΔL,...,2N L ΔL}, Θ γloc ={0},Θ γdis ={0}, This is because the values of the frequency dependence factor α and the azimuth dependence factor γ have little effect on the attribute scattering center echo signal and can be temporarily ignored. αloc ,Θ αdis ,Θ γloc Set to {0}. At this point, the initialization dictionary Φ is established.

[0051] b) Select atoms from the initial dictionary Φ and select the atom θ that has the greatest similarity with the current residual image R(x,y)i_chose As the rough estimation result of the current i-th ASC parameter.

[0052] Specifically, for a given initial dictionary Φ, the first step of the NOMP algorithm is to select the atom from Φ that best matches the residual image R(x,y) (with the largest inner product), as shown in the following formula:

[0053]

[0054] in,(·) * Represents the conjugate operation.

[0055] c) Determine the currently selected atom θ i_chose Are the position parameters of the last selected atom the same? If they are the same, abandon the currently selected atom θ i_chose , and reselect to remove the θ i_chose The atom with the largest similarity after the calculation is taken as the rough estimation result of this round; otherwise, execute step d).

[0056] Specifically, this embodiment optimizes the atom selection method and records the currently selected dictionary atom as θ i_chose , that is, θ i_chose =θ i , the dictionary atom selected in the last step b) is recorded as θ i_last , compare θ i_chose and θ i_last Are they the same atoms? If θ i_last ≠θ i_chose , then continue with the following steps to estimate parameters; if θ i_last =θ i_chose , then abandon the atom with the largest current similarity and select and remove θ i_last The atom with the largest similarity after the next step is taken as the θ of this round. i_chose .

[0057] d) The rough estimate θ i_chose As the initial point, the i-th ASC parameter is accurately estimated to obtain the accurate estimation result θ i,opt , and put it into the selected atom set Φ Gen middle.

[0058] Specifically, since the values of most attribute scattering center parameters are continuous, the parameters obtained in step b) are inaccurate and need to be further estimated. Therefore, the rough estimation result θ of the current i-th ASC parameter obtained in step b) is i_chose As a starting point, use Newton's method to solve the following equation:

[0059]

[0060] Where θ / A represents the set of parameters other than parameter A.

[0061] Note Φ Gen is the parameter θ i,opt The corresponding attribute scattering center image set and initialize That is:

[0062]

[0063] e) Using Φ Gen The atoms in are used to approximate the input image S(x,y) and the sparse coefficients are solved using the least squares method.

[0064] Specifically, using the least squares estimation, Φ Gen The atoms in are used to approximate the input S(x,y), as shown below:

[0065]

[0066] in Represents the coefficient group corresponding to each atom, Represents the use of Φ Gen To approximate the optimal coefficients of the input S(x,y).

[0067] f) Update the residual image according to the sparse coefficients.

[0068] Specifically, the update formula is:

[0069]

[0070] g) Repeat steps b) to f) until no valid attribute scattering centers can be extracted from the current residual image R(x, y), and exit the loop.

[0071] At this point, the extraction of the attribute scattering center of the observed target using the improved algorithm is completed.

[0072] Preferably, in this embodiment, 25 attribute scattering centers are extracted from each SAR complex image using the above method.

[0073] The present invention improves and optimizes the shortcomings of the recently proposed fast ASC extraction algorithm based on the image domain. By changing the way the residual image is processed when solving the sparse coefficients and the way the initial dictionary atoms are selected, the artificial zeroing operation is eliminated during the ASC extraction process of the target. Therefore, the final ASC parameter estimation result of the target is more accurate and the reconstruction error of the target is smaller.

[0074] Step 2: Perform global and local image reconstruction on the attribute scattering center, and obtain global and local reconstruction images of different scales by downsampling.

[0075] 21) Substitute the attribute scattering center parameters of the observed target obtained in step 1 into the definition expression of the attribute scattering center model to obtain the backscattered echo data of the entire target and the backscattered echo data of the individual attribute scattering center.

[0076] Specifically, according to the definition of the attribute scattering center model, the backscattered echo of a target in the high-frequency region can be regarded as the superposition of many independent scattering point echoes, as shown in the following formula:

[0077]

[0078] in, Indicates that the signal is a frequency domain signal; Represents the set of all parameters, from left to right, representing the backscattering coefficient, frequency dependence factor, length, tilt angle, azimuth dependence factor, range coordinate and azimuth coordinate; represents the backscattered echo signal of the observed target; Q represents the total number of attribute scattering centers that make up the current complex target; θ q Represents the parameter set of the qth attribute scattering center.

[0079] in, Represents the echo signal of the qth individual attribute scattering center; represents additive white Gaussian noise; where The specific representation is as follows:

[0080]

[0081] in, sinc(·)=sin(·) / (·), f0 represents the center frequency of the radar; c represents the speed of light; A q represents the backscattering coefficient; α q represents the frequency dependence factor; x q with y q Represents the position coordinates in the range and azimuth directions respectively; L q , γ q The three parameters describe the length, tilt angle and azimuth dependence factor of the attribute scattering center respectively.

[0082] Substituting the estimated results of the scattering center parameters of the observed target into formula (13) can obtain the backscattered echo of a single scattering point: Combined with formula (12), we can obtain the target backscatter echo reconstructed using the extracted attribute scattering center result: Then, the linear imaging operator Β{·} is applied to the frequency domain echo to obtain the reconstructed image S(x,y).

[0083] 22) Applying linear imaging operators to the backscatter echo data of the entire target and the backscatter echo data of the individual attribute scattering centers, respectively, to obtain a global reconstruction image and several local reconstruction images of individual scattering points;

[0084] The size of the global reconstructed image and the size of the local reconstructed image are consistent with the size of the original SAR complex image, both of which are 128×128. Since there are 25 attribute scattering centers extracted in this embodiment, there are also 25 local reconstructed images obtained.

[0085] 23) Downsample the global reconstruction image of size 128×128 to obtain the global reconstruction image S of size 64×64. recon_all_64 and the global reconstruction map S of size 32×32 recon_all_32 , downsample the local reconstruction image of size 128×128 to obtain a local reconstruction image S of size 64×64 recon_single_64 .

[0086] It is understandable that this embodiment also requires setting corresponding thresholds for the three images obtained after the downsampling operation to obtain three types of binarized images. The specific steps are as follows:

[0087] For the global reconstruction graph S recon_all_64 , set the threshold t = 0.01, and S recon_all_64 The values of all pixels on the image are compared with the threshold t. The pixel values greater than the threshold t are set to 255, and the pixel values less than the threshold t are set to 0, and the binary image B is obtained. recon_all_64 .

[0088] For the 25 local reconstruction images S recon_single_64 , set the threshold t = 0.01, and perform the 25 S recon_single_64 Perform binary processing. recon_single_64 All pixel values on the image are compared with the threshold t. Pixels with values greater than the threshold t are set to 255, and pixels with values less than the threshold t are set to 0. 25 binary images B are obtained. recon_single_64 .

[0089] For the global reconstruction graph S recon_all_32 , set the threshold t = 0.01, and S recon_all_32 The values of all pixels on the image are compared with the threshold t. The pixel values greater than the threshold t are set to 255, and the pixel values less than the threshold t are set to 0, and the binary image B is obtained. recon_all_32 .

[0090] Step 3: Construct a deep neural network including a feature extraction module and a feature fusion module;

[0091] The feature extraction module is used to extract multi-scale features from the amplitude image corresponding to the original complex SAR image to obtain a multi-scale depth feature map;

[0092] The feature fusion module is used to fuse features at different levels between the extracted multi-scale depth feature map and the binary image corresponding to the reconstructed image.

[0093] First, construct the feature extraction module.

[0094] In this embodiment, the constructed feature extraction module includes 12 convolutional layers and 3 maximum pooling layers, and its structure is as follows: the first convolutional layer L C1 , the second convolutional layer L C2 , the third maximum pooling layer L p3 , the fourth convolutional layer L C4 , the fifth convolutional layer L C5 , the sixth maximum pooling layer L p6 , the seventh convolutional layer L C7 , the eighth convolutional layer L C8 , the ninth convolutional layer L C9 , the tenth maximum pooling layer L p10 , the eleventh convolutional layer L C11 , the twelfth convolutional layer L C12 , the thirteenth convolutional layer L C13 , the fourteenth convolutional layer L C14 , the fifteenth convolutional layer L C15 ; Among them, the first 13 layers of the network structure are the same as the first 13 layers of the VGG16Net network.

[0095] Among them, the fifth convolutional layer L C5 The output deep global features are used as the first-scale deep features to be fused by the feature extraction module;

[0096] The ninth convolutional layer L C9 The output deep global features are used as the second-scale deep features to be fused by the feature extraction module;

[0097] The fifteenth convolutional layer L C15 The output deep global features are used as the third-scale deep features to be fused by the feature extraction module.

[0098] Specifically, the parameters of each layer are set as follows: the number of convolution kernels of the 12 convolutional layers are set to 64, 64, 128, 128, 256, 256, 512, 512, 512, 256, and 32 respectively, the convolution kernel size is set to 3×3, the convolution kernel step size is set to 1, and the activation function uses the ReLU activation function; the kernel size of the 3 maximum pooling layers is 2×2, and the step size is set to 2.

[0099] Since the network structure of the first 13 layers of the feature extraction module constructed in this embodiment is exactly the same as the VGG16Net network structure, when the network is subsequently trained, the parameter results trained on the ImageNet dataset can be used as the initial parameters of the first 13 layers of the feature extraction network of this embodiment.

[0100] Then, a feature fusion module is constructed.

[0101] In this embodiment, the feature fusion module includes a local feature map level fusion unit and an overall feature map level fusion unit; wherein,

[0102] The local feature map level fusion unit fuses the first-scale depth features to be fused and the second-scale depth features to be fused extracted by the feature extraction module with the binary images corresponding to the three different reconstructed images in multiple times, and obtains three fused feature maps accordingly;

[0103] The overall feature map level fusion unit is used to fuse the three fused feature maps with the third-scale depth features to be fused extracted by the feature extraction module to obtain the final fused features.

[0104] Specifically, if Figure 4 As shown, for local feature fusion, this embodiment performs three times, namely, the 5th layer of deep global features and the 25 binary images B reconstructed by a single scattering point. recon_single_64 Fusion, deep global features of the 5th layer and the overall reconstructed binary image B recon_all_64 Fusion, deep global features of the 9th layer and the overall reconstructed binary image B recon_all_32 Fusion. The specific operations are as follows:

[0105] Step 1: First fusion:

[0106] Binary image B reconstructed from a single scattering point recon_single_64 The size is 64×64×25, which is multiplied with the deep global feature of the 5th layer of the deep feature extraction module with a size of 64×64×128 along the channel dimension to obtain a fusion feature of 64×64×25×128, and then a global average pooling (GAP) operation is performed on it to obtain a 25×128-dimensional feature.

[0107] In order to compress these 25 component features into one vector, statistical functions can be used at the corresponding positions of the 25 vectors. Considering the representativeness and computational complexity, this embodiment uses the max(·) and mean(·) statistical functions to complete the above vector compression process, as shown in formula (14):

[0108] C(·)=max(·)+mean(·) (14)

[0109] Here, C(·) represents the fusion method used, that is, max(·) and mean(·) are used at the corresponding positions of the 25 local component feature vectors, and then the results are added together to finally obtain a 1×128-dimensional feature, which is the first fusion feature.

[0110] Step 2 Second fusion:

[0111] Reconstruct the binary image B with a size of 64×64 recon_all_64 Multiply it with the global feature of the 5th layer of the deep feature extraction module with a size of 64×64×128 along the channel dimension to obtain a 64×64×128 fusion feature, and then perform a GAP operation on it to obtain a 1×128-dimensional feature, which is the second fusion feature.

[0112] Step 3 The third fusion:

[0113] Reconstruct the binary image B with a size of 32×32 recon_all_32 Multiply it with the global feature of the 9th layer of the deep feature extraction module with a size of 32×32×256 along the channel dimension to obtain a 32×32×256 fusion feature, and then perform a GAP operation on it to obtain a 1×256-dimensional feature, which is the third fusion feature.

[0114] For the overall feature map level, this embodiment integrates a total of four features, namely, the depth feature extraction module finally outputs the depth feature with a size of 16×16×32, which is then subjected to the GAP operation to obtain a global network feature of size 1×32 (that is, the third-scale depth feature to be fused), the first fusion feature of 1×128 dimensions generated by the first fusion in the local feature fusion, the second fusion feature of 1×128 dimensions generated by the second fusion in the local feature fusion, and the third fusion feature of 1×256 dimensions generated by the third fusion in the local feature fusion. These four features are spliced along the first dimension to obtain an overall fusion feature of size 1×544.

[0115] Finally, a fully connected network is constructed to classify the final fusion features and obtain the target classification results.

[0116] Among them, the fully connected network FC includes two fully connected layers, one activation layer, one Dropout layer and one classifier layer. Its structures are as follows: the first fully connected layer L F1 , the second activation layer L F2 , the third Dropout layer L d3 , the fourth fully connected layer L F4 , the fifth classification layer L F5 ; The input of the network is the fused 544-dimensional feature vector, and the output of the network is a 3-dimensional category prediction vector

[0117] The parameters of each layer are set as follows: the dimensions of the two fully connected layers are 544×512 and 512×3; the activation layer uses the ReLU activation function; the dropout probability of the Dropout layer is 0.5; and the classifier layer uses the softmax classifier.

[0118] Press this part Figure 4 The sequence shown is combined together to obtain the deep neural network Ψ.

[0119] It is understandable that after building a deep neural network, the network needs to be trained and then used to perform target recognition.

[0120] In this embodiment, the deep neural network is trained in the following manner:

[0121] The attribute scattering centers are extracted from the labeled measured SAR complex images, and global and local reconstructions are performed. Then, global and local reconstruction images of different scales are obtained by downsampling.

[0122] The labeled SAR complex image and the binary images corresponding to the global and local reconstruction images of different scales are input into the constructed deep neural network for forward propagation, such as Figure 4 As shown; at the same time, the parameter results trained on the ImageNet dataset are used as the initial parameters of the first 13 layers in the feature extraction module;

[0123] Calculate the classification loss and update the network parameters through back propagation to obtain a trained network. The classification loss uses the cross entropy loss function as shown below:

[0124]

[0125] Among them, n represents the number of training samples, y i represents the category label of the i-th input image in one-hot encoding form, Represents the corresponding predicted category label.

[0126] This embodiment fully utilizes the physical properties of the target reflected by the ASC model, uses a precisely estimated set of ASC parameters in the image domain to perform various types of reconstructions of the observed target, and fuses them with deep features at different scales, providing more information for the network and thus improving target recognition performance. This method also requires fewer scattering points to be extracted, is more efficient, and saves memory.

[0127] Step 4: Input the original complex SAR image of the test data and the binary image corresponding to the reconstructed image into the trained deep neural network for processing, and output the target recognition result.

[0128] The present invention adopts a parameter migration method in the deep network training process. By using the first 13 layers of VGG16Net parameters fully trained on the ImageNet dataset as the initial parameters of the deep network of the present invention, the network of the present invention can not only achieve better recognition performance with fewer training times, but also improve training efficiency.

[0129] Example 2

[0130] The method proposed in the present invention is simulated below using a specific scenario as an example to verify the effectiveness of the attribute scattering center extraction algorithm in the present invention.

[0131] Specifically, this experiment Figure 5 The original T72 SAR image shown in FIG is used as the extraction object, and the attribute scattering center extraction and image reconstruction are performed using the method of the present invention. The results are shown in FIG. Figure 6 and 7 shown.

[0132] Figure 6 The results of attribute scattering center extraction and image reconstruction of the T72 original SAR image using the algorithm of the present invention are shown in Figure 1. (a) is a global reconstruction image of 128 pixels reconstructed using the extracted 25 scattering center points, (b) is a global reconstruction image of 64 pixels obtained after downsampling, (c) is a local reconstruction image of 64 pixels obtained after downsampling, and (d) is a global reconstruction image of 32 pixels obtained after downsampling.

[0133] Figure 7 yes Figure 6 The binary images corresponding to the reconstruction results in . Among them, (a) is the global reconstructed binary image with a size of 64 pixels obtained after the downsampling operation, (b) is the local reconstructed binary image with a size of 64 pixels obtained after the downsampling operation, and (c) is the global reconstructed binary image with a size of 32 pixels obtained after the downsampling operation.

[0134] In order to further verify the SAR target recognition method based on the fusion of ASC features and multi-scale depth features proposed in the present invention, this embodiment also performs recognition on the public MSTAR dataset of moving and stationary targets.

[0135] The MSTAR dataset used in this experiment consists of complex images with a resolution of 0.3m × 0.3m and a size of 128 × 128 pixels. This experiment uses the MSTAR dataset for three-category target recognition scenarios. The three target data types are T72, BMP2, and BTR70. Both BMP2 and T72 contain three different sequence numbers. The training data for each class contains only one sequence number of each type, while the test data contains all sequence numbers of each type. The specific experimental data settings are shown in Table 1.

[0136] Table 1 MSTAR three-category target recognition scenarios

[0137]

[0138] Table 2 below gives the recognition results of the method of the present invention on the MSTAR three-category target recognition data shown in Table 1 above, and compares them with the existing recognition method, the SAR ATR method combining attribute scattering center and convolutional neural network (abbreviated as ACNNC, from the article A Convolutional Neural Network Combined with Attributed Scattering Centers for SAR ATR, IEEE Transactions on Geoscience and Remote Sensing, Zhou Y, 2021), and the multiscale SAR ATR convolutional neural network based on component analysis (abbreviated as CA-MCNN, from the article Multiscale CNN Based on Component Analysis for SAR ATR, IEEE Transactions on Geoscience and Remote Sensing, Li Y, 2021).

[0139] Table 2 Detailed recognition results of different recognition methods on MSTAR 3-category target data

[0140] How to use Recognition accuracy The method proposed by the present invention 0.9890 ACNNC 0.9795 CA-MVCC 0.9861

[0141] Due to the problem of insufficient training data in SAR images, the small sample problem is very prominent in SAR image recognition. To further verify the effectiveness of the present invention, a small sample experiment was conducted on the three types of MSTAR target data shown in Table 1. A certain proportion of training samples were randomly selected to simulate the small sample experimental conditions, and the average result of 10 experiments was selected as the recognition result. It is compared with existing small-sample recognition methods: SAR target recognition method based on limited training data of angular rotation generative network (ARGN, from the article SAR Target Recognition With Limited Training Data Based on Angular Rotation Generative Network, IEEE Geoscience and Remote Sensing Letters, Sun Y, 2019), SAR ATR method based on improved polar coordinate mapping classifier (M-PMC, from the article Modified Polar Mapping Classifier for SAR Automatic Target Recognition, IEEE Transactions on Aerospace and Electronic Systems, Park J, 2014), SAR target recognition method based on data augmentation convolutional neural network (DA-CNN, from the article Convolutional Neural Network with Data Augmentation for SAR Target Recognition, IEEE Geoscience and Remote Sensing Letters, Ding J, 2016), and SAR image classification method based on deep convolutional network (A-ConvNet, from the article Target Classification using the Deep Convolutional Networks for SAR images, IEEE Transactions on Geoscience and Remote Sensing, Chen S, 2016). The recognition results of the present invention and the above method in a small sample environment are shown in Table 3.

[0142] Table 3 Comparison of recognition performance between the proposed method and some existing methods in a small sample environment

[0143]

[0144]

[0145] The results in Table 3 above were all trained by randomly selecting training samples of corresponding proportions. The sample ratio represents the ratio of the number of randomly selected samples to the number of all training samples. Ten experiments were conducted for each sample ratio value, and the average value of the 10 experiments was taken as the final recognition result of the sample ratio. It can be seen that when only less than half of the training samples are used, the recognition accuracy of the method of the present invention is better than that of other comparison methods. This effectively proves the effectiveness of the model of the present invention when the training data is insufficient. When the sample ratio is 0.1, that is, when there are only 22 samples in each category of training data, the average recognition accuracy of the 1,365 test samples reaches 89.46%.

[0146] The above experimental results show that in the recognition experiments of three types of MSTAR target data, the proposed method achieved good recognition results under both the full training sample experimental conditions and the small sample experimental conditions of less than 50%. This verifies the effectiveness of the proposed method and further demonstrates that the proposed method can better utilize global and local information to obtain effective and robust target feature representation, showing its certain effectiveness and feasibility.

[0147] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A SAR target recognition method based on the fusion of ASC features and multi-scale depth features, characterized in that: include: Step 1: Obtain the original SAR complex image of the observed target and extract the attribute scattering center corresponding to each SAR complex image based on the improved image domain sparse representation ASC parameter estimation algorithm; 11) Target backscatter echo data in the original SAR complex image data and the echo data of the qth attribute scattering center Apply the same linear imaging operator and estimate its attribute scattering center parameters; where, Respectively represent discrete f represents the operating frequency of the radar, represents the synthetic aperture range, θ q represents the parameter set of the qth scattering center; 12) Use NOMP algorithm to solve the attribute scattering center parameters of S(x,y), and add parameter fine correction process in the solution process to obtain several attribute scattering centers; where S(x,y) represents The corresponding image domain representation, x, y respectively represent the pixel coordinates after conversion to the image domain; a) Establish the initial dictionary Φ; at the same time, initialize the residual image R(x,y) and set R(x,y) = S(x,y); b) Perform atom selection on the initial dictionary Φ and select the atom θ with the greatest similarity to the current residual image R(x,y) i_chose As the rough estimation result of the current i-th ASC parameter; c) Determine the currently selected atom θ i_chose Are the position parameters of the last selected atom the same? If they are the same, abandon the currently selected atom θ i_chose , and reselect to remove the θ i_chose The atom with the largest similarity after the calculation is taken as the rough estimation result of this round; otherwise, execute step d); d) Using the rough estimation result as the starting point, perform a precise estimation on the i-th ASC parameter, obtain the precise estimation result and add it to the selected atomic set Φ Gen middle; e) using the selected atomic set Φ Gen All atoms in are used to approximate the input image S(x,y) and the sparse coefficients are solved using the least squares method; f) updating the residual image according to the sparse coefficients; g) Repeat steps b) to f) until no valid attribute scattering centers can be extracted from the current residual image R(x, y), and then exit the loop; Step 2: reconstruct the image globally and locally for the attribute scattering center, and obtain global and local reconstruction images of different scales by downsampling; Step 3: Construct a deep neural network including a feature extraction module and a feature fusion module; The feature extraction module is used to perform multi-scale feature extraction on the amplitude image corresponding to the original SAR complex image to obtain a multi-scale depth feature map; The feature fusion module is used to perform feature fusion at different levels on the extracted multi-scale depth feature map and the binarized image corresponding to the reconstructed image; Step 4: Input the original SAR complex image and the binarized image corresponding to the reconstructed image into the trained deep neural network for processing, and output the target recognition result.

2. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 1 is characterized in that: In step 11), the linear imaging operator is expressed as: The attribute scattering center parameter is expressed as: Among them, K and H represent the number of discrete points in frequency and azimuth respectively, f0 represents the center frequency, c represents the speed of light, Q represents the total number of attribute scattering centers, σ q represents the sparse coefficient, ε represents the error coefficient, and ε>0, D(x,y;θ q )express The corresponding image domain representation.

3. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 1 is characterized in that: In step 1, 25 attribute scattering centers are extracted for each SAR complex image.

4. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 1 is characterized in that: Step 2 includes: 21) Substituting the attribute scattering center parameters of the observed target obtained in step 1 into the definition expression of the attribute scattering center model to obtain the backscattered echo data of the entire target and the backscattered echo data of the individual attribute scattering centers; 22) applying the same linear imaging operator to the backscatter echo data of the entire target and the backscatter echo data of the individual attribute scattering centers, respectively, to obtain a global reconstruction image and a plurality of local reconstruction images of the individual scattering points; The size of the global reconstructed image and the size of the local reconstructed image are consistent with the size of the original SAR complex image, both of which are 128×128; 23) Down-sampling the global reconstruction image to obtain a global reconstruction image S of size 64×64. recon_all_64 and the global reconstruction map S of size 32×32 recon_all_32 , downsample the local reconstruction image to obtain a local reconstruction image S of size 64×64 recon_single_64 .

5. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 4 is characterized in that: In step 3, the feature extraction module constructed includes 12 convolutional layers and 3 maximum pooling layers, and its structure is as follows: the first convolutional layer L C1 , the second convolutional layer L C2 , the third maximum pooling layer L p3 , the fourth convolutional layer L C4 , the fifth convolutional layer L C5 , the sixth maximum pooling layer L p6 , the seventh convolutional layer L C7 , the eighth convolutional layer L C8 , the ninth convolutional layer L C9 , the tenth maximum pooling layer L p10 , the eleventh convolutional layer L C11 , the twelfth convolutional layer L C12 , the thirteenth convolutional layer L C13 , the fourteenth convolutional layer L C14 , the fifteenth convolutional layer L C15 ; Among them, the first 13 layers of network structure are the same as the first 13 layers of VGG16Net; Among them, the fifth convolutional layer L C5 The output deep global features are used as the first-scale deep features to be fused by the feature extraction module; The ninth convolutional layer L C9 The output deep global features are used as the second-scale deep features to be fused by the feature extraction module; The fifteenth convolutional layer L C15 The output deep global features are used as the third-scale deep features to be fused by the feature extraction module.

6. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 5 is characterized in that: In step 3, the constructed feature fusion module includes a local feature map level fusion unit and an overall feature map level fusion unit; wherein, The local feature map layer fusion unit fuses the first-scale depth features to be fused and the second-scale depth features to be fused extracted by the feature extraction module with the binarized images corresponding to the three different reconstructed images multiple times to obtain three fused feature maps; The overall feature map level fusion unit is used to fuse the three fused feature maps with the third-scale depth feature to be fused extracted by the feature extraction module to obtain the final fused feature.

7. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 6 is characterized in that: The local feature map layer fusion unit fuses the first-scale depth features to be fused and the second-scale depth features to be fused extracted by the feature extraction module with the binary images corresponding to different reconstructed images three times to obtain three fused feature maps, including: The local reconstruction image S of the individual scattering point recon_single_64 Multiply the corresponding binary image and the first-scale depth feature to be fused along the channel dimension, and then perform a GAP operation on the image to obtain a first fused feature map; The global reconstruction graph S recon_all_64 Multiply the corresponding binary image and the first-scale depth feature to be fused along the channel dimension, and then perform a GAP operation on it to obtain a second fused feature map; The global reconstruction graph S recon_all_32 The corresponding binary image is multiplied by the second-scale depth feature to be fused along the channel dimension, and then a GAP operation is performed on it to obtain a third fused feature map.

8. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 7 is characterized in that: In step 3, it also includes: Constructing a fully connected network to classify the final fusion features to obtain a target classification result; The fully connected network includes two fully connected layers, one activation layer, one Dropout layer and one classifier layer, and its structure is as follows: the first fully connected layer L F1 , the second activation layer L F2 , the third Dropout layer L d3 , the fourth fully connected layer L F4 , the fifth classification layer L F5 ; The input of the fully connected network is the feature vector corresponding to the final fused image output by the feature fusion module, and the output is a 3D category prediction vector.

9. The SAR target recognition method based on the fusion of ASC features and multi-scale depth features according to claim 8, characterized in that: The deep neural network is trained in the following way: Extract attribute scattering centers from labeled measured SAR complex images and reconstruct them to obtain global and local reconstruction images at different scales. Input the labeled SAR complex image and the binarized images corresponding to the global and local reconstruction images at different scales into the constructed deep neural network for forward propagation; at the same time, use the parameter results trained on the ImageNet dataset as the initial parameters of the first 13 layers in the feature extraction module; Calculate the classification loss and update the network parameters through back propagation to obtain a trained network.

Images