Target identification method for improving image translation invariance of synthetic aperture radar
By introducing a pooling method based on SigmaPool in synthetic aperture radar (SAR) target recognition, the problem of decreasing recognition accuracy during image translation is solved, and higher translation invariance and robustness are achieved.
Patent Information
- Application Number
- CN202411626359.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-03-11
AI Technical Summary
The existing synthetic aperture radar (SAR) target recognition methods have decreased the recognition accuracy during image translation, especially in images containing coherent spot noise, and it is difficult to maintain translation invariance.
A pooling method based on SigmaPool principle is proposed. By setting the pooling convolution kernel parameters, the threshold value of the activation area in the feature map is calculated, whether the activation area is an isolated scatterer, and corresponding pooling calculation is performed to improve the translation invariance of the image.
It effectively alleviates the contradiction between high-frequency information and coherent speckle noise, improves the translation invariance of convolutional neural networks, and enhances the robustness of SAR image target recognition.
Smart Images
Figure CN119600255A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of synthetic aperture radar image target recognition, and in particular to a target recognition method for improving the translation invariance of synthetic aperture radar images. Background Art
[0002] In recent years, convolutional neural networks have achieved remarkable results in the field of SAR (synthetic aperture radar) target recognition, even surpassing human recognition capabilities. It is generally believed that CNN (convolutional neural network) is translationally invariant because a large number of convolution operations are used in the CNN (convolutional neural network), and the shift filter of the convolution kernel will map the features of the previous layer to the features of the next layer and keep the relative position macroscopically unchanged. However, in the past two years, some papers have pointed out that a slight translation of the target will have a great impact on the recognition accuracy of the target. The convolution operation in the neural network is translationally invariant (translation robustness), and it is the pooling operation that really makes the neural network lose its translation invariance.
[0003] Taking max pooling as an example, max pooling essentially preserves high-frequency information in the image, which meets the requirements of feature extraction in recognition problems, but max pooling cannot resist aliasing, and often small pixel translation will cause large fluctuations in the results, especially for SAR images containing coherent speckles. Pooling is essentially a downsampling process, which is similar to traditional filtering, but the physical process of coherent speckle noise formation in SAR images is fundamentally different from the process of noise formation in conventional images, which also leads to the inability to treat these two types of noise equivalently during image processing.
[0004] At present, some studies have attempted to enhance the translation robustness of the network by improving the pooling method or introducing other mechanisms (such as attention mechanism). In addition, some studies focus on designing a special architecture to achieve better translation invariance, such as using Steerable CNNs and other technologies. However, the current methods have more or less problems such as not considering the target translation problem caused by the actual complex environment, large amount of calculation, complex design, and not considering the physical process of the formation of coherent speckle noise in SAR images is essentially different from the formation process of noise in conventional images.
[0005] The statements herein merely provide background art related to the present invention and do not necessarily constitute prior art. Summary of the invention
[0006] The purpose of the present invention is to propose a target recognition method for improving the translation invariance of synthetic aperture radar images, which can alleviate the contradiction between high-frequency information and coherent speckle noise, and at the same time alleviate the problem that the network does not have translation invariance.
[0007] To achieve the above object, the present invention proposes a target recognition method for improving the translation invariance of synthetic aperture radar images, comprising the following steps:
[0008] S1. For the feature map, set the pooling convolution kernel parameters and calculate the threshold of the activation area in the feature map according to the SigmaPool principle;
[0009] S2, comparing each feature value in the activation area with the threshold of the activation area in turn to obtain a template of the activation area;
[0010] S3. Based on the template obtained in S2, a template judgment threshold is introduced to judge whether the current activation area is an isolated scatterer, and the corresponding pooling calculation is performed to obtain the pooling output.
[0011] Preferably, the pooling method of steps S1 to S3 is used as a pooling module and inserted into a convolutional neural network for target recognition, and the translation invariance measurement method is used to judge the recognition result to determine the improvement of SigmaPool on the translation invariance of the convolutional neural network, and the translation invariance measurement map is used to estimate the azimuth range of the target.
[0012] Preferably, step S1 comprises the following steps:
[0013] S11. Given an S×S feature map, set the pooling convolution kernel parameters and determine the activation area;
[0014] S12. According to the SigmaPool principle, calculate the average value of all features in the activation area and the standard deviation σ of the activation area ij ;
[0015] S13, based on the average value of the activated area and standard deviation σ ij , calculate the threshold of the activation area.
[0016] Preferably, the average value of all features in the activation area is and the standard deviation σ of the activation area ij for
[0017]
[0018] Among them, k and l are the coordinate positions of the feature points in the activation area, L is the convolution kernel size of the activation area, k≤L, l≤L, and g(k,l) is the feature value in the activation area.
[0019] Preferably, the threshold of the activation area is a low threshold and high threshold
[0020] Preferably, in S2, the formula followed for comparing each characteristic value in the activation area with the threshold value of the activation area in turn is:
[0021]
[0022] Preferably, in S3, the template judgment threshold K=(L+1) / 2, when the characteristic value of the activation area falls within 2σ F If the number of features in the range is less than or equal to the template judgment threshold K, the activated area is judged as an isolated scatterer, and the average value of the feature values in the activated area is used as the pooling output; when the feature value of the activated area falls within 2σ F If the number within the range is greater than the template judgment threshold K, the activated area is weighted and calculated as the pooling output.
[0023] Preferably, cosine similarity is used to measure translation invariance, and the formula is as follows:
[0024]
[0025] in The feature map output by a certain layer of the network indicates that the target is at the exact center of the image.
[0026] Represents the output feature map of the target translated image at a certain layer.
[0027] Compared with the prior art, the technical solution of the present invention has the following advantages and beneficial effects:
[0028] This scheme proposes a target recognition method for improving the translation invariance of synthetic aperture radar images, comprising the steps of: setting pooling convolution kernel parameters, and calculating the threshold of the activation area in the feature map according to the SigmaPool principle; comparing each characteristic value in the activation area with the threshold of the activation area in turn to obtain a template of the activation area; based on the obtained template, introducing a template judgment threshold to judge whether the current activation area is an isolated scatterer, and performing corresponding pooling calculation to obtain a pooling output; the method of the invention designs a new pooling method based on the traditional SAR filtering method to alleviate the contradiction between high-frequency information and coherent speckle noise, and at the same time alleviates the problem that the network does not have translation invariance. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A flow chart of a target recognition method for improving the translation invariance of synthetic aperture radar images according to the present invention;
[0030] Figure 2 is a feature map and an activation area therein;
[0031] Figure 3is a feature map and another activation area therein;
[0032] Figure 4 for Figure 2 The activation value corresponding to the activation area in ;
[0033] Figure 5 for Figure 3 The activation value corresponding to another activated area in ;
[0034] Figure 6 Images of the test set under different translations;
[0035] Figure 7 This is the translation-prediction probability change map based on A-ConvNets;
[0036] Figure 8 is the translation invariance metric map based on VGG19;
[0037] Fig. 9 It is the translation invariance metric diagram for different azimuth angles. DETAILED DESCRIPTION
[0038] The technical solutions, structural features, achieved objectives and effects of the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0039] It should be noted that the drawings are in a very simplified form and use non-precise proportions. They are only used to conveniently and clearly assist in explaining the embodiments of the present invention, and are not used to limit the conditions for the implementation of the present invention. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical content disclosed by the present invention.
[0040] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only the elements explicitly listed, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0041] The present invention proposes a target recognition method for improving the translation invariance of synthetic aperture radar images. The method uses a convolutional neural network based on SigmaPool to perform target recognition. Figure 1 As shown, the following steps are included:
[0042] S1. For the feature map, set the pooling convolution kernel parameters and calculate the threshold of the activation area in the feature map according to the SigmaPool principle;
[0043] S11. Given an S×S feature map, set the pooling convolution kernel parameters, that is, set the convolution kernel parameters of the activation area to be detected to L×L, and the stride to z, where L is the convolution kernel size, L≤S, z≤SL;
[0044] S12. According to the SigmaPool principle, calculate the average value of all features in the activation area and the standard deviation σ of the activation area ij , specifically:
[0045]
[0046] In the above formula, k and l are the coordinate positions of the feature points in the activation area, k≤L, l≤L, and g(k,l) is the feature value in the activation area;
[0047] S13, based on the average value of the activated area and standard deviation σ ij , calculate the threshold of the activation area;
[0048] The threshold of the activation area is the low threshold and high threshold
[0049] S2. Compare each feature value in the activation area with the threshold of the activation area in turn to obtain a template of the activation area; wherein the comparison is based on the formula:
[0050]
[0051] The above formula means that each feature value in the activation area is compared with the threshold of the activation area, and those within the threshold range are set to 1, and those outside the threshold range are set to 0. Through the above comparison, the template M of the activation area is obtained.
[0052] In the above process, the reason why and As the upper and lower limits of the threshold, it is because the random samples of the Gaussian distribution fall within 2σ F The probability of being within the interval is 93.5%, so SigmaPool only activates the gray value 2σ of the central pixel in the activation area F The pixels in the range are averaged and the average value is used as the feature value after pooling, while 2σ FFeature points outside the range are regarded as edge parts and do not participate in the pooling process. This processing method can distinguish high-frequency and low-frequency signals to some extent.
[0053] S3, based on the template M obtained in S2, introduce a template judgment threshold to determine whether the current activation area is an isolated scatterer, and perform corresponding pooling calculations;
[0054] Introduce the template judgment threshold K, K = (L + 1) / 2, L is the size of the convolution kernel; when the eigenvalue of the activated area falls within 2σ F If the number of 1s in the range (i.e., 1 in the template M) is less than or equal to the template judgment threshold K, the activated area is judged to be an isolated scatterer, and the average value of the eigenvalues in the activated area is used as the pooling output, which is the commonly used average pooling; when the eigenvalue of the activated area falls within 2σ F If the number of 1s in the range (i.e., 1 in the template M) is greater than the template judgment threshold K, the activation area is weighted and the activation value R is obtained as the pooling output. The specific calculation formula is:
[0055]
[0056] S4. The pooling method obtained in steps S1 to S3 is used as a pooling module, inserted into the convolutional neural network for target recognition, and the translation invariance measurement method is used to judge the recognition result to determine the improvement of the translation invariance of the convolutional neural network by SigmaPool, and the azimuth range of the target is estimated using the translation invariance measurement map.
[0057] The translation invariance measurement method used in this embodiment is specifically as follows:
[0058] use Representing a picture with a resolution of H×W, a CNN (convolutional neural network) with M layers can be used as a feature extractor m∈{0,1,2,…,M} means that each feature map can be upsampled to the same resolution as X. m∈{0,1,2,…,M}, where Shift Δh,Δw (g) represents a translation of Δh pixels in the length direction and Δw pixels in the width direction. In this embodiment, cosine similarity is used to measure the degree of translation invariance, and the formula is as follows:
[0059]
[0060] in The feature map output by a certain layer of the network indicates that the target is at the exact center of the image. Represents the output feature map of the target translated image at a certain layer.
[0061] The result of the above formula should be between -1 and 1. The larger the value, the greater the cosine similarity and the better the translation without deformation.
[0062] The above process is described with a specific embodiment. Figure 2 and Figure 3 As shown in the figure, it is a 5×5 feature map. The size parameter of the activation area is set to 3×3 and the step size is 2. Taking the activation areas in the upper left and upper right corners of the feature map as examples, the template M of the activation area is calculated according to the formula δ(k,l). Figure 2 and Figure 3 The calculated activation area template M is shown in Figure 2. Then the template judgment threshold K is introduced to judge the activation area. Figure 2 The activation area in is the template of non-isolated points, and the pooling output is calculated using the formula in S3, that is, Figure 4 shown; Figure 3 The active region in is an isolated scatterer, corresponding to Figure 5 As shown, the average value of the feature value in the activation area is used as the pooling output.
[0063] The above method of the present invention is applied as a pooling template, which is inserted into various convolutional neural networks instead of maximum pooling or average pooling to obtain better image translation invariance.
[0064] Taking two typical convolutional neural networks A-ConvNets and VGG19 as examples, the maximum pooling is replaced by the pooling template based on SigmaPool of the present invention, and the convolution kernel and step size are set to be the same as the original backbone network, that is, A-ConvNets uses SigmaPool with a convolution kernel of 3 and a step size of 1, and VGG19 uses SigmaPool with a convolution kernel of 3 and a step size of 2. During the simulation experiment, the batch size is set to 16, the initial learning rate is set to 0.001, and the learning rate is multiplied by 0.1 every 30 epochs. The network is optimized using the SGD optimizer.
[0065] The simulation experiment uses the SOC condition of the MSTAR dataset, with a pitch angle of 15° for the training set and 17° for the test set. The minimum size of the synthetic aperture radar image of the MSTAR dataset is 128×128. In order to facilitate the translation invariance experiment, it is uniformly cropped from the center into a cropping box of size 88×88 as the training set. The test set will be re-cropped from the 128×128 large image to 88×88 images. Starting from the center, each pixel shifted to the upper left is set as a test set. There are a total of 21 test sets, denoted as S i , i represents the pixel value of translation, Figure 6 Gives a partial translation image instance.
[0066] After completing the training of the neural network, a picture is randomly selected from each of the four types of data, BTR70, T62, ZSU_23_4 and D7, and tested with the trained A-ConvNets. The test results are given. The horizontal axis represents the pixel value of the translation, and the vertical axis represents the probability that the current image is predicted to be the correct category. The greater the probability, the easier it is to be correctly identified. The present invention also defines a translation pixel tolerance T. When the translation pixel value is in the range of 0 to T, the algorithm can correctly identify the target. When the translation pixel value is greater than T, the algorithm will misidentify the target. The larger the T value, the better, indicating that the algorithm has a stronger tolerance for translation interference. Figure 7 It can be seen that the T value of A-ConvNets using SigmaPool is generally 1 to 2 pixels higher than that of A-ConvNets using MaxPool, which means that SigmaPool improves the translation invariance of the recognition network.
[0067] When the method of the present invention is applied to VGG19, Table 1 gives the recognition rates of 21 test sets on A-ConvNets and VGG19, and the bold part indicates that the recognition rate is higher than the Baseline. It can be seen from the table that the recognition rate decreases by about 10% for each pixel translation. When the translation is 20 pixels, the recognition rate of the network is approximately equal to the probability of random guessing by the untrained network. It can be seen that the translation of the target has a significant impact on the recognition rate. It can be seen from the experiments on VGG19 that SigmaPool is useful for translation within a small range (within 10 pixels). Experiments on A-ConvNets show that the improvement in recognition rate is mainly concentrated within 3 pixels, and the improvement in recognition rate beyond 3 pixels is not obvious. The main reason why the recognition rate of A-ConvNets is generally lower than that of VGG19 is that the number of layers of A-ConvNets is too small and the information extraction is insufficient.
[0068] Table 1 Translation pixels / recognition rate of the test set
[0069]
[0070] Furthermore, according to the translation invariance measurement method proposed in the present invention, a picture is randomly selected from class ZIL131, and tested using VGG19 to calculate the output of different pooling layers. and The cosine similarity between Figure 8As shown. The cosine similarity is normalized to [0,1]. The red color in the figure indicates good translation invariance, and the blue color indicates large deviation. The horizontal and vertical coordinates represent the pixels of image translation, and the value range is [-20,20]. At this time, the target is not limited to translation to the upper left, but can be translated in any direction. The first row indicates the use of MaxPool, and the second row indicates the use of SigmaPool. It can be seen from the figure that as the number of network layers increases, the red range of the metric graph gradually expands, indicating that the correlation of the rich semantic information about the target contained in the two deep features is enhanced, and the network has obtained a larger receptive field. The change is not obvious in the shallow network. Pool-5 significantly shows that SigmaPool enables the network to obtain a larger receptive field, thereby improving the recognition rate of SAR target recognition in the case of target translation.
[0071] In addition, by observing Figure 8 It can be observed that the metric graphs of the first few layers are in the form of a straight line with an oblique upper left angle. It is speculated that the angle of this line is related to the azimuth of the target. Fig. 9 As shown, it can be clearly seen that the translation invariance metric map is related to the azimuth of the target. The network is more likely to capture targets that are translated in the azimuth direction rather than in the direction perpendicular to the azimuth. This is very inspiring for candidate box selection and azimuth estimation of target detection.
[0072] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.
Claims
1. A target recognition method for improving the translation invariance of synthetic aperture radar images, characterized in that: The following steps are involved: S1. For the feature map, set the pooling convolution kernel parameters and calculate the threshold of the activation area in the feature map according to the SigmaPool principle; S2, comparing each feature value in the activation area with the threshold of the activation area in turn, so as to obtain a template of the activation area; S3. Based on the template of the activation area obtained in S2, a template judgment threshold is introduced to judge whether the current activation area is an isolated scatterer, and a corresponding pooling calculation is performed to obtain a pooling output.
2. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 1, characterized in that: Steps S1 to S3 are used as pooling modules and inserted into the convolutional neural network for target recognition. The translation invariance measurement method is used to judge the recognition results to determine the improvement of SigmaPool on the translation invariance of the convolutional neural network, and the translation invariance measurement map is used to estimate the azimuth range of the target.
3. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 1, characterized in that: Step S1 includes the following steps: S11. Given an S×S feature map, set the pooling convolution kernel parameters and determine the activation area; S12. According to the SigmaPool principle, calculate the average value of all features in the activation area and the standard deviation σ of the activation area ij ; S13, based on the average value of the activated area and standard deviation σ ij , calculate the threshold of the activation area.
4. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 3, characterized in that: The average value of all features in the activation area and the standard deviation σ of the activation area ij for Among them, k and l are the coordinate positions of the feature points in the activation area, L is the convolution kernel size of the activation area, k≤L, l≤L, and g(k,l) is the feature value in the activation area.
5. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 4, characterized in that: The threshold of the activation area is the low threshold and high threshold 6. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 5, characterized in that: In S2, the formula for comparing each feature value in the activation area with the threshold of the activation area in turn is:
7. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 6, characterized in that: In S3, the template judgment threshold K = (L + 1) / 2, when the feature value of the activation area falls within 2σ F If the number of features in the range is less than or equal to the template judgment threshold K, the activated area is judged as an isolated scatterer, and the average value of the feature values in the activated area is used as the pooling output; when the feature value of the activated area falls within 2σ F If the number within the range is greater than the template judgment threshold K, the activated area is weighted and calculated as the pooling output.
8. The target recognition method for improving the translation invariance of synthetic aperture radar images as claimed in claim 2, characterized in that: The cosine similarity is used to measure the translation invariance, and the formula is as follows: in The feature map output by a certain layer of the network indicates that the target is at the exact center of the image. Represents the output feature map of the target translated image at a certain layer.