Fisher information matrix attitude angle adaptive SAR target recognition method

Through the Fisher information matrix pose angle adaptive SAR target recognition method, the problem of performance degradation of SAR target recognition model when pose angle changes is solved, online training and parameter protection are realized, and recognition accuracy and response capabilities are improved.

CN118628719BActive Publication Date: 2024-12-06NANJING UNIV OF INFORMATION SCI & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410758703.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-13
Publication Date
2024-12-06
Estimated Expiration
2044-06-13

AI Technical Summary

Technical Problem

The performance of SAR target recognition model significantly decreases when pose angle changes, and traditional methods retrain the model time-consuming and memory-consuming, resulting in catastrophic forgetting problems.

Method used

The Fisher information matrix pose angle adaptive SAR target recognition method is adopted to dynamically adjust the parameter regularization intensity through data filtering, model initialization, pre-training, computing importance weights, protection parameters and online learning to realize online training and parameter protection of the model.

Benefits of technology

It alleviates catastrophic forgetting, enhances the model's adaptability to data features of different pose angles, improves the response and recognition accuracy of the SAR-ATR system, and adapts to the changes in target features under the new pose angles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118628719B_ABST
    Figure CN118628719B_ABST
Patent Text Reader

Abstract

The present invention provides a Fisher information matrix attitude angle adaptive SAR target recognition method, and constructs an attitude angle adaptive SAR target recognition continuous learning model by designing regularization terms and Fisher information matrices to protect the parameters of the learned SAR target features. The model learns the newly observed SAR target data in an online learning manner, and dynamically adjusts the regularization strength of the parameters through the Fisher information matrix to protect the learned model parameters that contribute greatly to the recognition task, enhance the model's adaptability to SAR target features at different attitude angles, and improve the accuracy of target recognition. Experiments are conducted on the MSTAR data set. The attitude angle adaptive SAR-ART model is able to learn SAR target data at different attitude angles online, continuously adapt to changes in SAR target features at different attitude angles, and effectively improve recognition accuracy. The present invention can be widely used in the technical field of SAR target recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of SAR target recognition, and in particular relates to a Fisher information matrix attitude angle adaptive SAR target recognition method. Background Art

[0002] The research on automatic target recognition (ATR) technology for synthetic aperture radar (SAR) has attracted widespread attention. Since the features of SAR images are highly sensitive to the target attitude angle, and in practical applications, it is difficult for radar to obtain data at all attitude angles of the target in one observation, the performance of SAR target recognition models often degrades significantly on data in unobserved attitude angle domains. When new observation data arrives, training the existing model based solely on the new observation data will cause the model to catastrophically forget the SAR target features at the attitude angles that have been learned.

[0003] In practical applications, the performance of the SAR-ATR model is limited by the integrity and diversity of the data, and the radar target attitude angle has an important influence on the SAR image features. As the attitude angle changes, the same target may present different geometric shapes and reflection characteristics, which leads to the SAR target recognition model often showing a significant performance degradation on the unobserved attitude angle domain data. In one observation, it is difficult for the radar to obtain data at all attitude angles of the target. When new observation data arrives, the traditional method needs to use all the data to retrain the entire model, but in practical applications, the amount of data continues to increase, making retraining time and memory consuming. When the model is updated only with new observation data, the data representation and decision boundary will change. As the parameters are adjusted, the model will have catastrophic forgetting of past knowledge, resulting in a significant performance degradation. Summary of the invention

[0004] Purpose of the invention: In order to cope with the performance degradation problem of the SAR-ATR system caused by incomplete observation of the attitude angle of SAR target data, the present invention provides a Fisher information matrix (Fisher information matrix) attitude angle adaptive SAR target recognition method by introducing a continuous learning algorithm, comprising the following steps:

[0005] Step 1, data preparation: SAR target data set {Da1, Da2, Da3, …, Da N} Waiting to receive data and filtering data at the same time; D aN represents the SAR target data set under the initial attitude angle aN, where N is the number of samples;

[0006] Step 2, model initialization: Use the Pytorch initializer to automatically initialize the Synthetic Aperture Radar Automatic Target Recognition (SAR-ATR) model F EWC (·);

[0007] Step 3, pre-training model: Let the hyperparameter l = 0, and the hyperparameter λ be the learned data set D a1 With the new learning dataset D a2 The weight factor between them is used to train the model F by extracting SAR target features on the SAR target dataset sequence. EWC (·);

[0008] Step 4, calculate the importance weight: calculate the importance of network parameters to the SAR target data set sequence based on the Fisher information matrix;

[0009] Step 5, Protection Parameters: Model F EWC (·) protecting the key features of the trained SAR target dataset sequence according to importance weights;

[0010] Step 6: Learn new tasks: Let l = g, where γ is the updated importance weight factor, and model F EWC (·) Learning SAR target data under new attitude angles;

[0011] Step 7, repeating steps 4 to 6, facing the continuous new SAR image data stream, the importance weights must be updated each time the SAR target data under a new attitude angle is learned;

[0012] Step 8, output y = F EWC (·), where y is the class label.

[0013] Step 1 includes:

[0014] The following formula is used for data filtering:

[0015] f(x)=10lg[x+10 -3 ]+30

[0016] Where x is the original SAR image and f(x) is the SAR image after logarithmic filtering.

[0017] Step 2 includes: the radar obtains a complete data set at all attitude angles of the target through K observation tasks Where N ak is the sample of the Nth sample under the attitude angle ak; let the data set The number of samples included is N = |D|; is the i-th SAR image sample in the dataset D, W represents the image width, H represents the image height, and y i is the i-th category label; represents the real number space;

[0018] Model F EWC (·) The soft-max layer is used as the output layer, and M is the model F EWC (·) The number of network parameters;

[0019] Input SAR image sample x i , F EWC The output of (·) is the probability distribution of the predicted category Satisfy p s ∈[0,1] and p s is the probability of the sth category, S is the number of categories of the predicted target, then the index of the predicted SAR target category is:

[0020] C(x)=argmax s (F(x i ))∈[1,2,…,S]

[0021] Where C(x) is the index of the SAR target category, argmax means taking the independent variable that maximizes the value of a function. Here, s is the independent variable that maximizes the value of F(x). i ) is the largest index, S is the number of categories of the predicted target, F(x i ) represents the prediction result of the i-th SAR image sample in the SAR target dataset D.

[0022] Step 3 includes: using the following formula to obtain the SAR target data set D under the initial attitude angle a1 a1 Extract SAR target features:

[0023]

[0024] Where L CE is the total loss function, p i is the probability distribution of the i-th predicted category.

[0025] In step 3, the minimum cross entropy method is used to train the model F on the SAR target dataset sequence. EWC (·).

[0026] Step 4 includes: calculating the SAR target data set D according to the following formula a1 The Fisher information matrix I a1 , and use the following formula to calculate and update the importance weight:

[0027]

[0028] in, is the inverse of the Fisher information matrix, ▽ q l j (q i ) is the jth SAR image sample x j The log-likelihood function On the i-th parameter q of the Aperture Radar Automatic Target Recognition SAR-ATR model i The gradient of N a1 SAR target dataset The number of samples, for is an approximation of , where T represents the matrix transpose.

[0029] In step 5, the following formula is used to protect the parameters:

[0030]

[0031] where p(θ|D a1 ) is the SAR target data set D a1 The probability distribution of The Fisher information matrix for the model evaluation F EWC (·) network parameters for SAR target dataset The importance parameter, q a1,i is the i-th network parameter under the attitude angle a1, for q a1,i The conjugate transpose of is the regularization term.

[0032] Step 6 includes: let l = 0, and use the following formula to convert the SAR target data set The feature extraction to SAR-ATR model F EWC In the parameter space of (·):

[0033]

[0034] Where L(q) is the total loss function, L CE (q) is the SAR target data set at the new attitude angle a2 Train the model F EWC The total loss function of (·), is the regularization term.

[0035] Step 7 includes: SAR target data set at the new attitude angle a2 In the above, the SAR target features are extracted by the method in step 3, and the data is calculated by the method in step 4. Fisher information matrix Let l = g, and use the method in step 6 to convert the data Feature extraction to Aperture Radar Automatic Target Recognition SAR-ATR model F EWC (·) in the parameter space; the two stages are continuously alternating, so that the model can achieve the purpose of online training.

[0036] The present invention has the following beneficial effects:

[0037] (1) The regularization term is used to limit the parameter changes of the learned SAR target features in the model to alleviate catastrophic forgetting;

[0038] (2) By using the Fisher information matrix, the regularization strength of each parameter is dynamically adjusted according to the influence of SAR target data at different attitude angles on the model performance, thereby enhancing the model's adaptability to data features at different attitude angles.

[0039] (3) The online training of the model is realized, and the responsiveness of the SAR-ATR system is enhanced. The attitude angle adaptive SAR target recognition continuous learning model proposed in this paper is verified on the MSATR dataset. It only needs to learn the SAR image data under the new attitude angle online to adapt to the changes in target features under the new attitude angle. It improves the SAR target recognition accuracy of the SAR-ATR system and enhances the real-time performance. It provides strong support for target recognition in practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of a method for attitude angle adaptive SAR target recognition based on Fisher information matrix;

[0041] Figure 2 Continuous learning process for attitude angle adaptive SAR target recognition;

[0042] Figure 3 It is the continuous learning process of the EWC-based SAR-ATR model;

[0043] Figure 4 This is the Resnet-18 network structure diagram;

[0044] Figure 5 This is a comparison chart of SAR images before and after filtering;

[0045] Figure 6 Performance comparison chart for four online learning tasks;

[0046] Figure 7 Learn performance metrics for the model online;

[0047] Figure 8 Performance under different learning orders. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0049] like Figure 1 As shown, the present invention provides a Fisher information matrix attitude angle adaptive SAR target recognition method. First, SAR-ATR is introduced. Suppose SAR image data set The number of samples included is N = |D|. is the i-th SAR image sample in the dataset D, the image size is W′H, y i Its category label. SAR-ATR model F EWC (·) The soft-max layer is used as the output layer, and M is the model F EWC (·) The number of network parameters. Input SAR image sample x i , F EWC The output of (·) is the probability distribution of the predicted category Satisfy p s ∈[0,1] and S is the number of categories of predicted targets, and the index of the predicted SAR target category is:

[0050] C(x)=argmax s (F(x i ))∈[1,2,…,S]

[0051] The SAR-ATR model F is trained using the minimum cross entropy method on the SAR image dataset D. EWC (·), then the total loss function is:

[0052]

[0053] The network structure of the SAR-ATR model is as follows: Figure 2 shown.

[0054] Figure 3 The continuous learning process of the SAR-ATR model based on EWC is shown. Among them, S is the learning stage; C is the compression stage; EWC represents F EWC (·)Model; is the SAR target data at the initial attitude angle a1; is the SAR target data at the initial attitude angle a2; is the SAR target data at the initial attitude angle a3; and is the Fisher information matrix at the corresponding angle; and is the network parameter at the corresponding angle; and is the conjugate transpose of the network parameters.

[0055] Specifically, the experiment of the present invention uses the Resnet-18 network as the SAR-ATR model F EWC (·), the input x is a SAR target image of 100′100 pixels, model F EWC (·) Structure like Figure 4 shown.

[0056] The data of the three tasks are augmented by the method of attitude synthesis. The specific training and test data are shown in Table 1 (training and test data of SAR targets in four tasks).

[0057] Table 1

[0058]

[0059] Since speckle noise appears as multiplicative noise on SAR images, and neural networks are better at processing additive noise, logarithmic transformation is used to convert multiplicative noise into additive noise.

[0060] Suppose the original SAR image is x, and the SAR image after logarithmic filtering is f(x) as shown below:

[0061] f(x)=10lg[x+10 -3 ]+30

[0062] The comparison of SAR images before and after filtering is shown in the figure below. Figure 5 shown.

[0063] The experimental program running environment is shown in Table 2 (experimental environment parameters).

[0064] Table 2

[0065]

[0066] The SAR-ATR model training hyperparameters are shown in Table 3 (SAR-ATR model training hyperparameter table).

[0067] Table 3

[0068] category Specification Iterations 80 Learning Rate 0.0001 Weight Factor 100 Loss Function Stochastic Gradient Descent Optimization Methods Cross Entropy Loss

[0069] In this embodiment, three comparison models are used to analyze the attitude angle adaptive SAR target recognition continuous learning model F EWC (·) Performance of online learning for multiple tasks: Model F based on traditional methods Base (·), Model F based on L2 norm regularization L2 (·) and the model F based on the experience replay (ER) strategyER (·).

[0070] (1) Performance Analysis

[0071] This example aims to evaluate the attitude angle adaptive SAR target recognition continuous learning model F EWC (·) Online learning of SAR target data task D under four different attitude angles a1 ,D a2 ,D a3 ,D a4 performance.

[0072] The performance on the four tasks is as follows Figure 6 As shown in the figure, the line graph from top to bottom represents the various models in D a1 ,D a2 ,D a3 ,D a4 The dotted line from left to right indicates the performance of the model on the test set of D a1 ,D a2 ,D a3 ,D a4 Training phase on training data. In the first task learning phase, all models performed well in learning SAR target features at attitude angle a1, while the performance of the models decreased significantly for SAR target data at other attitude angles, and the performance decrease trend became more obvious as the attitude angle difference increased. As the number of training tasks increased, the differences between different models began to emerge. Model F Base (·) The degree of forgetting of past tasks is the most serious, model F EWC (·) Best performance; Model F performs best in terms of its ability to learn new tasks. Base (·) The most severe decline, model F EWC (·) is still optimal.

[0073] (2) Further analyze the online learning performance indicators of the model.

[0074] Online learning performance indicators such as Figure 7 As shown, F Base (·) The model performs the worst. As the number of learning tasks increases, the model improves only after the second task, but then the performance drops sharply. This is because the model does not take measures to prevent catastrophic forgetting, resulting in the forgetting of previous task knowledge. At the same time, the model training falls into some minimum points, resulting in a significant increase in uncompromising and inability to effectively learn the data features of new tasks.

[0075] F performed slightly better L2 (·) When the number of learning tasks increases, the average accuracy of the model increases in the early stage, but then it also drops sharply. L2(·) The model protects the network parameters about past task knowledge through regularization, but the regularization strength is insufficient and the protection of key parameters is insufficient, resulting in the forgetting of past task knowledge. At the same time, as the learning tasks increase, the accumulation of regularization strength leads to an increase in the intransigence of the model, reducing the ability to learn new task knowledge.

[0076] The better performing model F ER (·) In the learning tasks, as the number of tasks increases, the average precision increases significantly, but then decreases slightly. This is due to the limitation of the number of playback samples, and the proportion of overall SAR target features covered by the samples gradually decreases, causing the model ER to forget the previously learned SAR target features. Because the model ER relies too much on previous experience, it becomes more uncompromising and cannot effectively adapt to the requirements of new data tasks.

[0077] The best performer is F EWC (·) model, the average accuracy continues to rise as the number of learning tasks increases. The model can dynamically adjust the regularization strength of each parameter through the Fisher information matrix, increase the protection of key parameters, and weaken the regularization strength of irrelevant parameters, thereby reducing intransigence and enhancing the model's ability to learn new task data.

[0078] (3) Analysis of the influence of weight factor λ

[0079] This experiment aims to explore the effect of weight factor λ on attitude angle adaptive SAR target recognition continuous learning model F EWC (·) Performance impact on SAR target task data under continuous learning of multiple attitude angles.

[0080] The experimental results are shown in Table 4 (Comparison of the impact of hyperparameter λ on performance). As λ increases, F L2 (·) and F EWC (·) The model will be more inclined to protect the learned SAR target features, the degree of forgetting will gradually decrease, and the intransigence will gradually increase. However, there is a fluctuation between λ=10 and 100. A moderate λ value may make it easier to learn more subtle features of the SAR target, resulting in improved performance. In order to ensure the overall improvement of the model performance, it is necessary to select a suitable λ value based on the data characteristics. Model F EWC (·) is much less sensitive to the weight factor λ than model F L2 (·), this is because F EWC (·) It can dynamically adjust the regularization strength of each parameter based on the knowledge of past learning tasks, thus ensuring low forgetting rate and uncompromising performance.

[0081] Table 4

[0082]

[0083] (4) Analysis of the impact of data learning order under different posture angles

[0084] The experiment aims to explore the effect of the learning sequence of SAR target task data under different attitude angles on the attitude angle adaptive SAR target recognition continuous learning model F. EWC The performance impact of (·) is analyzed by training SAR target data at three attitude angles according to six different training sequences {(D a1 ,D a2 ,D a3 ),(D a1 ,D a3 ,D a2 ), (D a2 ,D a1 ,D a3 ),(D a2 ,D a3 ,D a1 ),(D a3 ,D a1 ,D a2 ),(D a3 ,D a2 ,D a1 )} for online learning.

[0085] like Figure 8 As shown, the horizontal axis represents the order of training data from left to right, and the vertical axis represents the three corresponding test data. EWC (·) Only learn SAR target data D at attitude angles of 22.5° to 44.9° a2 When the model F is used, it performs well on the data of 0°~22.4° and 45°~67.4° under the unlearned posture angle. After learning all the data sets, the model F EWC (·) First learn the dataset D a2 The overall recognition performance of the dataset D a1 and dataset D a3 This shows that the model is good at SAR target data D at attitude angles of 22.5° to 44.9°. a2 The above captures some features of SAR targets at attitude angles of 0°~22.4° and 45°~67.4°. In continuous learning tasks, the order of task learning will significantly affect the model F. EWC (·) Overall performance. Different learning orders have fluctuations in overall performance. This is because when learning SAR target data at different attitude angles, parameter constraints are made based on the importance of the SAR target data set at the attitude angles learned in the past, and this importance is affected by the order of learning the task data set. If the previously learned data set is highly correlated with the newly learned data set, model F EWC(·) Pay more attention to retaining the characteristics of the dataset learned in the past, so as to perform better on the dataset learned in the past. Therefore, the order of learning data selection is crucial to the performance of this model, and it needs to be reasonably adjusted and optimized according to the specific application scenario.

[0086] The present invention provides a Fisher information matrix attitude angle adaptive SAR target recognition method. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. Fisher information matrix attitude angle adaptive SAR target recognition method, It is characterized in that The following steps are involved: Step 1, data preparation: SAR target dataset {Da 1 ,Da 2 ,Da 3 ,…,Da N } Waiting to receive data and filtering data at the same time; D aN represents the SAR target data set under the initial attitude angle aN, where N is the number of samples; Step 2, model initialization: Use the Pytorch initializer to automatically initialize the synthetic aperture radar automatic target recognition SAR-ATR model F EWC (·); Step 3, pre-training model: Let the hyperparameter l = 0, and the hyperparameter λ be the learned data set D a1 With the new learning dataset D a2 The weight factor between them is used to train the model F by extracting SAR target features on the SAR target dataset sequence. EWC (·); Step 4, calculate the importance weight: calculate the importance of network parameters to the SAR target data set sequence based on the Fisher information matrix; Step 5, Protection Parameters: Model F EWC (·) protecting the key features of the trained SAR target dataset sequence according to importance weights; Step 6: Learn new tasks: Let l = g, where γ is the updated importance weight factor, and model F EWC (·) Learning SAR target data under new attitude angles; Step 7, repeating steps 4 to 6, facing the continuous new SAR image data stream, the importance weights must be updated each time the SAR target data under a new attitude angle is learned; Step 8, output y = F EWC (·), where y is the class label.

2. The method according to claim 1, It is characterized in that Step 1 includes: The following formula is used for data filtering: f(x)=10lg[x+10 -3 ]+30 Where x is the original SAR image and f(x) is the SAR image after logarithmic filtering.

3. The method according to claim 2, It is characterized in that Step 2 includes: the radar obtains a complete data set at all attitude angles of the target through K observation tasks Where N ak is the sample of the Nth sample under the attitude angle ak; let the data set The number of samples included is N = |D|; is the i-th SAR image sample in the dataset D, W represents the image width, H represents the image height, and y i is the i-th category label; represents the real number space; Model F EWC (·) The soft-max layer is used as the output layer, and M is the model F EWC (·) The number of network parameters; Input SAR image sample x i , F EWC The output of (·) is the probability distribution of the predicted category Satisfy p s ∈[0,1] and p s is the probability of the sth category, S is the number of categories of the predicted target, then the index of the predicted SAR target category is: C(x)=argmax s (F(x i ))∈[1,2,…,S] Where C(x) is the index of the SAR target category, argmax means taking the independent variable that maximizes the value of a function, and s is the value that makes F(x i ) is the largest index, S is the number of categories of the predicted target, F(x i ) represents the prediction result of the i-th SAR image sample in the SAR target dataset D.

4. The method according to claim 3, It is characterized in that Step 3 includes: using the following formula to obtain the SAR target data set D under the initial attitude angle a1 a1 Extract SAR target features: Where L CE is the total loss function, p i is the probability distribution of the i-th predicted category.

5. The method according to claim 4, It is characterized in that In step 3, the minimum cross entropy method is used to train the model F on the SAR target dataset sequence. EWC (·).

6. The method according to claim 5, It is characterized in that Step 4 includes: calculating the SAR target data set D according to the following formula a1 The Fisher information matrix I a1 , and use the following formula to calculate and update the importance weight: in, is the inverse of the Fisher information matrix, is the jth SAR image sample x j The log-likelihood function On the i-th parameter q of the Aperture Radar Automatic Target Recognition SAR-ATR model i The gradient of a1 SAR target dataset The number of samples, for is an approximation of , where T represents the matrix transpose.

7. The method according to claim 6, It is characterized in that In step 5, the following formula is used to protect the parameters: where p(θ|D a1 ) is the SAR target data set D a1 The probability distribution of The Fisher information matrix for the model evaluation F EWC (·) network parameters for SAR target dataset The importance parameter, q a1,i is the i-th network parameter under the attitude angle a1, for q a1,i The conjugate transpose of is the regularization term.

8. The method according to claim 7, It is characterized in that Step 6 includes: let l = 0, and use the following formula to convert the SAR target data set The feature extraction to SAR-ATR model F EWC In the parameter space of (·): Where L(q) is the total loss function, L CE (q) is the angle a in the new posture 2 SAR target dataset Train the model F EWC The total loss function of (·), is the regularization term.

9. The method according to claim 8, It is characterized in that Step 7 includes: at the new posture angle a 2 SAR target dataset In the above, the SAR target features are extracted by the method in step 3, and the data is calculated by the method in step 4. Fisher information matrix Let l = g, and use the method in step 6 to convert the data Feature extraction to Aperture Radar Automatic Target Recognition SAR-ATR model F EWC (·) in the parameter space.

Citation Information

Patent Citations

  • Multi-task comparative learning target identification method in radar attitude angle unknown scene

    CN117289230A

  • Few-shot radar target recognition method based on hierarchical meta transfer

    WO2024082374A1