SAR and optical image fusion-oriented modal collaborative adaptive weight space target identification method

Abstract

The invention discloses an SAR and optical image fusion-oriented modal collaborative adaptive weight space target identification method, which comprises the steps of S1, constructing a multi-modal data set of satellite optical and SAR images, the data set comprising N pairs of registered SAR and optical images and corresponding category labels; s2, respectively extracting depth feature representations of the optical image and the SAR image obtained in the S1; s3, predicting the imaging quality of each modal of the depth features extracted in the S2, realizing QADA quality perception fusion through quality-guided asymmetric cross attention, constructing quality-enhanced fusion feature representation, and calculating the basic loss of an optical branch, an SAR branch and a fusion branch; s4, based on the three-branch basic loss and the prediction probability calculated in the S3, dynamically calculating respective loss weights through five cooperative factors MSAW, and jointly optimizing three-branch network parameters by using weighted total loss and consistency regularization until convergence; and S5, inputting the test data into the network to predict and identify the satellite category.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing and multi-modal fusion, and in particular discloses a method for spatial target recognition based on SAR and optical image fusion with adaptive weight of modal coordination. BACKGROUND

[0002] Existing SAR and optical image fusion methods can be mainly divided into three categories: pixel-level fusion, feature-level fusion and decision-level fusion. Pixel-level fusion directly performs weighted average or transform domain fusion on the original image, which is simple to calculate but is easily disturbed by SAR speckle noise and is difficult to capture high-level semantics; decision-level fusion separately identifies each modality and then votes or weightedly outputs, which is robust but loses deep collaborative information between modalities. Feature-level fusion is the current research hotspot, which extracts high-dimensional feature representations of each modality through deep neural networks and then performs fusion, which can learn cross-modal correlation while preserving modality specificity.

[0003] With the development of deep learning technology, the fusion framework based on multi-task learning (MTL) has gradually become mainstream. Multi-task learning models the classification and fusion of each modality as multiple related tasks, and realizes collaborative learning through shared representation and joint optimization. However, the core challenge of multi-task learning is how to reasonably allocate the loss weights of different tasks. The loss values of different tasks differ significantly in numerical scale, convergence speed and optimization difficulty, and simple equal-weight summation will lead to some tasks dominating the training process, making other tasks learn insufficiently, and ultimately affecting the overall performance.

[0004] To solve the problem of weight distribution, researchers have proposed various multi-task learning weight adjustment methods. Fixed Weighting determines the weight of each task by manual setting or grid search, which is simple and controllable but cannot adapt to training dynamics and has poor generalization. Uncertainty Weighting automatically balances the weight by learning the logarithmic variance of the task, reducing the need for manual parameter tuning, but does not consider the collaborative relationship between modalities and the dynamic changes in imaging quality. GradNorm prevents a task from dominating the training by balancing the gradient norm of each task, but requires the calculation of shared layer gradients, resulting in large computational overhead and unstable training. DWA adjusts the weight based on the loss change rate, which can respond to training dynamics but has large fluctuations at the beginning of training and lacks explicit modeling of the relationship between tasks. Auto-Lambda learns the weight strategy end-to-end through meta-learning, which is highly adaptive but prone to overfitting and increases model complexity by introducing additional learnable parameters. LossBalanced normalizes the loss value to balance the numerical scale of different tasks, which is simple to implement but relies too much on the absolute value of the loss and ignores the actual importance of the task.

[0005] In particular, in the practical application scenario of satellite identification, the imaging characteristics of different satellites show significant differences. Some satellites are equipped with large-area high-reflectivity solar panels, which have obvious features and high confidence in optical images, while the surface radar wave-absorbing coating causes weak SAR signals; some satellites use stealth design, which has no obvious optical features but unique SAR scattering center distribution; some satellites have drastic attitude changes, resulting in dynamic changes in image quality and reliability of the same target in two modal images at different times. In addition, there are essential differences in the difficulty of identifying different satellite categories: large communication satellites are relatively easy to identify due to their distinctive structural features, while small cube satellites are difficult samples due to their small size and weak features. The existing static weight or dynamic weight method based only on loss statistics cannot capture the quality differences at the sample level and the modal collaborative mode. For simple satellite samples with obvious features, single-modal prediction is sufficient, and over-reliance on fusion introduces unnecessary complexity; for difficult satellite samples with blurred features, the complementary advantages of multi-modal fusion need to be fully utilized, but the global weight strategy of existing methods cannot achieve adaptive adjustment at the sample level, resulting in unstable fusion performance. SUMMARY

[0006] The application aims at the deficiencies of the prior art, and provides a SAR and optical image fusion satellite identification method based on modal synergy adaptive weighting, a modal synergy adaptive weighting (MSAW) multi-task learning framework and a quality-aware dynamic attention fusion mechanism (QADA) are designed, problems that existing multi-task learning methods ignore modal synergy, lack sample-level adaptive ability, cannot cope with dynamic changes in modal quality, and are not designed in the training stage are solved, and the identification accuracy, robustness and computational efficiency of different satellite targets under complex observation conditions are improved.

[0007] The technical scheme is that the application provides a spatial target identification method based on modal synergy adaptive weighting for SAR and optical image fusion, and the method comprises the following steps:

[0008] S1, a multi-modal data set of satellite optical and SAR images is constructed, the data set comprises N pairs of registered SAR and optical images and corresponding class labels, size normalization processing is performed on all images, and the pixel resolution is uniformly adjusted to Data augmentation processing is performed on the training set images;

[0009] S2, a parameter-independent double-encoder architecture is used to extract deep feature representations of the optical images and SAR images obtained in S1;

[0010] S3, a two-layer fully connected neural network is used to predict the imaging quality of the deep features extracted in S2, quality-aware fusion is realized through quality-guided asymmetric cross-attention, quality-enhanced fusion feature representations are constructed, and basic losses and prediction probabilities of an optical branch, an SAR branch and a fusion branch are calculated;

[0011] S4, loss weights of the optical branch, the SAR branch and the fusion branch are calculated based on the three-branch basic losses and the prediction probabilities calculated in S3, three-branch network parameters are jointly optimized using a weighted total loss and consistency regularization until convergence; the three-branch network comprises a double-encoder feature extraction network, a QADA quality-aware fusion network and a three-branch classifier network;

[0012] S5, a model under the best parameters is saved, test data is input into the network for prediction and identification of satellite categories.

[0013] Further, in step S1, the method for constructing the simulation data set is as follows:

[0014] A SAR and optical image registration dataset containing C-class satellite targets was obtained. The satellite categories include 10 categories, i.e., C=10. The optical images were acquired through visible light or infrared imaging, with an imaging wavelength range of 400-900nm and a spatial resolution better than 0.5m. The SAR images were acquired through X-band synthetic aperture radar imaging, with a center frequency of 9.6GHz, a bandwidth of 600MHz, and a resolution of 1m×1m.

[0015] The entire dataset is represented as ,in, Let be the optical image of the k-th sample, where Indicates the height of the image. Indicates the width of the image. For the corresponding SAR image, The samples are 10,000 pairs, with 5,000 pairs used as training samples and 5,000 pairs used as test samples.

[0016] Furthermore, in step S1, the normalization and feature enhancement methods are as follows:

[0017] All images in the dataset were uniformly adjusted using bicubic interpolation. Pixel resolution, pixel value from Normalization to Then use the mean and standard deviation Standardization was performed; data augmentation was applied to the training set images, including random horizontal flipping with a probability of 0.5. Random rotation within the range, adjusting the brightness, contrast, and saturation of the optical image. Random adjustments within the range; adjustments only to brightness and contrast of SAR images; and the addition of a standard deviation to both SAR and optical images with a probability of 0.3. Gaussian noise; all geometric transformation operations use the same parameters for the same sample of SAR and optical images to ensure consistent spatial correspondence; the test set is only normalized in size and pixel values, without data augmentation operations.

[0018] Furthermore, in step S2, the dual encoder extraction step is as follows:

[0019] The optical image after preprocessing in step S1 and SAR images Dual encoders with independent input parameters and In this model, the two encoders use the same backbone network architecture but with completely independent parameters to learn modality-specific features. The feature extraction calculation formula is as follows:

[0020]

[0021] where, is the optical depth feature vector of the i-th sample, denotes the real number field, is the corresponding SAR depth feature vector, B is the batch size, and D is the feature dimension.

[0022] Further, in step S3, the quality-aware fusion method is realized by quality-guided asymmetric cross-attention as follows:

[0023] The feature matrix obtained in step S2 is input into two layers of fully connected quality assessment networks, respectively: The network structure includes two fully connected layers: the first layer maps the D-dimensional feature to 512 dimensions, and the second layer maps the 512-dimensional feature to 1 dimension; then the global average pooling is performed: to compress it to and then the Sigmoid activation is performed to obtain the quality score , which is normalized to the relative quality weight:

[0024]

[0025] The matrices of the optical and SAR modalities and the optical modality are obtained by the Xavier initialization method:

[0026]

[0027]

[0028] where, is the input feature dimension, is the output feature dimension, denotes a uniform distribution, represents uniform sampling from to , and represents the query projection matrix of the optical modality, represents the key projection matrix of the optical modality, represents the value projection matrix of the optical modality, represents the query projection matrix of the SAR modality, represents the key projection matrix of the SAR modality, represents the value projection matrix of the SAR modality.

[0029] The query, key, and value vectors required for cross-attention mechanism are calculated by the obtained projection matrices:

[0030]

[0031] where​ query vector representing the optical modality, key vector representing the optical modality, value vector representing the optical modality, query vector representing the SAR modality, key vector representing the SAR modality, value vector representing the SAR modality;

[0032] Computing quality-guided asymmetric cross-attention to get quality-enhanced features :

[0033]

[0034] wherein, is the transpose matrix of, is the transpose matrix of;

[0035] Concatenating strong features into fusion features :

[0036]

[0037] Get the prediction probability under three modalities by three independent classifiers:

[0038]

[0039] wherein, C is the total number of classes, represents the optical classifier, represents the SAR classifier, represents the fusion modality classifier;

[0040] Calculate the optical modality cross-entropy loss :

[0041]

[0042] wherein, is the probability of the optical modality when the class is c, is the one-hot encoding of the class;

[0043] Calculate the SAR modality cross-entropy loss :

[0044]

[0045] wherein, is the probability of the SAR modality when the class is c;

[0046] Calculate the fusion modality cross-entropy loss :

[0047]

[0048] where, is the one-hot encoding under the current class c, is the probability of the fusion modality when the class is c.

[0049] Further, the method for calculating the loss weight of the optical branch, the SAR branch and the fusion branch in step S4 is as follows:

[0050] The maximum probability value of each sample is extracted from the prediction probability obtained in step S3 and the optical modality confidence is calculated by batch averaging:

[0051]

[0052] where, represents the confidence of the optical modality, represents the prediction probability of the i-th sample in the optical branch to the c-th class;

[0053] The SAR modality confidence is calculated as:

[0054]

[0055] where i is the index of the i-th sample in the batch, represents the prediction probability of the i-th sample in the SAR branch to the c-th class;

[0056] The cosine similarity of the prediction distribution is calculated sample by sample for the training samples and the consistency is obtained by batch averaging:

[0057]

[0058] where, represents the consistency of the sample distribution, and are the optical and SAR prediction probability vectors of the i-th sample, respectively, represents the cosine similarity between and ;

[0059] The fusion loss is obtained by Sigmoid mapping to obtain the sample difficulty :

[0060]

[0061] where, represents the sigmoid function;

[0062] According to the training progress of the model Phase weight, wherein is the current training progress, is the current training round, is the total training round, when ;

[0063] When is ;

[0064] When is .

[0065] wherein, denotes the weight of the optical modal in different stages, denotes the weight of the SAR modal in different stages, denotes the weight of the fusion modal in different stages;

[0066] The branch weight is calculated by integrating four factors:

[0067]

[0068] Normalization makes ;

[0069] The weighted total loss is calculated:

[0070]

[0071] The consistency regularization is calculated:

[0072]

[0073] wherein, denotes the divergence between and ;

[0074] The total loss is calculated:

[0075] .

[0076] Further, the method of step S5 is as follows:

[0077] Load the model weight with the highest validation set accuracy in the training process, input the test set 5000 pairs of SAR and optical image pairs into the network, and sequentially pass through the double encoder feature extraction, quality perception fusion to obtain the fusion branch prediction probability; for each test sample, the class corresponding to the maximum probability is taken as the prediction result, and the satellite class recognition is completed. ​​​

[0078] Compared with the prior art, the technical scheme of the present application has the following beneficial technical effects:

[0079] (1) The present application proposes a MSAW mode coordination adaptive weight mechanism and a QADA quality-aware dynamic attention fusion mechanism, realizing a double technical breakthrough in the field of multi-modal fusion. The MSAW mechanism designs five coordination factors: modal confidence perception, cross-modal consistency reward, sample difficulty adaptation, time sequence guidance training and modal specificity loss, and for the first time explicitly utilizes the synergistic modal pair characteristics of SAR and optical images to adjust the weight. When the two modal predictions are consistent, the fusion weight is automatically reduced to avoid overcomplication, and when the predictions are different, the fusion weight is automatically increased to fully exert the complementary advantages, solving the fundamental problem of existing multi-task learning methods that ignore the modal coordination relationship. The QADA mechanism realizes the adaptive transfer of knowledge from high-quality modal to low-quality modal through a quality evaluation network and a quality-guided asymmetric cross-attention, automatically reduces the contribution of a modal when its imaging quality decreases and retains the discriminative features of high-quality modal, avoids noise pollution of the low-quality modal in the fusion representation, and significantly improves the robustness and stability of the model under complex observation conditions.

[0080] (2) The system comparison experiment on the mainstream deep learning backbone network shows that the classification accuracy of the fusion branch is significantly better than that of the existing multi-task learning weight adjustment method under all backbone configurations, with an average accuracy improvement of 2-5 percentage points, and the advantage is more obvious in difficult samples and quality imbalance scenarios. Through sample difficulty adaptation and time sequence guidance training mechanism, sample-level weight individualization adjustment and the progressive learning strategy of "first divide and conquer, then overall optimization" are realized, overcoming the global unified weight limitation of existing methods such as fixed weight method. In terms of computational efficiency, the MSAW method dynamically calculates the weight based on the forward propagation result, does not introduce any learnable parameters, and the weight calculation time complexity is only O(B·C), with a training time increase of only 3-5%; the QADA module has a parameter amount of O(D²), which is less than 2% compared with the pure encoder, and overall maintains the simplicity and efficiency of the model, significantly better than the automatic Lambda method which needs to train an additional meta-network and the gradient normalization method which needs to calculate the gradient of the shared layer.

[0081] (3) The method has high theoretical explainability and wide application expandability. The five synergistic factors of the MSAW mechanism have clear physical meaning and theoretical basis, each component can be independently analyzed and visualized, which is convenient for understanding the model decision process and targeted improvement, and provides a theoretically complete and practically effective solution for multi-modal remote sensing image processing. In terms of application expandability, the method can support the introduction of auxiliary tasks for multi-task joint learning, and can adapt to different types of backbone networks without re-tuning. Experimental verification shows that the best accuracy of 96.06% is achieved on a dataset containing 10 types of satellites, the confusion matrix shows that the recognition performance of each category is balanced, and the t-SNE visualization shows that the fused features have better intra-class compactness and inter-class separability, proving the important practical value of the method. BRIEF DESCRIPTION OF DRAWINGS

[0082] Figure 1 is a flowchart of the method of the present application;

[0083] Figure 2 is an ROC curve graph under the best accuracy of the present application;

[0084] Figure 3 is a confusion matrix graph under the best accuracy of the present application;

[0085] Figure 4 is a t-SNE visualization graph under the best accuracy of the present application. DETAILED DESCRIPTION

[0086] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings, but the protection scope of the present application is not limited to the described embodiments.

[0087] Experimental results:

[0088] Experiment 1, MSAW effectiveness experiment:

[0089] To comprehensively verify the effectiveness and generalization ability of the MSAW method, a system comparison experiment was conducted on a dataset containing 10 different satellite targets (5000 pairs of training samples and 5000 pairs of test samples). The experiment used 9 representative deep learning backbone networks as feature extractors, covering convolutional neural network classes (ResNet, ResNeXt, DenseNet, EfficientNet), lightweight network classes (MobileNet, MobileViT), Transformer classes (ViT, SwinTransformer), and state space model classes (MambaVision), and compared them with 6 mainstream multi-task learning weight adjustment methods (fixed weight method Fixed, uncertainty weighting method Uncertainty, gradient normalization method GradNorm, dynamic weight average method DWA, automatic Lambda method Auto-Lambda, loss scale normalization method Loss Balanced), a total of 63 independent experiments. The experimental results show that the MSAW method achieves the best performance under all 9 backbone network configurations, with an average fusion branch classification accuracy of 94.12%, significantly better than the second-best method Fixed of 93.21% (0.91 percentage points higher), DWA of 92.98% (1.14 percentage points higher), Auto-Lambda of 92.92% (1.20 percentage points higher), Loss Balanced of 92.85% (1.27 percentage points higher), Uncertainty of 92.42% (1.70 percentage points higher), and GradNorm of 92.38% (1.74 percentage points higher). In particular, on the Swin Transformer backbone network, the MSAW method achieved a peak accuracy of 96.06%, an increase of 0.90 percentage points over the second-best method DWA (95.16%) under this configuration; on the ResNet backbone network, the MSAW method achieved an accuracy of 91.23%, an increase of 1.32 percentage points over the second-best method Loss Balanced (89.91%) under this configuration, with the largest increase in all backbones, indicating that MSAW can provide significant gains for relatively weak backbones. Performance analysis of different architecture types shows that the Transformer class architecture (Swin, ViT, MambaVision) achieved an average accuracy of 95.22% under the MSAW framework, the convolutional neural network class achieved an average accuracy of 93.77%, and the lightweight network class achieved an average accuracy of 93.16%, fully verifying the broad applicability and architecture independence of the MSAW method.

[0090] The specific experimental accuracy Acc results are shown in the following table (unit %):

[0091]

[0092] Based on the comprehensive comparative experiments of nine backbone networks, the MSAW method achieved an average fusion branch classification accuracy of 94.12%, a 0.91 percentage point improvement compared to the fixed-weight baseline's 93.21%, and a 1.20 percentage point improvement compared to the suboptimal Auto-Lambda method's 92.92%. It achieved optimal performance in all 63 comparative experiments, fully validating the effectiveness of the modal collaborative adaptive weight adjustment mechanism. Under the optimal configuration represented by the Swin Transformer, the fusion branch accuracy reached 96.06%. Confusion matrix analysis showed that the recognition precision for all 10 types of satellite targets was above 90%, and the recall rate was above 89%. The recognition performance across categories was balanced, with no obvious bias. In particular, the model effectively distinguished between communication satellites equipped with different numbers of solar panels, navigation satellites, and reconnaissance satellites without solar panels. The average AUC value of the ROC curve reached 0.996, indicating that the model has excellent classification confidence and class separability, outputting high-confidence correct predictions even on difficult samples. t-SNE dimensionality reduction visualization analysis shows that, compared with single-modal features, fused features exhibit more compact intra-class clustering and more obvious inter-class separation in high-dimensional space. Feature points of satellites of the same class are tightly clustered, while the boundaries between feature clusters of satellites of different classes are clear, verifying that the synergistic effect of the MSAW adaptive weight mechanism can effectively extract discriminative feature representations. Regarding training efficiency, experimental measurements show that the MSAW weight calculation overhead (including dynamic calculation of confidence, consistency, difficulty, and temporal factors, as well as weight normalization) accounts for approximately 3.2% of the total training time, but in return, it achieves an average accuracy improvement of 0.91-1.74 percentage points (compared to different comparative methods). The efficiency-performance ratio is significantly better than the GradNorm method, which requires gradient norm calculation, and the Auto-Lambda method, which requires training extraneous networks. This demonstrates that the present invention achieves a good balance between accuracy and efficiency, providing a feasibility guarantee for practical engineering applications.

[0093] Experiment 2, QADA effectiveness experiment:

[0094]

[0095] To fully verify the effectiveness and superiority of the QADA (Query-Aware Dynamic Adaptive) fusion method proposed in this invention, a comparative experiment of system fusion methods was conducted under the optimal configuration (Swin Transformer backbone network + MSAW weight adjustment method). The experiment selected 12 representative multimodal fusion methods for comparison, covering basic concatenation (Concat), attention mechanisms (Cross Attention, Co-Attention, Transformer), adaptive fusion (PAMF, SCMF, Adaptive Fusion, MOFE), gated network (DGFN), feature alignment (PFDA, MAMF), and the QADA method proposed in this invention, totaling 12 independent experiments. Experimental results show that the QADA method of this invention achieves optimal performance, with a fusion branch classification accuracy of 96.06%, which is significantly better than the second-best methods PAMF (95.60%, an improvement of 0.46 percentage points), DGFN (95.36%, an improvement of 0.70 percentage points), Cross Attention and Concat (95.34%, an improvement of 0.72 percentage points), SCMF (95.20%, an improvement of 0.86 percentage points), MAMF (94.74%, an improvement of 1.32 percentage points), PFDA (94.70%, an improvement of 1.36 percentage points), Transformer (94.66%, an improvement of 1.40 percentage points), MOFE (94.32%, an improvement of 1.74 percentage points), Co-Attention (92.68%, an improvement of 3.38 percentage points), and Adaptive Fusion (91.50%, an improvement of 4.56 percentage points).

[0096] Performance analysis of different fusion strategies shows that adaptive fusion methods achieve an average accuracy of 94.27%, but with significant internal differences (QADA 96.06%, PAMF 95.60%, SCMF 95.20%, while traditional Adaptive Fusion is only 91.50%), indicating that the design of the adaptive mechanism is crucial. Attention mechanism methods achieve an average accuracy of 94.67%, with Cross Attention performing exceptionally well (95.34%), but Co-Attention being relatively weak (92.68%), demonstrating that the architecture design of the attention mechanism has a significant impact on performance. Among gated network methods, DGFN achieves a high accuracy of 95.36%, while MOFE achieves 94.32%, validating the effectiveness of the dynamic gating mechanism. The basic concatenation method Concat also achieves a good performance of 95.34%, showcasing the competitiveness of simple methods under a good backbone network. In particular, the QADA method improves performance by 4.56 percentage points compared to the traditional Adaptive Fusion method and by 0.72 percentage points compared to the basic Concat method, fully demonstrating the innovation and effectiveness of the query-aware dynamic adaptive mechanism.

[0097] Based on the comprehensive comparative experiments of 12 fusion methods, the QADA method achieved a fusion branch classification accuracy of 96.06%, which is 0.46 percentage points higher than the second-best method PAMF (95.60%) and 4.56 percentage points higher than the traditional adaptive fusion baseline (91.50%). It achieved the best performance in all 12 comparative experiments, fully validating the effectiveness of the query-aware dynamic adaptive fusion mechanism. Confusion matrix analysis showed that the QADA method achieved an accuracy of over 93% and a recall of over 92% for all 10 types of satellite targets. Compared to the second-best method PAMF, it improved the accuracy of difficult-to-classify categories (such as communication satellites and navigation satellites with similar solar panel configurations) by 1.2-2.1 percentage points, significantly improving the inter-class confusion problem. ROC curve analysis shows that the average AUC of the QADA method reaches 0.997, which is an improvement over PAMF's 0.994 and Concat's 0.993. The improvement in the true positive rate is particularly significant in the low false positive rate range (<1%), indicating stronger robustness of the model in high-confidence prediction scenarios. Feature visualization analysis, through attention weight heatmaps, shows that the QADA method can dynamically adjust the fusion weights according to the discriminative power of different modal features: for samples with obvious sailboard features, the optical branch weight automatically increases to 0.68-0.75; for samples where shape and contour are the main discriminative features, the SAR branch weight automatically adjusts to 0.62-0.71. This adaptive weight allocation mechanism results in better intra-class compactness (average reduction of intra-class distance by 8.3%) and inter-class separability (average increase of inter-class distance by 12.7%) in the high-dimensional space, verifying the synergistic enhancement effect of the query-aware mechanism. Training efficiency and convergence analysis show that the QADA method achieves optimal performance and triggers early stopping mechanism in an average of 12 epochs under the same training conditions. Compared with PAMF's 13 epochs, DGFN's 18 epochs, Transformer's 21 epochs, and Adaptive Fusion's 22 epochs, the convergence speed is 8.3%, 33.3%, 42.9%, and 45.5% faster, respectively. This proves that the query-aware dynamic adjustment mechanism not only improves the final accuracy but also accelerates model convergence. In terms of computational overhead, QADA's query-aware attention computation (including query generation, key-value mapping, and dynamic weight calculation) accounts for approximately 5.8% of the training time per epoch. Although this is slightly higher than the simple concatenation method Concat (0.2%) and the traditional attention method Cross Attention (4.1%), it actually reduces the total training time by about 38% through faster convergence speed (saving 10 epochs) and gains an accuracy improvement of 0.72-4.56 percentage points. The overall balance of efficiency, performance, and convergence speed is better than all the compared methods.Ablation experiments further validated that removing the query-aware mechanism reduced accuracy to 94.82% (a decrease of 1.24 percentage points), replacing it with fixed-weight fusion reduced accuracy to 95.34% (a decrease of 0.72 percentage points), and removing the dynamic adjustment module reduced accuracy to 95.18% (a decrease of 0.88 percentage points). This fully demonstrates the necessity and synergistic effect of the core components of QADA. In summary, the QADA method achieves optimal performance in accuracy, convergence speed, and feature discriminability, providing an efficient and reliable fusion solution for multimodal satellite target recognition tasks, and possesses significant theoretical innovation value and promising engineering application prospects.

Claims

1. A spatial target recognition method based on modal cooperative adaptive weighting for SAR and optical image fusion, characterized in that, The method includes the following steps: S1. Construct a multimodal dataset of satellite optical and SAR images. The dataset contains N pairs of registered SAR and optical images and their corresponding category labels. All images are normalized and uniformly adjusted to the appropriate size. Pixel resolution, performing data augmentation processing on training set images; S2. A parameter-independent dual encoder architecture is used to extract the depth feature representations of the optical imagery and SAR images obtained in S1, respectively. S3, using a two-layer fully connected neural network Predict the imaging quality of each modality of the depth features extracted from S2, achieve quality-aware fusion through quality-guided asymmetric cross attention, construct a quality-enhanced fusion feature representation, and calculate the basic loss and prediction probability of the three branches: optical branch, SAR branch, and fusion branch. S4. Based on the three-branch basic loss and prediction probability calculated in S3, calculate the loss weights of the optical branch, SAR branch and fusion branch respectively, and use the weighted total loss and consistency regularization to jointly optimize the parameters of the three-branch network until convergence; the three-branch network includes a dual encoder feature extraction network, a QADA quality-aware fusion network and a three-branch classifier network. S5. Save the model with the best parameters, and input the test data into the network to predict and identify satellite categories.

2. The spatial target recognition method with modal cooperative adaptive weighting for SAR fusion as described in claim 1, characterized in that, In step S1, the method for constructing the simulation dataset is as follows: A SAR and optical image registration dataset containing C-class satellite targets was obtained. The satellite categories include 10 categories, i.e., C=10. The optical images were acquired through visible light or infrared imaging, with an imaging wavelength range of 400-900nm and a spatial resolution better than 0.5m. The SAR images were acquired through X-band synthetic aperture radar imaging, with a center frequency of 9.6GHz, a bandwidth of 600MHz, and a resolution of 1m×1m. The entire dataset is represented as ,in, Let be the optical image of the k-th sample, where Indicates the height of the image. Indicates the width of the image. For the corresponding SAR image, The samples are 10,000 pairs, with 5,000 pairs used as training samples and 5,000 pairs used as test samples.

3. The spatial target recognition method based on modal cooperative adaptive weighting for SAR and optical image fusion as described in claim 1, characterized in that, In step S1, the normalization and feature enhancement methods are as follows: All images in the dataset were uniformly adjusted using bicubic interpolation. Pixel resolution, pixel value from Normalization to Then use the mean and standard deviation Standardize; Data augmentation of the training set images includes: random horizontal flipping with a probability of 0.5, and... Random rotation within the range, adjusting the brightness, contrast, and saturation of the optical image. Random adjustments within the range; adjustments only to brightness and contrast of SAR images; and the addition of a standard deviation to both SAR and optical images with a probability of 0.

3. Gaussian noise; all geometric transformation operations use the same parameters for the same sample in both SAR and optical images to ensure consistent spatial correspondence; The test set only undergoes size normalization and pixel value standardization, without any data augmentation operations.

4. The spatial target recognition method based on modal cooperative adaptive weighting for SAR and optical image fusion as described in claim 1, characterized in that, In step S2, the dual encoder extraction steps are as follows: The optical image after preprocessing in step S1 and SAR images Dual encoders with independent input parameters and In this model, the two encoders use the same backbone network architecture but with completely independent parameters to learn modality-specific features. The feature extraction calculation formula is as follows: in, Let be the optical depth feature vector of the i-th sample. Represents the real number field. Here, B represents the corresponding SAR depth feature vector, B is the batch size, and D is the feature dimension.

5. The spatial target recognition method based on modal cooperative adaptive weighting for SAR and optical image fusion according to claim 4, characterized in that, In step S3, the method for achieving quality perception fusion through quality-guided asymmetric cross-attention is as follows: The feature matrix obtained in step S2 Input two fully connected quality assessment networks respectively The network structure consists of two fully connected layers: the first layer maps D-dimensional features to 512 dimensions, and the second layer maps 512 dimensions to 1 dimension; then global average pooling is used. Compress to The mass fraction is then obtained by sigmoid activation. Normalized to relative quality weights: The optical and SAR modes, and the matrix of the optical mode, are obtained through the Xavier initialization method: in, It is the input feature dimension. To output feature dimensions, Indicates uniform distribution. Representative from arrive Uniform sampling The query projection matrix representing the optical modes. The bond projection matrix representing the optical modes. The projection matrix represents the values ​​of the optical modes. The query projection matrix representing the SAR mode. The key projection matrix representing the SAR mode. The projection matrix represents the values ​​of the SAR modes; Calculate the query, key, and value vectors required for the cross-attention mechanism using the obtained projection matrix: in The query vector representing the optical mode. The key vector representing the optical mode. The value vector representing the optical mode. The query vector representing the SAR mode. The key vector representing the SAR mode. The value vector representing the SAR mode; Quality-enhanced features are obtained by computationally guiding asymmetric cross-attention. : in, yes The transpose of the matrix, yes The transpose of the matrix; Concatenate the enhanced features into fused features : The predicted probabilities for three modalities are obtained using three independent classifiers: Where C represents the total number of categories. Represents an optical classifier. Represents the SAR classifier. This represents a fusion modality classifier; Calculate optical modal cross-entropy loss : in, Let c be the probability of the optical mode for category c. One-hot encoding for the category; Calculate SAR modal cross-entropy loss : in, This represents the probability of the SAR mode when the class is c; Calculate the cross-entropy loss of the fused modes : in, For the one-hot encoding of the current category c, This represents the probability of the fused mode when the category is c.

6. The spatial target recognition method based on modal cooperative adaptive weighting for SAR and optical image fusion as described in claim 5, characterized in that, The method for calculating the loss weights of the optical branch, SAR branch, and fusion branch in step S4 is as follows: The predicted probability obtained from step S3 Extract the maximum probability value for each sample and calculate the optical modality confidence score using batch averaging: in, Indicates the confidence level of the optical mode. This represents the predicted probability of the i-th sample for the c-th class in the optical branch; Calculate SAR mode confidence: Where i is the index of the i-th sample in the batch. This represents the predicted probability of the lower i samples for class c in the SAR branch; The cosine similarity of the predicted distribution is calculated for each training sample and then averaged in batches to obtain the consistency. in, This represents the consistency of the sample distribution. and Let be the optical and SAR prediction probability vectors of the i-th sample, respectively. Indicates calculation and Cosine similarity between them; fusion loss The sample difficulty is obtained through the Sigmoid mapping. : in, Represents the sigmoid function; Based on the model's training progress Stage weights, where, This is the current training progress. This is the current training round. This refers to the total number of training rounds; when hour, ; when hour, All ; when hour for ; in, This represents the weight of the optical mode at different stages. This represents the weight of the SAR mode at different stages. This indicates the weights of the fusion modes at different stages; Calculate the branch weights by combining four factors: Normalization ; Calculate the weighted total loss: Compute consistency regularization: in, Indicates calculation and The divergence between them; The total loss was calculated. : 。 7. The spatial target recognition method based on modal cooperative adaptive weighting for SAR and optical image fusion as described in claim 6, characterized in that, The method for step S5 is as follows: Load the model weights with the highest accuracy on the validation set during training, input 5000 pairs of SAR and optical image pairs from the test set into the network, and obtain the fusion branch prediction probability by sequentially passing through dual encoder feature extraction and quality-aware fusion. For each test sample, the category with the highest probability is taken as the prediction result to complete satellite category identification.

Images