A method, system, and medium for high-resolution large-scene SAR and optical image land cover classification based on cross-modal spatial dynamic fusion networks.

Abstract

A high-resolution large-scene SAR and optical imagery land cover classification method, system, and medium based on a cross-modal spatial dynamic fusion network are described. The method involves: acquiring SAR and optical images of the same area and forming a training dataset; constructing a cross-modal spatial dynamic fusion network model; training the cross-modal spatial dynamic fusion network model using the training dataset; acquiring SAR and optical images to be predicted, classifying them using the trained cross-modal spatial dynamic fusion network model to obtain the final semantic segmentation result of the image patches; merging the classification results of the image patches to obtain the final land cover classification result. The system, equipment, and medium are used to implement this method. Based on the multi-scale features of SAR and optical images, this invention utilizes a cross-modal attention mechanism for feature interaction to achieve inter-modal information exchange. Through a spatial dynamic fusion module, it efficiently fuses complementary information from SAR and optical features, thereby improving the performance of land cover classification.

Description

Technical Field

[0001] This invention belongs to the field of image land cover classification technology, and specifically relates to a method, system and medium for high-resolution large-scene SAR and optical image land cover classification based on cross-modal spatial dynamic fusion network. Background Technology

[0002] Land cover classification technology plays a crucial role in ecological surveillance, environmental protection, emergency disaster relief, and urban planning, and is one of the hot research topics in the field of remote sensing technology. With the widespread availability of remote sensing big data, it has become possible to acquire multi-source remote sensing images of the same land area. Utilizing the complementary characteristics of remote sensing images acquired through different imaging modalities provides a unique opportunity to improve the accuracy of land cover classification. Using multiple data sources instead of a single data source helps mitigate the ambiguity caused by the correlation between remote sensing imaging characteristics and certain land objects, and generates more diverse representations. For example, research is being conducted on joint land cover classification techniques using spectral information contained in optical images and scattering information provided by Synthetic Aperture Radar (SAR) images.

[0003] Specifically, SAR imagery, with its unique advantages of all-day and all-weather monitoring, has become a powerful tool for crop monitoring. SAR is an active sensor, sensitive to dielectric effects, texture, and orientation. SAR data contains phase and intensity information from various imaging modes, including single-mode, dual-mode, and polarization-mode images, which provide background information on crop phenology needed for deep learning-based feature extraction. However, soil roughness and vegetation growth can pose challenges to the application of SAR data, as SAR signal saturation, attributed to different combinations of soil roughness and leaf area index values, can produce identical signals, leading to indistinguishability. Optical imagery, providing richer spatial spectral information, can compensate for the shortcomings of SAR imagery. Therefore, combining SAR imagery with optical imagery within a spatiotemporal data fusion framework can effectively utilize rich spatial and spectral data, which are crucial for land cover classification.

[0004] The main challenges in optical and SAR image fusion can be attributed to two aspects. First, the semantic gap between the original image and the class label is difficult to bridge. Second, traditional manual feature extraction mechanisms cannot fully reflect land cover information in complex scenes. Recently, deep convolutional neural networks have achieved superior performance as powerful feature extractors. In computer vision tasks, they use layer-by-layer nonlinear transformations to intelligently extract deep semantic features, thereby narrowing the semantic gap between the original image and the class label. Since the imaging mechanisms of optical and SAR images are completely different, and there is a semantic gap between optical and SAR features, designing a reasonable heterogeneous modality fusion strategy, establishing good interaction between SAR and optical features, and effectively fusing optical and SAR features have become key issues that need to be addressed.

[0005] Patent application CN113420838A discloses a SAR and optical image classification method based on multi-scale attention feature fusion. This method uses a convolutional neural network for feature extraction, then uses spatial and spectral attention to enhance the extracted SAR and optical features, respectively, before concatenating them and using a fully connected layer for classification. The drawback of this method is that it only utilizes the features from the last layer of the network for concatenation and classification, without leveraging features from different scales for fusion classification. Because it does not fully utilize features at different scales, this method exhibits low robustness and poor classification accuracy when dealing with land cover classification fused from SAR and optical images.

[0006] Patent application CN116486142A discloses a multimodal image land cover type classification method based on network architecture search. This method utilizes a feature extraction network for SAR and optical feature extraction, constructs a multimodal feature fusion architecture search space based on the SMBO algorithm, and performs feature fusion to ultimately obtain the land cover type classification result. The drawback of this method is that when the network depth is very deep, the SMBO algorithm-based architecture search space requires a long time to search for the overall structure, and cannot guarantee the best fusion parameter strategy, resulting in poor search performance and insufficient classification accuracy. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, the present invention aims to provide a high-resolution large-scene SAR and optical image land cover classification method, system, and medium based on a cross-modal spatial dynamic fusion network. This method utilizes multi-scale features of SAR and optical images for fusion classification, effectively fusing shallow, low-level spatial features with deep, high-level semantic features for rich feature learning. When constructing features for fusion, a cross-modal attention mechanism is used to ensure good interaction between SAR and optical image features, thereby balancing SAR and optical features and providing highly discriminative features for fusion classification. The constructed spatial dynamic fusion module can effectively extract complementary information from SAR and optical images, suppress redundant information, and achieve high-precision SAR and optical image land cover fusion classification.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A high-resolution large-scene SAR and optical image land cover classification method based on cross-modal spatial dynamic fusion network includes the following steps:

[0010] Step 1. Collect SAR and optical images of the same area, register them, divide the SAR and optical images into blocks and crop them to obtain corresponding SAR image blocks and optical image blocks respectively, set labels for the SAR image blocks and optical image blocks, and use SAR images, optical images and manually labeled real labels as samples to form a training dataset.

[0011] Step 2. Construct a cross-modal space dynamic fusion network model;

[0012] Step 3. Input the SAR image patches and optical image patches in the training dataset obtained in Step 1 into the cross-modal space dynamic fusion network model constructed in Step 2 for classification. Use the loss function to train the cross-modal space dynamic fusion network model based on the manually labeled real labels to obtain the trained cross-modal space dynamic fusion network model.

[0013] Step 4. Acquire SAR and optical images to be predicted, and then divide and crop the SAR and optical images to be predicted into blocks to obtain image blocks to be classified. Input the image blocks to be classified into the cross-modal spatial dynamic fusion network model trained in Step 3 to obtain the final semantic segmentation result of the image blocks.

[0014] Step 5: The final semantic segmentation results of the image blocks obtained in Step 4 are stitched together according to the cropping order in Step 4 to obtain the final SAR image and optical image fusion land cover classification result.

[0015] The method for constructing the cross-modal space dynamic fusion network model in step 2 is as follows:

[0016] Assuming two initial images, denoted as X for the SAR image and Y for the optical image, let...

[0017]

[0018]

[0019] in, The feature extraction operation corresponding to the SAR image is performed, and features at three scales are obtained through calculation using formula (2). The corresponding Y can also be obtained using formula (2) to obtain the features at three scales.

[0020] Subsequently, the relevant scores are calculated using formula (3), as follows:

[0021]

[0022] in, and It is a linear transformation, which can be considered as two learnable weight matrices. Based on the relevance score, an attention mechanism is applied to the nodes. The relevance coefficient is defined to measure the influence of X on Y. It is obtained by normalizing the relevance score using the SoftMax function, as shown in the following formula:

[0023]

[0024] Next, we obtain richer cross-modal augmentation features. The formula is as follows:

[0025]

[0026] Where || denotes the vector concatenation operation, W v It is a learnable weight matrix, and ReLU represents a nonlinear mapping operation; correspondingly, the enhancement features of the Y images at three scales can also be obtained through the above formulas (3)(4)(5).

[0027] The features are then obtained using formula (6). The spatial attention factor is calculated as follows:

[0028]

[0029] Where g(·) represents the spatial average pooling operation, the spatial attention factors at three scales are obtained through the calculation of the above formula (6). Corresponding features The spatial attention factors at three scales can also be obtained using the above formula (6).

[0030] The obtained and The spatial fusion coefficients d1, d2, and d3 are obtained through formula (7), as follows:

[0031]

[0032] Where m(·,·) represents the feature multiplication operation; the fusion coefficients are nonlinearly mapped using the Sigmoid function, as shown in the following formula:

[0033]

[0034] Where Sigmoid(·) represents a nonlinear mapping operation; finally, the fusion features n1, n2, n3 at three scales are obtained through formula (9), as follows:

[0035]

[0036] Where u(·) represents the feature addition operation, and * represents the feature multiplication operation;

[0037] Finally, by upsampling and stitching the fused features n1, n2, and n3 at the three scales, the final classification features are obtained for land cover classification.

[0038] The loss function used in step 3 is constructed as follows:

[0039] Network loss function: cross-entropy loss L

[0040]

[0041] Where C represents the number of categories, y i p represents the true label set for the i-th class. i This represents the prediction result for the i-th class.

[0042] This invention also provides a high-resolution large-scene SAR and optical image land cover classification system based on a cross-modal spatial dynamic fusion network, comprising:

[0043] The training dataset acquisition module is used to collect SAR and optical images of the same area, register them, divide and crop the SAR and optical images into blocks to obtain corresponding SAR image blocks and optical image blocks, set labels for the SAR image blocks and optical image blocks, and use SAR images, optical images and manually labeled real labels as samples to form the training dataset.

[0044] A cross-modal space dynamic fusion network model building module is used to construct cross-modal space dynamic fusion network models.

[0045] The cross-modal dynamic fusion network model training module is used to input SAR image patches and optical image patches from the training dataset into the constructed cross-modal dynamic fusion network model for classification. Based on the manually labeled real labels, the loss function is used to train the cross-modal dynamic fusion network model to obtain the trained cross-modal dynamic fusion network model.

[0046] The final semantic segmentation result acquisition module is used to acquire SAR images and optical images to be predicted, and to sequentially segment and crop the SAR images and optical images to be predicted to obtain image blocks to be classified. The image blocks to be classified are then input into the trained cross-modal space dynamic fusion network model to obtain the final semantic segmentation result of the image blocks.

[0047] The land cover classification result acquisition module is used to stitch together the final semantic segmentation results of image blocks according to the corresponding cropping order to obtain the final SAR image and optical image fusion land cover classification result.

[0048] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the aforementioned high-resolution large-scene SAR and optical image land cover classification method based on a cross-modal spatial dynamic fusion network.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] 1. This invention uses multi-scale feature extraction to fuse and classify the multi-scale features of extracted SAR and optical images. This effectively achieves the fusion of shallow low-level spatial features and deep high-level semantic features during network training, thereby enabling the model to learn rich feature representations.

[0051] 2. This invention utilizes a multi-scale cross-modal attention mechanism to perform feature interaction between SAR and optical images based on the extracted multi-scale features, thereby achieving feature balance between SAR and optical images and providing well-balanced and highly discriminative semantic features for land cover classification.

[0052] 3. The present invention, through the construction of a cross-modal spatial dynamic fusion network model, can effectively extract complementary information from SAR images and optical images, and can suppress redundant information during the fusion process, thereby achieving the technical effect of high-precision SAR image and optical image land cover classification.

[0053] In summary, compared with existing technologies, the high-resolution large-scene SAR and optical image land cover classification method proposed in this invention, which utilizes the multi-scale features of SAR and optical images, fully leverages the cross-modal attention mechanism to facilitate feature interaction between SAR and optical images, enabling intermodal information exchange. Finally, the spatial dynamic fusion module efficiently fuses complementary information from SAR and optical features, thereby improving the performance of high-resolution large-scene SAR and optical image land cover classification. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0055] Figure 2 This is the SAR image corresponding to Xi'an-4 used in this invention.

[0056] Figure 3 This is the optical image corresponding to Xi'an-4 used in this invention.

[0057] Figure 4 This is the result map of the actual land cover classification corresponding to Xi'an No. 4 used in this invention.

[0058] Figure 5 This is the result of land cover classification of the SAR image corresponding to Xi'an-4 using the method of this invention.

[0059] Figure 6 This is the result of land cover classification of the optical image corresponding to Xi'an No. 4 using the method of this invention.

[0060] Figure 7 This is the result of land cover classification of SAR and optical images corresponding to Xi'an-4 using the method of this invention.

[0061] Figure 8 This is the SAR image corresponding to Xi'an-5 used in this invention.

[0062] Figure 9 This is the optical image corresponding to Xi'an-5 used in this invention.

[0063] Figure 10 This is the result map of the actual land cover classification corresponding to Xi'an No. 5 used in this invention.

[0064] Figure 11 This is the result of land cover classification of the SAR image corresponding to Xi'an-5 using the method of this invention.

[0065] Figure 12 This is the result of land cover classification of the optical image corresponding to Xi'an No. 5 using the method of this invention.

[0066] Figure 13 This is the result of land cover classification using the method of this invention for SAR and optical images corresponding to Xi'an-5. Detailed Implementation

[0067] The technical solution adopted by the present invention will be further described below with reference to the accompanying drawings.

[0068] like Figure 1 As shown, the high-resolution large-scene SAR and optical image land cover classification method based on cross-modal spatial dynamic fusion network includes the following steps:

[0069] Step 1. Collect SAR and optical images of the same area, register them, and then crop the SAR and optical images into blocks to obtain corresponding SAR image blocks and optical image blocks respectively. Block cropping facilitates efficient parallel processing and makes full use of computing resources. Then, label the SAR image blocks and optical image blocks, and use the SAR images, optical images, and manually labeled real labels as samples to form a training dataset;

[0070] Step 2. Construct a cross-modal space dynamic fusion network model;

[0071] The method for constructing the cross-modal space dynamic fusion network model in step 2 is as follows:

[0072] Assuming two initial images, denoted as X for the SAR image and Y for the optical image, let...

[0073]

[0074]

[0075] in, The feature extraction operation corresponding to the SAR image is performed, and features at three scales are obtained through calculation using formula (2). The corresponding Y can also be obtained using formula (2) to obtain the features at three scales. This allows the model to learn rich feature representations;

[0076] Subsequently, the relevant scores are calculated using formula (3), as follows:

[0077]

[0078] in, and It is a linear transformation, which can be considered as two learnable weight matrices. Based on the relevance score, an attention mechanism is applied to the nodes. The relevance coefficient is defined to measure the influence of X on Y. It is obtained by normalizing the relevance score using the SoftMax function, as shown in the following formula:

[0079]

[0080] Next, we obtain richer cross-modal augmentation features. The formula is as follows:

[0081]

[0082] Where || denotes the vector concatenation operation, W v It is a learnable weight matrix, and ReLU represents a nonlinear mapping operation; correspondingly, the enhancement features of the Y images at three scales can also be obtained through the above formulas (3)(4)(5). This enables feature interaction between SAR and optical images, providing well-balanced and highly discriminative semantic features for land cover classification.

[0083] The features are then obtained using formula (6). The spatial attention factor is calculated as follows:

[0084]

[0085] Where g(·) represents the spatial average pooling operation, the spatial attention factors at three scales are obtained through the calculation of the above formula (6). Corresponding features The spatial attention factors at three scales can also be obtained using the above formula (6).

[0086] The obtained and The spatial fusion coefficients d1, d2, and d3 are obtained through formula (7), as follows:

[0087]

[0088] Where m(·,·) represents the feature multiplication operation; the fusion coefficients are nonlinearly mapped using the Sigmoid function, as shown in the following formula:

[0089]

[0090] Where Sigmoid(·) represents a nonlinear mapping operation; finally, the fusion features n1, n2, n3 at three scales are obtained through formula (9), as follows:

[0091]

[0092] Where u(·) represents the feature addition operation and * represents the feature multiplication operation, effectively extracting complementary information from SAR images and optical images and suppressing redundant information;

[0093] Finally, land cover classification is performed by upsampling and stitching the fused features n1, n2, and n3 at the three scales.

[0094] Step 3. Input the SAR image patches and optical image patches in the training dataset obtained in Step 1 into the cross-modal space dynamic fusion network model constructed in Step 2 for classification. Train the cross-modal space dynamic fusion network model according to the manually labeled real labels to obtain the trained cross-modal space dynamic fusion network model.

[0095] The loss function used in step 3 is constructed as follows:

[0096] Network loss function: cross-entropy loss L

[0097]

[0098] Where C represents the number of categories, y i p represents the true label set for the i-th class. i This represents the prediction result for the i-th class.

[0099] Step 4. Acquire SAR and optical images to be predicted, and then divide and crop the SAR and optical images to be predicted into blocks to obtain image blocks to be classified. Input the image blocks to be classified into the cross-modal spatial dynamic fusion network model trained in Step 3 to obtain the final semantic segmentation result of the image blocks.

[0100] Step 5: The final semantic segmentation results of the image blocks obtained in Step 4 are stitched together according to the cropping order in Step 4 to obtain the final SAR image and optical image fusion land cover classification result.

[0101] Existing technologies for land cover fusion classification using SAR and optical imagery suffer from insufficient utilization of multi-scale features, lack of cross-modal feature interaction, and reliance on simple stitching methods during fusion. These methods fail to effectively utilize the complementary information and filter redundant information from SAR and optical imagery, resulting in low performance, insufficient robustness, and a lack of accuracy. This invention utilizes a cross-modal attention mechanism and spatial fusion strategy to improve the accuracy of SAR-optical land cover fusion classification. This solves the problem of insufficient performance in existing CNN-based fusion classification methods due to the lack of good interaction between SAR and optical features and the inadequate utilization of complementary information.

[0102] Experimental Analysis

[0103] The dataset used in the simulation experiment of this invention consists of five SAR and optical image data covering the Xi'an area of ​​Shaanxi Province. The original images were cropped and randomly divided into training, validation, and test sets. The dataset contains 5999 image patches of size 256×256. The data parameters used in this invention are shown in Table 1 below.

[0104] Table 1. Detailed parameters of the data used in this invention.

[0105] size 11641*12475 9537*10923 9989*10419 10650*11725 9960*11076

[0106] The quantitative evaluation results of the simulation experiment of this invention for Xi'an No. 4 data are shown in Table 2 below, and the quantitative evaluation results for Xi'an No. 5 data are shown in Table 3 below. Figure 2 , Figure 3 and Figure 4 These are the results of SAR imagery, optical imagery, and ground true land cover classification corresponding to Xi'an No. 4 data. Figure 5 , Figure 6 and Figure 7 The images show land cover classification results using SAR imagery-CNN method, optical imagery-CNN method, and the method of this invention, respectively, corresponding to Xi'an Satellite-5 data. Figure 5 and Figure 6 It can be seen that the land cover classification results from optical imagery are better than those from SAR imagery. This is inseparable from the rich spectral features of optical imagery itself. Figure 7 As can be seen, this invention utilizes SAR and optical imagery for cross-modal spatial dynamic fusion land cover classification, achieving a significant improvement over SAR image classification and optical image classification alone. This is inseparable from the spatial dynamic fusion strategy proposed in this invention, which effectively utilizes the complementary features of SAR and optical imagery and filters redundant features during the fusion process, thereby improving the accuracy of land cover classification. Accordingly, Figure 8 , Figure 9 and Figure 10 These are the results of SAR imagery, optical imagery, and ground real land cover classification corresponding to Xi'an No. 5 data. Figure 11 , Figure 12 and Figure 13 The images show land cover classification results using SAR imagery-CNN method, optical imagery-CNN method, and the method of this invention, respectively, corresponding to Xi'an Satellite-5 data. Figure 11 , Figure 12 and Figure 13The advantages of the proposed method can also be seen in the Xi'an No. 5 data. Compared with single-source SAR imagery or optical imagery, the proposed cross-modal spatial dynamic fusion strategy achieves better performance. The advantages of this invention can also be seen in the quantitative evaluation indicators OA and Kappa coefficients in Tables 2 and 3. This invention provides more refined and accurate classification results for categories such as water bodies and villages compared to single-source SAR imagery and optical imagery coverage classification methods.

[0107] Table 2. Quantitative evaluation results of the classification results of this invention on Xi'an City Data No. 4.

[0108] SAR-CNN method 70.49％ 51.64％ Optical-CNN method 78.94％ 66.78％ Method of the present invention 83.37％ 74.09％

[0109] Table 3. Quantitative evaluation results of the classification results of this invention on Xi'an City Data No. 5.

[0110] SAR-CNN method 79.65％ 58.00％ Optical-CNN method 82.85％ 65.62％ Method of the present invention 84.54％ 68.18％

[0111] The simulation experiments above show that, compared with the SAR image CNN land cover classification method and the optical image CNN land cover classification method, the land cover fusion classification performance of this invention, based on a cross-modal spatial dynamic fusion strategy, is significantly improved. Figure 7 and Figure 13 It can be seen that the method of the present invention has better classification integrity in various land cover categories and achieves better classification performance.

[0112] This invention also provides a high-resolution large-scene SAR and optical image land cover classification system based on a cross-modal spatial dynamic fusion network, comprising:

[0113] The training dataset acquisition module is used to collect SAR and optical images of the same area in step 1, register them, divide and crop the SAR and optical images into blocks to obtain corresponding SAR image blocks and optical image blocks, set labels for the SAR image blocks and optical image blocks, and use SAR images, optical images and manually labeled real labels as samples to form a training dataset.

[0114] A cross-modal space dynamic fusion network model construction module is used to construct the cross-modal space dynamic fusion network model in step 2.

[0115] The cross-modal dynamic fusion network model training module is used to input the SAR image blocks and optical image blocks in the training dataset obtained in step 1 into the cross-modal dynamic fusion network model constructed in step 2 for classification in step 3. The cross-modal dynamic fusion network model is trained using a loss function based on the manually labeled real labels to obtain the trained cross-modal dynamic fusion network model.

[0116] The final semantic segmentation result acquisition module is used to acquire the SAR image and optical image to be predicted in step 4, and to divide and crop the SAR image and optical image to be predicted into blocks in sequence to obtain the image block to be classified. The image block to be classified is input into the cross-modal space dynamic fusion network model trained in step 3 to obtain the final semantic segmentation result of the image block.

[0117] The land cover classification result acquisition module is used to stitch together the final semantic segmentation results of the image blocks obtained in step 4 according to the cropping order in step 4 in step 5, so as to obtain the final SAR image and optical image fusion land cover classification result.

[0118] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the aforementioned high-resolution large-scene SAR and optical image land cover classification method based on a cross-modal spatial dynamic fusion network.

Claims

1. A high-resolution large-scene SAR and optical image land cover classification method based on cross-modal spatial dynamic fusion network, characterized in that, Includes the following steps: Step 1. Collect SAR and optical images of the same area, register them, divide the SAR and optical images into blocks and crop them to obtain corresponding SAR image blocks and optical image blocks respectively, set labels for the SAR image blocks and optical image blocks, and use the SAR images, optical images and manually labeled real labels as samples to form a training dataset. Step 2. Construct a cross-modal space dynamic fusion network model; Step 3. Input the SAR image patches and optical image patches in the training dataset obtained in Step 1 into the cross-modal space dynamic fusion network model constructed in Step 2 for classification. Use the loss function to train the cross-modal space dynamic fusion network model based on the manually labeled real labels to obtain the trained cross-modal space dynamic fusion network model. Step 4. Acquire SAR and optical images to be predicted, and then divide and crop the SAR and optical images to be predicted into blocks to obtain image blocks to be classified. Input the image blocks to be classified into the cross-modal spatial dynamic fusion network model trained in Step 3 to obtain the final semantic segmentation result of the image blocks. Step 5. The final semantic segmentation results of the image patches obtained in Step 4 are stitched together according to the cropping order in Step 4 to obtain the final SAR image and optical image fusion land cover classification results. The method for constructing the cross-modal space dynamic fusion network model in step 2 is as follows: Assuming two initial images, denoted as X for the SAR image and Y for the optical image, let... （1） （2） in, The feature extraction operation corresponding to the SAR image is performed, and features at three scales are obtained through calculation using formula (2). , , The corresponding Y can also be obtained using formula (2) to obtain the features of the three scales. , , ; Subsequently, the relevant scores are calculated using formula (3), as follows: （3） in, and It is a linear transformation, which can be considered as two learnable weight matrices. Based on the relevance score, an attention mechanism is applied to the nodes. The relevance coefficient is defined to measure the influence of X on Y. It is obtained by normalizing the relevance score using the SoftMax function, as shown in the following formula: （4） Next, we obtain richer cross-modal augmentation features. The formula is as follows: （5） in, This represents a vector concatenation operation. It is a learnable weight matrix, and ReLU represents a nonlinear mapping operation; correspondingly, the enhancement features of the Y images at three scales can also be obtained through the above formulas (3), (4), and (5). , , ; The features are then obtained using formula (6). The spatial attention factor is given by the following formula: （6） in, Representing the spatial average pooling operation, the spatial attention factors at three scales are obtained through the calculation using the above formula (6). , , Corresponding features The spatial attention factors at three scales can also be obtained using the above formula (6). , , ; The obtained and The spatial fusion coefficient is obtained through formula (7). , , The formula is as follows: （7） in, This represents the feature multiplication operation; the fusion coefficients are non-linearly mapped using the Sigmoid function, as shown in the following formula: （8） in, This represents a nonlinear mapping operation; finally, the fused features of the three scales are obtained through formula (9). , , The formula is as follows: （9） in, This indicates a feature addition operation. This indicates a feature multiplication operation; Finally, through the fusion features of the three scales... , , Upsampling and stitching operations are performed to obtain the final classification features for land cover classification.

2. The high-resolution large-scene SAR and optical image land cover classification method based on cross-modal spatial dynamic fusion network according to claim 1, characterized in that, The loss function used in step 3 is constructed as follows: Network loss function: cross-entropy loss : （10） in, Indicates the number of categories. Indicates the first The actual label set by the class Indicates the first The prediction results for the class.

3. A high-resolution large-scene SAR and optical image land cover classification system based on a cross-modal spatial dynamic fusion network, used to implement the method described in claim 1, characterized in that, include: The training dataset acquisition module is used to collect SAR and optical images of the same area, register them, divide and crop the SAR and optical images into blocks to obtain corresponding SAR image blocks and optical image blocks, set labels for the SAR image blocks and optical image blocks, and use SAR images, optical images and manually labeled real labels as samples to form the training dataset. A cross-modal space dynamic fusion network model building module is used to construct cross-modal space dynamic fusion network models. The cross-modal dynamic fusion network model training module is used to input SAR image patches and optical image patches from the training dataset into the constructed cross-modal dynamic fusion network model for classification. Based on the manually labeled real labels, the loss function is used to train the cross-modal dynamic fusion network model to obtain the trained cross-modal dynamic fusion network model. The final semantic segmentation result acquisition module is used to acquire SAR images and optical images to be predicted, and to sequentially segment and crop the SAR images and optical images to be predicted to obtain image blocks to be classified. The image blocks to be classified are then input into the trained cross-modal space dynamic fusion network model to obtain the final semantic segmentation result of the image blocks. The land cover classification result acquisition module is used to stitch together the final semantic segmentation results of image blocks according to the corresponding cropping order to obtain the final SAR image and optical image fusion land cover classification result.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the high-resolution large-scene SAR and optical image land cover classification method based on a cross-modal spatial dynamic fusion network as described in any one of claims 1-2.

Images