A DCNN-based land use semantic segmentation method for multi-source remote sensing images

Abstract

The invention discloses a multi-source remote sensing image land use semantic segmentation method based on DCNN, belonging to the field of deep learning technology. The method steps of the invention are as follows: S100, network input and feature extraction, image preprocessing; S200, using several independent convolutional flows to process optical images and SAR images; S300, designing a multimodal attention module, using 1×1Conv to transform the features of multispectral images into three identical feature maps; S400, using the GELU activation function to obtain a self-attention map of a single data source; S500, the low-level optical feature map and SAR feature map are downsampled and stacked to obtain a high-level feature map; S600, the feature map is sent to the context information enhancement module to extract multi-scale feature representation, and upsampled to obtain a multi-scale joint feature map; S700, the low-level joint feature map is stacked and sent to the decoder.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a DCNN-based multi-source remote sensing image land use semantic segmentation method. Background Art

[0002] Remote sensing image semantic segmentation is an important remote sensing image processing task, which aims to accurately segment and label different objects and object categories in the image. Its main goal is to classify images into different categories, such as buildings, roads, water bodies, vegetation, etc., based on the pixel-level information in the image, so as to provide detailed object classification and boundary information. Remote sensing image semantic segmentation has a wide range of applications in many fields, including urban planning, land management, agricultural monitoring, natural disaster warning, etc. By providing high-precision object information, it helps to better support various practical decisions and applications. This process is usually achieved using computer vision and deep learning techniques.

[0003] Changes in geographic regions, seasons, sensors, and environmental conditions pose a challenge to the generalization ability of the model. Since remote sensing images usually cover multiple geographic regions and seasons, the model needs to be able to adapt to these changes without losing segmentation accuracy. Images acquired by different sensors have different characteristics, noise, and resolution. Effectively fusing multiple remote sensing images to improve the performance of the model is a complex problem that requires consideration of differences between sensors and data fusion methods. In the training data, there may be an unbalanced distribution of object categories, which may cause the model to perform poorly on some categories. Dealing with unbalanced data distribution is an important challenge, and effective strategies are needed to balance the samples of each category. Summary of the invention

[0004] The object of the present invention is to provide a rehabilitation data transmission system and method adapted for an ultrasonic uterine involution instrument, so as to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] A DCNN-based land use semantic segmentation method for multi-source remote sensing images, the steps are as follows:

[0007] S100, performing network input and feature extraction, and preprocessing the image;

[0008] S200, using several independent convolution streams to process optical images and SAR images, each convolution stream focuses on the image type it processes;

[0009] S300, design a multimodal attention module and use 1×1Conv to transform the features of multispectral images into three identical feature maps;

[0010] S400, using a GELU activation function to obtain a self-attention map of a single data source;

[0011] At S500, the low-level optical feature maps and SAR feature maps are downsampled and stacked to obtain high-level feature maps;

[0012] S600, the feature map is sent to the context information enhancement module to extract multi-scale feature representation, and up-sampled to obtain a multi-scale joint feature map;

[0013] S700, stack the low-level joint feature maps and send them to the decoder.

[0014] According to step S100, the input content of the network includes independent multispectral images, hyperspectral images, original images of SAR images and label maps of SAR images, and the images and labels correspond to each other pixel by pixel;

[0015] Aiming at the differences in imaging characteristics of optical and SAR images, a multi-branch structure is designed. In the initial stage of feature extraction, the image is preprocessed to eliminate noise, correct radiometric data, and adjust image geometric transformation. Then, the optical and SAR image blocks are input into independent convolutional streams with the same structure for feature extraction without sharing parameters.

[0016] According to step S200, in the convolution layer, the convolution kernel is used to perform convolution operation with the image to extract information about the edge, texture, color and shape of the image; optical images and SAR images may have significant differences in expressing objects due to their different physical properties. For example, optical images record the reflection and absorption of objects, while SAR images depict objects by measuring the return of electromagnetic waves. This difference may cause the same object to show completely different characteristics in different images. The design of the multi-branch structure of the present invention can effectively counteract this interference.

[0017] By using several independent convolutional streams to process optical images and SAR images, the network is allowed to extract the most representative features of optical and SAR images respectively, and each convolutional stream focuses on the type of image it processes. Such an architecture allows the network to extract the most representative features of optical and SAR images respectively without paying too much attention to the differences between them. This capability of the multi-branch structure can alleviate the impact of differences in the expression of ground objects, allowing the model to better learn the respective features of optical and SAR images and effectively integrate them into the final feature representation, which helps to improve the generalization ability and performance of the model.

[0018] Here we take the high-level feature flow as an example to explain it in detail. The low-level feature flow is similar: the optical and SAR images are input into the convolution flow respectively, and the high-level features of any depth unit L of the optical image are expressed as follows through recursive operation:

[0019]

[0020] in, is the high-level feature of multispectral images, i is the unit of each convolutional layer, l is a shallow unit, is a linear projection, is a feature map learned from multispectral image patches;

[0021] The high-level features of hyperspectral images are expressed as:

[0022]

[0023] in, is the high-level feature of the hyperspectral image, i is each convolutional layer unit, l is a shallow unit, is a linear projection, is a feature map learned from a hyperspectral image patch;

[0024] The high-level features of SAR images are expressed as:

[0025]

[0026] in, is the high-level feature of SAR image, i is each convolutional layer unit, l is a shallow unit, is a linear projection, is a feature map learned from SAR image patches.

[0027] According to step S300, 1×1Conv is used to transform the features of the multispectral image into three identical feature maps V MSI , Q MSI and K MSI , 1×1Conv is used to convert feature maps; multispectral images contain several spectral bands, each of which represents a different spectral information. By performing 1×1Conv operations on the features in the multispectral images, they are converted into three identical feature maps;

[0028] During the conversion process, 1×1Conv is used to transform the number of channels in the original multispectral image into three identical feature maps. In fact, each feature is recombined in the feature space through linear transformation, and the same number of feature maps are generated. The three feature maps obtained after the transformation compress and re-encode the features of the original multispectral image to a certain extent, but each feature map still retains part of the original feature information, which helps to extract and strengthen certain features in the data. The generation of this feature map reduces the complexity of the data while retaining key features, and improves the ability of subsequent models to understand and extract features. The same is true for HSI and SAR. The feature matrix Q MSI and K MSI Multiply the transpose of to get the pixel-level product result C, the formula is as follows:

[0029]

[0030] C represents the relationship or mutual information between the corresponding features at different positions in the input image. This operation is used to enhance the connection between features or emphasize their importance, which helps to improve feature representation so that it can be better used for subsequent neural network layers or models for processing.

[0031] According to step S400, the self-attention map of a single data source is obtained using the GELU activation function. The GELU activation function is used because the idea of ​​random regularization is added to GELU, which improves the accuracy of the network. Secondly, GELU introduces the properties of Gaussian distribution to make the activation function smoother and continuously differentiable, which helps to alleviate the gradient vanishing problem. In addition, GELU can better model the nonlinear relationship of input data, improve the model's fitting ability, and achieve better performance in some deep neural networks. Its smooth characteristics help reduce the gradient explosion or gradient vanishing problems that occur during network training, and thus help improve the training stability and convergence speed of the model.

[0032] Capturing the feature matrix through matrix multiplication and K MSI The internal correlation between them is realized in the neural network through the attention mechanism. The matrix multiplication calculation of the attention mechanism assigns a weight to each feature, and the weight indicates the correlation degree between the current feature and other features. The matrix multiplication allows the model to learn the long-range dependency between features, which is used to model the context information when the features are not directly adjacent or far away in the input.

[0033] Since the information contained in MSI, HSI and SAR is independent and complementary, the information contained in MSI, HSI and SAR images is used to obtain the attention map of high-level feature cross-fusion using Hadamard product, and the attention map of high-level feature cross-fusion is obtained by element-by-element multiplication; the attention map reflects the correlation and co-occurrence pattern between different source data, which helps the subsequent deep learning model to better understand the information of the entire data set; the low-level matrix and the high-level matrix are weighted, and the weight W is determined according to the needs and experimental design a , W b and W c , calculate the weighted feature matrix:

[0034] A weight =A·W a

[0035] B weight =B·W b

[0036] C weight =C.W c

[0037] Among them, the matrices are A, B and C respectively.

[0038] The weighted feature matrix is ​​cross-fused to generate a joint attention map. The cross-fusion is completed by element-by-element multiplication. The formula is as follows:

[0039] D=A weight ·B weight ·C weight

[0040] D represents the final joint attention map matrix. During the cross-fusion process, the weighted operation adjusts each feature matrix and assigns different weights according to the importance or relevance of each feature matrix. Finally, the weighted feature maps are cross-fused to obtain the joint attention maps of MSI, HSI and SAR.

[0041] According to step S600, the input features are branched in multiple ways, and the obtained features have receptive fields of different sizes. By setting the void ratio, the corresponding receptive field and the global receptive field brought by global pooling are obtained; the use of receptive fields of several scales in deep neural networks allows the model to observe input data from different angles and capture contextual information of different scales; multi-scale feature extraction can mine relevant information in high-level semantics, and the relevant information includes the edges, textures and overall structures of objects; by utilizing several receptive fields, the model obtains features of different granularities, thereby understanding the input data.

[0042] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0043] The present invention uses multi-scale dilated convolution, pyramid pooling and other technologies to further improve the fusion of multi-scale information. In some cases, the segmentation task needs to capture both detail information and context information. The pyramid structure used in this model contains multiple parallel convolution branches, each of which processes information of different scales.

[0044] Deep learning models require a lot of storage space and computing resources in semantic segmentation, which has always been a pain point in the field of semantic segmentation. The present invention can use different backbone networks. This flexibility enables the model to adapt to different application requirements and computing resources, balancing performance and efficiency.

[0045] This model uses multispectral data, hyperspectral data and radar data as data sources, eliminating the need to choose between spectral information and geometric information, which improves the quality of the data and ensures the high accuracy of the model in semantic segmentation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1 It is a schematic diagram of the steps of a multi-source remote sensing image land use semantic segmentation method based on DCNN of the present invention;

[0048] Figure 2 It is a schematic diagram of context information enhancement steps of a DCNN-based multi-source remote sensing image land use semantic segmentation method of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] See also Figure 1 and Figure 2 , the present invention provides a technical solution:

[0051] According to one embodiment of the present invention, Figure 1 A method for land use semantic segmentation of multi-source remote sensing images based on DCNN is shown in the schematic diagram. The steps of a method for land use semantic segmentation of multi-source remote sensing images based on DCNN are as follows:

[0052] S100, performing network input and feature extraction, and preprocessing the image;

[0053] S200, using several independent convolution streams to process optical images and SAR images, each convolution stream focuses on the image type it processes;

[0054] S300, design a multimodal attention module and use 1×1Conv to transform the features of multispectral images into three identical feature maps;

[0055] S400, using a GELU activation function to obtain a self-attention map of a single data source;

[0056] At S500, the low-level optical feature maps and SAR feature maps are downsampled and stacked to obtain high-level feature maps;

[0057] S600, the feature map is sent to the context information enhancement module to extract multi-scale feature representation, and up-sampled to obtain a multi-scale joint feature map;

[0058] S700, stack the low-level joint feature maps and send them to the decoder.

[0059] According to step S100, the input content of the network includes independent multispectral images, hyperspectral images, original images of SAR images and label maps of SAR images, and the images and labels correspond to each other pixel by pixel;

[0060] Aiming at the differences in imaging characteristics of optical and SAR images, a multi-branch structure is designed. In the initial stage of feature extraction, the image is preprocessed to eliminate noise, correct radiometric data, and adjust image geometric transformation. Then, the optical and SAR image blocks are input into independent convolutional streams with the same structure for feature extraction without sharing parameters.

[0061] According to step S200, in the convolution layer, the convolution kernel is used to perform convolution operation with the image to extract information about the edge, texture, color and shape of the image; optical images and SAR images may have significant differences in expressing objects due to their different physical properties. For example, optical images record the reflection and absorption of objects, while SAR images depict objects by measuring the return of electromagnetic waves. This difference may cause the same object to show completely different characteristics in different images. The design of the multi-branch structure of the present invention can effectively counteract this interference.

[0062] By using several independent convolutional streams to process optical images and SAR images, the network is allowed to extract the most representative features of optical and SAR images respectively, and each convolutional stream focuses on the type of image it processes. Such an architecture allows the network to extract the most representative features of optical and SAR images respectively without paying too much attention to the differences between them. This capability of the multi-branch structure can alleviate the impact of differences in the expression of ground objects, allowing the model to better learn the respective features of optical and SAR images and effectively integrate them into the final feature representation, which helps to improve the generalization ability and performance of the model.

[0063] Here we take the high-level feature flow as an example to explain it in detail. The low-level feature flow is similar: the optical and SAR images are input into the convolution flow respectively, and the high-level features of any depth unit L of the optical image are expressed as follows through recursive operation:

[0064]

[0065] in, is the high-level feature of multispectral images, i is the unit of each convolutional layer, l is a shallow unit, is a linear projection, is a feature map learned from multispectral image patches;

[0066] The high-level features of hyperspectral images are expressed as:

[0067]

[0068] in, is the high-level feature of the hyperspectral image, i is each convolutional layer unit, l is a shallow unit, is a linear projection, is a feature map learned from a hyperspectral image patch;

[0069] The high-level features of SAR images are expressed as:

[0070]

[0071] in, is the high-level feature of SAR image, i is each convolutional layer unit, l is a shallow unit, is a linear projection, is a feature map learned from SAR image patches.

[0072] According to step S300, 1×1Conv is used to transform the features of the multispectral image into three identical feature maps V MSI , Q MSI and K MSI, 1×1Conv is used to convert feature maps; multispectral images contain several spectral bands, each of which represents a different spectral information. By performing 1×1Conv operations on the features in the multispectral images, they are converted into three identical feature maps;

[0073] During the conversion process, 1×1Conv is used to transform the number of channels in the original multispectral image into three identical feature maps. In fact, each feature is recombined in the feature space through linear transformation, and the same number of feature maps are generated. The three feature maps obtained after the transformation compress and re-encode the features of the original multispectral image to a certain extent, but each feature map still retains part of the original feature information, which helps to extract and strengthen certain features in the data. The generation of this feature map reduces the complexity of the data while retaining key features, and improves the ability of subsequent models to understand and extract features. The same is true for HSI and SAR. The feature matrix Q MSI and K MSI Multiply the transpose of to get the pixel-level product result C, the formula is as follows:

[0074]

[0075] C represents the relationship or mutual information between the corresponding features at different positions in the input image. This operation is used to enhance the connection between features or emphasize their importance, which helps to improve feature representation so that it can be better used for subsequent neural network layers or models for processing.

[0076] According to step S400, the self-attention map of a single data source is obtained using the GELU activation function. The GELU activation function is used because the idea of ​​random regularization is added to GELU, which improves the accuracy of the network. Secondly, GELU introduces the properties of Gaussian distribution to make the activation function smoother and continuously differentiable, which helps to alleviate the gradient vanishing problem. In addition, GELU can better model the nonlinear relationship of input data, improve the model's fitting ability, and achieve better performance in some deep neural networks. Its smooth characteristics help reduce the gradient explosion or gradient vanishing problems that occur during network training, and thus help improve the training stability and convergence speed of the model.

[0077] Capturing the feature matrix through matrix multiplication and K MSIThe internal correlation between them is realized in the neural network through the attention mechanism. The matrix multiplication calculation of the attention mechanism assigns a weight to each feature, and the weight indicates the correlation degree between the current feature and other features. The matrix multiplication allows the model to learn the long-range dependency between features, which is used to model the context information when the features are not directly adjacent or far away in the input.

[0078] Since the information contained in MSI, HSI and SAR is independent and complementary, the information contained in MSI, HSI and SAR images is used to obtain the attention map of high-level feature cross-fusion using Hadamard product, and the attention map of high-level feature cross-fusion is obtained by element-by-element multiplication; the attention map reflects the correlation and co-occurrence pattern between different source data, which helps the subsequent deep learning model to better understand the information of the entire data set; the low-level matrix and the high-level matrix are weighted, and the weight W is determined according to the needs and experimental design a , W b and W c , calculate the weighted feature matrix:

[0079] A weight =A·W a

[0080] B weight =B·W b

[0081] C weight =C.W c

[0082] Among them, the matrices are A, B and C respectively.

[0083] The weighted feature matrix is ​​cross-fused to generate a joint attention map. The cross-fusion is completed by element-by-element multiplication. The formula is as follows:

[0084] D=A weight ·B weight ·C weight

[0085] D represents the final joint attention map matrix. During the cross-fusion process, the weighted operation adjusts each feature matrix and assigns different weights according to the importance or relevance of each feature matrix. Finally, the weighted feature maps are cross-fused to obtain the joint attention maps of MSI, HSI and SAR.

[0086] According to step S600, the input features are branched in multiple ways, and the obtained features have receptive fields of different sizes. By setting the void ratio, the corresponding receptive field and the global receptive field brought by global pooling are obtained; the use of receptive fields of several scales in deep neural networks allows the model to observe input data from different angles and capture contextual information of different scales; multi-scale feature extraction can mine relevant information in high-level semantics, and the relevant information includes the edges, textures and overall structures of objects; by utilizing several receptive fields, the model obtains features of different granularities, thereby understanding the input data.

[0087] According to another embodiment of the present invention, Figure 2 A schematic diagram of the context information enhancement steps of a DCNN-based multi-source remote sensing image land use semantic segmentation method is shown in the figure. The receptive field of the deep convolution block in the traditional semantic segmentation model is small and fixed, which limits the retention of spatial information of the feature map and the capture of high-level semantic features. In order to optimize the utilization efficiency, the low-level optical feature maps and SAR feature maps are downsampled and stacked to obtain high-level feature maps. Subsequently, the feature maps are sent to the context information enhancement module to extract multi-scale feature representations and upsampled to obtain multi-scale joint feature maps. Finally, the low-level joint feature maps are stacked and sent to the decoder.

[0088] The input features are branched in multiple ways, and the features obtained have receptive fields of different sizes. Here, the void rates are set to 2, 3, 5, and 7, respectively, and the corresponding receptive fields are 5, 7, 9, 13, and the global receptive field brought by global pooling. The use of receptive fields of multiple scales in deep neural networks allows the model to observe the input data from different angles and capture contextual information of different scales. This multi-scale feature extraction helps to fully explore relevant information in high-level semantics, such as the edges, textures, and overall structures of objects. By utilizing multiple receptive fields, the model can obtain features of different granularities to better understand the input data. When processing multimodal data, the use of receptive fields of multiple scales helps to obtain more comprehensive information and enhances the model's grasp of the correlation information between different data modalities. Finally, the features from each scale are spliced ​​together, which helps to form a richer and more comprehensive feature expression, improves the utilization efficiency of multimodal high-level joint features and model performance, and this method can more comprehensively describe the data and provide important information support for more accurate pixel-level category determination.

[0089] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0090] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A DCNN-based land use semantic segmentation method for multi-source remote sensing images, characterized by: Here are the steps: S100, performing network input and feature extraction, and preprocessing the image; S200, using several independent convolution streams to process optical images and SAR images, each convolution stream focuses on the image type it processes; S300, design a multimodal attention module and use 1×1Conv to transform the features of multispectral images into three identical feature maps; According to step S300, 1×1Conv is used to transform the features of the multispectral image into three identical feature maps V MSI , Q MSI and K MSI , 1×1Conv is used to convert feature maps; multispectral images contain several spectral bands, each of which represents a different spectral information. By performing 1×1Conv operations on the features in the multispectral images, they are converted into three identical feature maps; During the conversion process, 1×1Conv is used to transform the number of channels in the original multispectral image into three identical feature maps. In fact, each feature is recombined in the feature space through linear transformation and the same number of feature maps are generated. MSI and K MSI The transpose of is multiplied, the formula is as follows: Among them, G is the pixel-level product result, which represents the relationship or interaction information between the corresponding features at different positions in the input image; S400, using a GELU activation function to obtain a self-attention map of a single data source; According to step S400, the self-attention map of a single data source is obtained using the GELU activation function, and the feature matrix is ​​captured by matrix multiplication. and K MSI Internal correlation between features. In neural networks, internal correlation is achieved through the attention mechanism. The matrix multiplication calculation of the attention mechanism assigns a weight to each feature, which indicates the degree of correlation between the current feature and other features. The matrix multiplication allows the model to learn the long-range dependency between features. At S500, the low-level optical feature maps and SAR feature maps are downsampled and stacked to obtain high-level feature maps; S600, the feature map is sent to the context information enhancement module to extract multi-scale feature representation, and up-sampled to obtain a multi-scale joint feature map; S700, stack the low-level joint feature maps and send them to the decoder.

2. The method for land use semantic segmentation of multi-source remote sensing images based on DCNN according to claim 1, characterized in that: According to step S100, the input content of the network includes independent multispectral images, hyperspectral images, original images of SAR images and label maps of SAR images, and the images and labels correspond to each other pixel by pixel; Aiming at the differences in imaging characteristics of optical and SAR images, a multi-branch structure is designed. In the initial stage of feature extraction, the image is preprocessed to eliminate noise, correct radiometric data, and adjust image geometric transformation. Then, the optical and SAR image blocks are input into independent convolutional streams with the same structure for feature extraction without sharing parameters.

3. The method for land use semantic segmentation of multi-source remote sensing images based on DCNN according to claim 1, characterized in that: According to step S200, in the convolution layer, the convolution kernel is used to perform convolution operation with the image to extract the edge, texture, color and shape information of the image; a number of independent convolution flows are used to process the optical image and the SAR image, allowing the network to extract the most representative features of the optical and SAR images respectively; the optical and SAR images are input into the convolution flows respectively, and the high-level features of any depth unit L of the optical image are expressed by recursive operation as follows: in, is the high-level feature of multispectral images, i is the unit of each convolutional layer, l is a shallow unit, is a linear projection, is a feature map learned from multispectral image patches; The high-level features of hyperspectral images are expressed as: in, is the high-level feature of the hyperspectral image, i is each convolutional layer unit, l is a shallow unit, is a linear projection, is a feature map learned from a hyperspectral image patch; The high-level features of SAR images are expressed as: in, is the high-level feature of SAR image, i is each convolutional layer unit, l is a shallow unit, is a linear projection, is a feature map learned from SAR image patches.

4. The method for land use semantic segmentation of multi-source remote sensing images based on DCNN according to claim 1, characterized in that: Since the information contained in MSI, HSI and SAR is independent and complementary, the information contained in MSI, HSI and SAR images is used to obtain the attention map of high-level feature cross-fusion using Hadamard product, and the attention map of high-level feature cross-fusion is obtained by element-by-element multiplication; the attention map reflects the correlation and co-occurrence pattern between different source data; the low-level matrix and the high-level matrix are weighted, and the weight W is determined according to the needs and experimental design a , W b and W c , calculate the weighted feature matrix: A weight =A·W a B weight =B·W b C weight =C·W c Among them, the matrices are A, B and C respectively.

5. The method for land use semantic segmentation of multi-source remote sensing images based on DCNN according to claim 4, characterized in that: The weighted feature matrix is ​​cross-fused to generate a joint attention map. The cross-fusion is completed by element-by-element multiplication. The formula is as follows: D=A weight ·B weight ·C weight D represents the final joint attention map matrix. During the cross-fusion process, the weighted operation adjusts each feature matrix and assigns different weights according to the importance or relevance of each feature matrix. Finally, the weighted feature maps are cross-fused to obtain the joint attention maps of MSI, HSI and SAR.

6. The method for land use semantic segmentation of multi-source remote sensing images based on DCNN according to claim 1, characterized in that: According to step S600, the input features are branched in multiple ways, and the obtained features have receptive fields of different sizes. By setting the void rate, the corresponding receptive field and the global receptive field brought by global pooling are obtained; the use of receptive fields of several scales in the deep neural network allows the model to observe the input data from different angles and capture contextual information of different scales; Multi-scale feature extraction can mine relevant information in high-level semantics, including the edges, textures, and overall structures of objects; by utilizing several receptive fields, the model obtains features of different granularities to understand the input data.

Images