Deep learning remote sensing change detection method based on mixed space-frequency expert

Abstract

The invention relates to a deep learning remote sensing change detection method based on a hybrid space-frequency expert, and the method comprises the steps: selecting a twin ConvNeXt network as a spatial domain expert, and extracting the coarse-fine spatial domain local difference features of a dual-time image; selecting twinborn discrete wavelet transform as a frequency domain expert, and extracting high-low frequency domain global difference characteristics of the dual-time image; aggregating local-global space-frequency features of corresponding scales, and constructing multi-scale mixed space-frequency monomer features; designing a random mixed space-frequency monomer fusion module, randomly selecting different space-frequency features for fusion and enhancing complementarity, and activating effective fusion according to contribution weight; randomly selecting mixed space-frequency monomers to form a group, performing probabilistic weighting, and adaptively selecting an optimal group; and constructing a lightweight decoder, and gradually restoring the enhanced multi-scale space-frequency features to the resolution of the original image. According to the invention, complementary optimization of cross-domain features is realized, invalid parameters are reduced, and remote sensing change detection precision and speed are improved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing change detection, and in particular to a remote sensing change detection method based on deep learning of hybrid space-frequency experts. Background Art

[0002] Change detection is a key application in remote sensing. Remote sensing technology can accurately capture dynamic temporal and spatial changes in ground features over a large scale, including changes in buildings, water bodies, cultivated land, and roads. This is crucial for post-disaster assessments and environmental monitoring.

[0003] With the development of deep learning, deep neural networks have gradually become the mainstream method for change detection. Compared with traditional methods such as image difference, deep neural networks can automatically learn high-dimensional abstract features of spatial and temporal dimensions in remote sensing images, which is conducive to improving the accuracy and robustness of change detection.

[0004] The current mainstream methods for change detection based on deep neural networks include the following two:

[0005] (1) Deep neural network based on spatial domain

[0006] These methods primarily rely on architectures such as convolutional neural networks, Transformers, and Mamba to achieve superior visual representation capabilities by extracting deep features in the spatial domain. Remote sensing objects exhibit significant scale effects, and current research typically employs multi-scale feature fusion to further improve the robustness and generalization of remote sensing object extraction. In particular, fine-scale features are beneficial for depicting spatial details, while coarse-scale features are adept at describing semantic information about objects. Therefore, by fusing multi-scale features, the richness of visual representations can be further enhanced, improving visual understanding of objects.

[0007] (2) Deep neural network based on frequency domain

[0008] This type of method uses frequency-domain transformations, such as discrete wavelet transforms, to convert remote sensing image features from the spatial domain to the frequency domain for spatiotemporal information expression. Current research often utilizes multi-spectral feature fusion to further enhance the representation of ground objects. High-frequency features are sensitive to edge information, while low-frequency features are good at capturing structural information. Therefore, fusing multi-frequency features can improve change detection accuracy.

[0009] In summary, current technical research still has shortcomings, such as: 1) Current research is limited to single feature domain representation, and the method of joint cross-domain representation is still imperfect, and cannot fully utilize the respective advantages to complement features; 2) In the cross-domain representation process, as the number of parameters increases, how to balance computational consumption and accuracy improvement is still worth considering; 3) In cross-domain feature combination, how to scientifically measure effective combination and reduce noise is still imperfect. Summary of the Invention

[0010] The present invention provides a deep learning remote sensing change detection method based on hybrid space-frequency experts. It uses expert networks in different feature domains as encoders to extract spatial local features and frequency global features, and then adopts a hybrid expert network for individual and group interactive decoding, thereby complementing the advantages of space-frequency features, enhancing the richness of representation, and reducing the computational consumption of redundant features, thereby improving the accuracy of remote sensing ground object change detection.

[0011] The present invention provides a deep learning remote sensing change detection method based on hybrid space-frequency experts, which adopts the following technical solutions:

[0012] The deep learning remote sensing change detection method based on hybrid space-frequency experts includes the following steps:

[0013] S1, selects the twin ConvNeXt network as the spatial domain expert to extract the multi-scale coarse-fine spatial domain local difference features of the dual-temporal remote sensing image;

[0014] S2, uses twin discrete wavelet transform as the frequency domain expert to extract the high-low frequency domain global difference features of the multi-spectral of the dual-temporal remote sensing image;

[0015] S3, constructs multi-scale mixed space-frequency monomer features by aggregating local-global space-frequency features of corresponding scales;

[0016] S4, designs a random mixed space-frequency monomer fusion module, randomly selects different space-frequency features for fusion, enhances complementarity and calculates contribution weights to activate effective fusion;

[0017] S5, design a random mixed space-frequency group decision module, randomly select different mixed space-frequency monomers to form a group and perform probabilistic weighting to adaptively select the optimal decision group;

[0018] S6, builds a lightweight decoder to gradually restore the enhanced multi-scale space-frequency features to the resolution of the original image and generate change detection results.

[0019] Preferably, said S1 comprises the following steps:

[0020] S11, extracting the deep spatial features of dual-temporal remote sensing images:

[0021] ConvNeXt is selected as the spatial domain expert and as the feature encoder. Based on the twin architecture, a weight sharing method is adopted to obtain multi-scale deep visual features in the spatial domain of the dual-phase remote sensing images T1 and T2. and Right now:

[0022]

[0023] S12, aligns the dual-temporal multi-scale coarse-fine spatial domain local difference features, namely:

[0024]

[0025] Among them, SAlign(·) selects the concatenate function for multi-dimensional spatiotemporal alignment, and S1, S2, S3, and S4 are four multi-scale spatial local difference features from coarse to fine.

[0026] Preferably, said S2 comprises the following steps:

[0027] S21, using the step-by-step decoupling method to select discrete wavelet transform to extract the multi-spectral frequency domain features of single-temporal remote sensing images, namely:

[0028] F 1 ,F 2 ,F 3 ,F 4 ,=MDWT(T) (3) The detailed iterative process of the step-by-step frequency decomposer of MDWT(·) is:

[0029]

[0030] Among them, DWT(·) is discrete wavelet transform, and Transformer(·) selects SwinTransformer block to enhance the deep visual representation ability of the global spectrum, F H Including high-high, high-low, low-high frequency components decomposed by DWT, F L including low-low frequency components;

[0031] S22, extracting global deep visual features in the multi-spectral frequency domain of dual-temporal remote sensing images using a twin architecture based on a step-by-step frequency decomposer and Right now:

[0032]

[0033] S23, align the frequency domain global difference features of the dual-phase high-low multi-spectrum, namely:

[0034]

[0035] Among them, FAlign(·) selects the concatenate function as the frequency domain feature alignment function, and F1, F2, F3, and F4 are the frequency domain global difference features of the multi-spectrum with four scales from high to low.

[0036] Preferably, the S3 is:

[0037] The corresponding multi-scale spatial domain local difference features and multi-spectral frequency domain global difference features are fused through the aggregation process and used as mixed space-frequency monomer features, namely:

[0038] [C1,C2,C3,C4]=Agg(S1,F1),Agg(S2,F2),Agg(S3,F3),Agg(S4,F4) (7)

[0039] Among them, Agg(·) selects the concatenate function as the aggregation function to aggregate space-frequency features, where C1, C2, C3, and C4 are the corresponding mixed space-frequency monomer features output.

[0040] Preferably, the S4 includes a random mixing space-frequency monomer fusion module, and the formula is:

[0041] G i =FSMoE(C i ,S,F,n) (8)

[0042] Among them, FSMoE(·) is a random mixed space-frequency monomer fusion module, where n represents the number of randomly extracted frequency and space monomer experts, C i is the i-th mixed space-frequency monomer, S = [S1, S2, S3, S4] is the mixed space feature monomer expert library, F = [F1, F2, F3, F4] is the mixed frequency monomer expert library, G i To output the result;

[0043] S4 includes the following steps:

[0044] S41, design random optimization strategy;

[0045] Using the sampling strategy without replacement, n spatial monomer experts are randomly selected from S and n frequency monomer experts are randomly selected from F as auxiliary monomer experts to enhance the mixed spatial-frequency monomer features.

[0046] S42, fusion of different spatial / frequency monomer experts to enhance the hybrid spatial / frequency monomer features;

[0047] The input mixed space-frequency monomer feature C iAs the monomer expert to be enhanced, for the randomly selected space / frequency monomer expert M m As an auxiliary monomer expert, where i and m are feature index values;

[0048] S421, align the feature dimensions of the auxiliary single expert and the single expert to be enhanced, namely:

[0049]

[0050] Among them, UConv(·) is the upsampling convolution function, DConv(·) is the downsampling convolution function, and Cat(·) is the concatenate function;

[0051] S422, design a hybrid spatial-frequency attention module, based on Mamba attention, to focus on and enhance the complementarity between different scale and spectral features, namely:

[0052]

[0053] Among them, Linear(·) is the linear projection layer, MAtten(·) is the multi-head attention, MLP(·) is the multi-layer perceptron layer, and Mamba(·) is:

[0054]

[0055] Among them, Linear(·) is the linear projection layer, Conv(·) is the convolutional layer, SiLU(·) is the SiLU activation function, and SSM(·) is the state space model, which uses vertical and horizontal modes to scan data;

[0056] S43, sequentially enhancing the mixed space-frequency monomer features based on step S42 for the randomly selected n space monomer experts and n frequency monomer experts;

[0057] S44, design a hybrid space-frequency monomer expert network, calculate the contribution weight of the fusion one by one through the gated network adaptive calculation, activate the effective features, and eliminate the invalid enhancement features, that is:

[0058]

[0059] Among them, D i is the output feature, softmax(·) is the softmax function, which is used to calculate the contribution probability. For the optimized mixed space-frequency monomer feature in step S43, GateN(·) is used as the route and the TopK algorithm is used to select the optimal feature as the output.

[0060] Preferably, the S5 includes a random mixed space-frequency group decision module, the formula of which is:

[0061] G i =GroMoE(D i ,G,n) (13)

[0062] Among them, Group i (·) represents the D i Enhanced, where D i represents the i-th feature in the enhanced mixed space-frequency monomer library D = {D1, D2, D3, D4}, G represents the constructed mixed space-frequency population, and n represents the number of constructed populations, including the following steps:

[0063] S5 includes the following steps:

[0064] S51, randomly select the optimized mixed space-frequency monomer characteristics to form a mixed space-frequency group, which is:

[0065]

[0066] in p represents the number of monomers randomly drawn from D, q represents the qth population constructed, i is the feature index, and the selected monomer is upsampled or downsampled with D i The feature dimensions are aligned, and the concatenate function is used to concatenate the features of all selected individuals to construct n groups;

[0067] S52, based on the constructed mixed space-frequency group, makes an optimal decision, namely:

[0068]

[0069] Among them, G i As the output, softmax(·) uses the softmax function, l represents the lth group, which is used to calculate the contribution probability of each group decision. GateN(·) is used as the routing and the TopK algorithm is used to select the optimal group decision result as the output;

[0070] S53, group decision making is used to perform group enhancement on each element of the optimized mixed space-frequency monomer D.

[0071] Preferably, the S6 comprises the following steps:

[0072] S61, preset lightweight decoder:

[0073]

[0074] Wherein, UpConv(·) represents the upsampling convolution layer, Cat(·) is the Cat operation, and Mamba(·) is the Mamba attention in step S422 used to further enhance feature expression;

[0075] S62, output results;

[0076] O=Softmax(Conv(H1)) (17)

[0077] Among them, Conv(·) represents the convolutional layer, and Softmax(·) is used to output the classification probability.

[0078] In summary, the present invention has the following beneficial technical effects:

[0079] 1. The present invention extracts spatial local features and frequency global features from different domains through an encoder, and adopts a hybrid expert network for individual and group interactive decoding, thereby complementing the advantages of spatial-frequency features, enhancing the richness of representation, reducing the computational consumption of redundant features, and improving the accuracy of remote sensing ground object change detection.

[0080] 2. This invention considers the representation capabilities of deep neural networks for remote sensing change detection from a multi-feature domain and multi-angle perspective, including: multi-scale-multi-spectral, global-local, edge-structure, and detail-semantic. It has the following advantages: 1) ensuring that network optimization focuses on improving specific tasks, reducing the mutual interference between the above-mentioned complex feature optimizations, thereby improving the modeling and expression capabilities of cross-domain features; 2) improving the richness of representations across feature domains, enhancing the ability to perceive spatial-level details and abstract features of remote sensing objects, and the ability to capture frequency-level edge and structural features of remote sensing objects;

[0081] 3. The present invention has the advantages of rationality, robustness, and generalization in the fusion of multi-scale spatial features and multi-spectral frequency features, which are reflected in: 1) Designing a corresponding hybrid expert network to calculate the contribution weight of the fusion between individual and group expert features can scientifically and reasonably evaluate the characterization ability of its fusion method; 2) By randomly selecting the fusion method of individual and group expert features, it helps to reduce the interference of individual deviation and noise, avoid network overfitting, and improve the generalization and robustness of the network. 3) Designing and selecting the method of fusion of individual and group experts, by fusing a variety of spatial-frequency monomer combinations, taking into account the possibility of multiple fusion methods, reducing the error and uncertainty of feature fusion through network optimization, and improving generalization ability.

[0082] 4. This invention considers the balance between computing power consumption and computational accuracy, which is conducive to achieving load balancing. By designing a hybrid expert network of individuals and groups, while fully considering the complementarity, richness, and diversity of the fusion of single-scale spectrum and multi-scale spectrum features, it further enhances the generalization and robustness of the model by activating beneficial features and discarding invalid features, while reducing a large amount of invalid calculations and saving network inference consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a schematic diagram of a flowchart in an embodiment of the present invention.

[0084] Figure 2 Schematic diagram of a random mixing space-frequency monomer fusion module in an embodiment of the present invention.

[0085] Figure 3 Schematic diagram of a hybrid space-frequency attention module according to an embodiment of the present invention.

[0086] Figure 4 Schematic diagram of the scanning method in the hybrid space-frequency attention module in an embodiment of the present invention.

[0087] Figure 5 This is the visual result of the method of the present invention and the comparison network on the WB-CD dataset in the embodiment of the present invention. DETAILED DESCRIPTION

[0088] The present invention will be further described in detail below with reference to the accompanying drawings.

[0089] The embodiment of the present invention discloses a remote sensing change detection method based on deep learning of hybrid space-frequency experts.

[0090] This example uses building change detection as an example and uses the open source remote sensing building change detection dataset WB-CD dataset. The dataset is sampled from two cities, Christchurch and New Zealand, and covers an urban area of 20.5 km. 2 , covering two time periods: 2012 and 2016. It includes a large number of independent buildings, increasing from 12,796 to 16,077. Its images will be cut into non-overlapping pairs of samples of 256×256 pixels in size, generating training, validation, and test datasets in a ratio of 1:1:8.

[0091] Reference Figure 1 , the deep learning remote sensing change detection method based on hybrid space-frequency experts includes the following steps:

[0092] S1, using the twin ConvNeXt network as the spatial domain expert to extract the multi-scale coarse-fine spatial domain local difference features of the dual-temporal remote sensing image:

[0093] Specifically, S1 includes the following steps:

[0094] S11, extracting the deep spatial features of dual-temporal remote sensing images:

[0095] ConvNeXt is selected as the spatial domain expert and as the feature encoder. Based on the twin architecture, a weight sharing method is adopted to obtain multi-scale deep visual features in the spatial domain of the dual-phase remote sensing images T1 and T2. and Right now:

[0096]

[0097] S12, aligns the dual-temporal multi-scale coarse-fine spatial domain local difference features, namely:

[0098]

[0099] Among them, SAlign(·) selects the concatenate function for multi-dimensional spatiotemporal alignment, and S1, S2, S3, and S4 are four multi-scale spatial local difference features from coarse to fine.

[0100] S2 uses the twin discrete wavelet transform as the frequency domain expert to extract the high-low frequency domain global difference features of the multi-spectral of the dual-temporal remote sensing image:

[0101] Specifically, S2 includes the following steps:

[0102] S21, using the step-by-step decoupling method to select discrete wavelet transform to extract the multi-spectral frequency domain features of single-temporal remote sensing images, namely:

[0103] F 1 ,F 2 ,F 3 ,F 4 ,=MDWT(T) (3) The detailed iterative process of the step-by-step frequency decomposer of MDWT(·) is:

[0104]

[0105] Among them, DWT(·) is discrete wavelet transform, and Transformer(·) selects SwinTransformer block to enhance the deep visual representation ability of the global spectrum, F H Including high-high, high-low, low-high frequency components decomposed by DWT, F L including low-low frequency components;

[0106] S22, extracting global deep visual features in the multi-spectral frequency domain of dual-temporal remote sensing images using a twin architecture based on a step-by-step frequency decomposer and Right now:

[0107]

[0108] S23, align the frequency domain global difference features of the dual-phase high-low multi-spectrum, namely:

[0109]

[0110] Among them, FAlign(·) selects the concatenate function as the frequency domain feature alignment function, and F1, F2, F3, and F4 are the frequency domain global difference features of the multi-spectrum with four scales from high to low.

[0111] Reference Figures 2 to 4 , S3, by aggregating the local-global space-frequency features of the corresponding scale, construct a multi-scale mixed space-frequency monomer feature, that is:

[0112] [C1,C2,C3,C4]=Agg(S1,F1),Agg(S2,F2),Agg(S3,F3),Agg(S4,F4) (7)

[0113] Among them, Agg(·) selects the concatenate function as the aggregation function to aggregate space-frequency features, where C1, C2, C3, and C4 are the corresponding mixed space-frequency monomer features output.

[0114] S4, designs a random mixed space-frequency monomer fusion module, randomly selects different space-frequency features for fusion, enhances complementarity and calculates contribution weights to activate effective fusion;

[0115] S4 includes a random mixing space-frequency monomer fusion module, the formula is:

[0116] G i =FSMoE(C i ,S,F,n) (8)

[0117] Among them, FSMoE(·) is a random mixed space-frequency monomer fusion module, where n represents the number of randomly extracted frequency and space monomer experts, C i is the i-th mixed space-frequency monomer, S = [S1, S2, S3, S4] is the mixed space feature monomer expert library, F = [F1, F2, F3, F4] is the mixed frequency monomer expert library, G i To output the result;

[0118] Specifically, S4 includes the following steps:

[0119] S41, design random optimization strategy;

[0120] Using the sampling strategy without replacement, n spatial monomer experts are randomly selected from S and n frequency monomer experts are randomly selected from F as auxiliary monomer experts to enhance the mixed spatial-frequency monomer features.

[0121] S42, fusion of different spatial / frequency monomer experts to enhance the hybrid spatial / frequency monomer features;

[0122] The input mixed space-frequency monomer feature C i As the monomer expert to be enhanced, for the randomly selected space / frequency monomer expert M m As an auxiliary monomer expert, where i and m are feature index values;

[0123] S421, align the feature dimensions of the auxiliary single expert and the single expert to be enhanced, namely:

[0124]

[0125] Among them, UConv(·) is the upsampling convolution function, DConv(·) is the downsampling convolution function, and Cat(·) is the concatenate function;

[0126] S422, design a hybrid spatial-frequency attention module, based on Mamba attention, to focus on and enhance the complementarity between different scale and spectral features, namely:

[0127]

[0128] Among them, Linear(·) is the linear projection layer, MAtten(·) is the multi-head attention, MLP(·) is the multi-layer perceptron layer, and Mamba(·) is:

[0129]

[0130] Among them, Linear(·) is the linear projection layer, Conv(·) is the convolutional layer, SiLU(·) is the SiLU activation function, and SSM(·) is the state space model, which uses vertical and horizontal modes to scan data;

[0131] S43, sequentially enhancing the mixed space-frequency monomer features based on step S42 for the randomly selected n space monomer experts and n frequency monomer experts;

[0132] S44, design a hybrid space-frequency monomer expert network, calculate the contribution weight of the fusion one by one through the gated network adaptive calculation, activate the effective features, and eliminate the invalid enhancement features, that is:

[0133]

[0134] Among them, Di is the output feature, softmax(·) is the softmax function, which is used to calculate the contribution probability. For the optimized mixed space-frequency monomer feature in step S43, GateN(·) is used as the route and the TopK algorithm is used to select the optimal feature as the output.

[0135] S5, design a random mixed space-frequency group decision module, randomly select different mixed space-frequency monomers to form a group and perform probabilistic weighting to adaptively select the optimal decision group;

[0136] S5 includes a random mixed space-frequency group decision module, namely:

[0137] G i =GroMoE(D i ,G,n) (13)

[0138] Among them, Group i (·) represents the D i Enhanced, where D i represents the i-th feature in the enhanced mixed space-frequency monomer library D = {D1, D2, D3, D4}, G represents the constructed mixed space-frequency population, and n represents the number of constructed populations. It includes the following steps:

[0139] Specifically, S5 includes the following steps:

[0140] S51, randomly select optimized mixed space-frequency monomer features to form a mixed space-frequency population, that is:

[0141]

[0142] in p represents the number of monomers randomly drawn from D, q represents the qth population constructed, i is the feature index, and the selected monomer is upsampled or downsampled with D i The feature dimensions are aligned, and the concatenate function is used to concatenate the features of all selected individuals to construct n groups;

[0143] S52, based on the constructed mixed space-frequency group, makes an optimal decision, namely:

[0144]

[0145] Among them, G iAs the output, softmax(·) uses the softmax function, l represents the lth group, which is used to calculate the contribution probability of each group decision. GateN(·) is used as the routing and the TopK algorithm is used to select the optimal group decision result as the output;

[0146] S53, group decision making is used to perform group enhancement on each element of the optimized mixed space-frequency monomer D.

[0147] S6, builds a lightweight decoder to gradually restore the enhanced multi-scale spatial-frequency features to the resolution of the original image and generate change detection results:

[0148] Specifically, S6 includes the following steps:

[0149] S61, preset lightweight decoder:

[0150]

[0151] Wherein, UpConv(·) represents the upsampling convolution layer, Cat(·) is the Cat operation, and Mamba(·) is the Mamba attention in step S422 used to further enhance feature expression;

[0152] S62, output results;

[0153] O=Softmax(Conv(H1)) (17)

[0154] Among them, Conv(·) represents the convolutional layer, and Softmax(·) is used to output the classification probability.

[0155] Reference Figure 5 As shown, this embodiment selects ChangeFormer and P2V-CD network as comparison methods to demonstrate the advantages of visual results in change detection of the present invention.

[0156] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep learning remote sensing change detection method based on hybrid space-frequency experts, characterized by: The following steps are involved: S1, selects the twin ConvNeXt network as the spatial domain expert to extract the multi-scale coarse-fine spatial domain local difference features of the dual-temporal remote sensing image; S2, uses twin discrete wavelet transform as the frequency domain expert to extract the high-low frequency domain global difference features of the multi-spectral of the dual-temporal remote sensing image; S3, constructs multi-scale mixed space-frequency monomer features by aggregating local-global space-frequency features of corresponding scales; S4, designs a random mixed space-frequency monomer fusion module, randomly selects different space-frequency features for fusion, enhances complementarity and calculates contribution weights to activate effective fusion; S5, design a random mixed space-frequency group decision module, randomly select different mixed space-frequency monomers to form a group and perform probabilistic weighting to adaptively select the optimal decision group; S6, builds a lightweight decoder to gradually restore the enhanced multi-scale space-frequency features to the resolution of the original image and generate change detection results.

2. The method for remote sensing change detection based on deep learning using hybrid spatial-frequency experts according to claim 1, characterized in that: Said S1 comprises the following steps: S11, extracting the deep spatial features of dual-temporal remote sensing images: ConvNeXt is selected as the spatial domain expert and as the feature encoder. Based on the twin architecture, a weight sharing method is adopted to obtain multi-scale deep visual features in the spatial domain of the dual-phase remote sensing images T1 and T2. and Right now: S12, aligns the dual-temporal multi-scale coarse-fine spatial domain local difference features, namely: Among them, SAlign(·) selects the concatenate function for multi-dimensional spatiotemporal alignment, and S1, S2, S3, and S4 are four multi-scale spatial local difference features from coarse to fine.

3. The method for remote sensing change detection based on deep learning using hybrid space-frequency experts according to claim 1, characterized in that: The S2 comprises the following steps: S21, using the step-by-step decoupling method to select discrete wavelet transform to extract the multi-spectral frequency domain features of single-phase remote sensing images, namely: F 1 ,F 2 ,F 3 ,F 4 ,=MDWT(T) (3) The detailed iterative process of the step-by-step frequency decomposer of MDWT(·) is: Among them, DWT(·) is discrete wavelet transform, and Transformer(·) selects SwinTransformer block to enhance the deep visual representation ability of the global spectrum, F H Including high-high, high-low, low-high frequency components decomposed by DWT, F L including low-low frequency components; S22, using a twin architecture based on a step-by-step frequency decomposer to extract global deep visual features in the multi-spectral frequency domain of dual-temporal remote sensing images and Right now: S23, align the frequency domain global difference features of the dual-phase high-low multi-spectrum, namely: Among them, FAlign(·) selects the concatenate function as the frequency domain feature alignment function, and F1, F2, F3, and F4 are the frequency domain global difference features of the multi-spectrum with four scales from high to low.

4. The method for remote sensing change detection based on deep learning using hybrid space-frequency experts according to claim 1, characterized in that: The S3 is: The corresponding multi-scale spatial domain local difference features and multi-spectral frequency domain global difference features are fused through the aggregation process and used as mixed space-frequency monomer features, namely: [C1,C2,C3,C4]=Agg(S1,F1),Agg(S2,F2),Agg(S3,F3),Agg(S4,F4) (7) Among them, Agg(·) selects the concatenate function as the aggregation function to aggregate space-frequency features, where C1, C2, C3, and C4 are the corresponding mixed space-frequency monomer features output.

5. The method for remote sensing change detection based on deep learning using hybrid space-frequency experts according to claim 1, characterized in that: The S4 includes a random mixed space-frequency monomer fusion module, which enhances the characterization capability of mixed space-frequency monomers one by one by fusing different spectrum frequency monomer experts and different scale space monomer experts, namely: G i =FSMoE(C i ,S,F,n) (8) Among them, FSMoE(·) is a random mixed space-frequency monomer fusion module, where n represents the number of randomly extracted frequency and space monomer experts, C i is the i-th mixed space-frequency monomer, S = [S1, S2, S3, S4] is the mixed space feature monomer expert library, F = [F1, F2, F3, F4] is the mixed frequency monomer expert library, G i To output the results, the following steps are included: S41, design random optimization strategy; Using the sampling strategy without replacement, n spatial monomer experts are randomly selected from S and n frequency monomer experts are randomly selected from F as auxiliary monomer experts to enhance the mixed spatial-frequency monomer features. S42, fusion of different spatial / frequency monomer experts to enhance the hybrid spatial / frequency monomer features; The input mixed space-frequency monomer feature C i As the monomer expert to be enhanced, for the randomly selected space / frequency monomer expert M m As an auxiliary monomer expert, where i and m are feature index values; S421, align the feature dimensions of the auxiliary single expert and the single expert to be enhanced, namely: Among them, UConv(·) is the upsampling convolution function, DConv(·) is the downsampling convolution function, and Cat(·) is the concatenate function; S422, design a hybrid spatial-frequency attention module, based on Mamba attention, to focus on and enhance the complementarity between different scale and spectral features, namely: C i ″=C i ′+MAtten(Linear(C i ′)) (10) E i =C i ″+MLP(Mamba(Linear(C i ″)) Among them, Linear(·) is the linear projection layer, MAtten(·) is the multi-head attention, MLP(·) is the multi-layer perceptron layer, and Mamba(·) is: O′ i =SSM(SiLU(Conv(Linear(O i )))) About i =SiLU(Conv(Linear(O i ))) (11) N i =Linear(Cat(O′ i ,O′ i )) Among them, Linear(·) is the linear projection layer, Conv(·) is the convolutional layer, SiLU(·) is the SiLU activation function, and SSM(·) is the state space model, which uses vertical and horizontal modes to scan data; S43, sequentially enhancing the mixed space-frequency monomer features based on step S42 for the randomly selected n space monomer experts and n frequency monomer experts; S44, design a hybrid space-frequency monomer expert network, calculate the contribution weight of the fusion one by one through the gated network adaptive calculation, activate the effective features, and eliminate the invalid enhancement features, that is: Among them, D i is the output feature, softmax(·) is the softmax function, which is used to calculate the contribution probability. For the optimized mixed space-frequency monomer feature in step S43, GateN(·) is used as the route and the TopK algorithm is used to select the optimal feature as the output.

6. The method for remote sensing change detection based on deep learning using hybrid space-frequency experts according to claim 1, characterized in that: The design random mixed space-frequency group decision module in S5 is: G i =GroMoE(D i ,G,n) (13) Among them, Group i (·) represents the D i Enhanced, where D i represents the i-th feature in the enhanced mixed space-frequency monomer library D = {D1, D2, D3, D4}, G represents the constructed mixed space-frequency population, and n represents the number of constructed populations, including the following steps: S51, randomly select optimized mixed space-frequency monomer features to form a mixed space-frequency population, that is: in p represents the number of monomers randomly drawn from D, q represents the qth population constructed, i is the feature index, and the selected monomer is upsampled or downsampled with D i The feature dimensions are aligned, and the concatenate function is used to concatenate the features of all selected individuals to construct n groups; S52, based on the constructed mixed space-frequency group, makes an optimal decision, namely: Among them, G i As the output, softmax(·) uses the softmax function, l represents the lth group, which is used to calculate the contribution probability of each group decision. GateN(·) is used as the routing and the TopK algorithm is used to select the optimal group decision result as the output; S53, group decision making is used to perform group enhancement on each element of the optimized mixed space-frequency monomer D.

7. The method for remote sensing change detection based on deep learning using hybrid space-frequency experts according to claim 1, characterized in that: The S6 comprises the following steps: S61, preset lightweight decoder: H3=Mamba(Cat(UpConv(G4)+G3)) H2=Mamba(Cat(UpConv(G3)+G2)) (16) H1=Mamba(Cat(UpConv(G1)+G2)) Wherein, UpConv(·) represents the upsampling convolution layer, Cat(·) is the Cat operation, and Mamba(·) is the Mamba attention in step S422 used to further enhance feature expression; S62, output results; O=Softmax(Conv(H1)) (17) Among them, Conv(·) represents the convolutional layer, and Softmax(·) is used to output the classification probability.

Images