Remote sensing image change detection method and related equipment

Abstract

The invention relates to the technical field of network detection, and discloses a remote sensing image change detection method and related equipment, and the method comprises the steps: obtaining a remote sensing image; establishing a remote sensing image change detection model; and inputting the obtained remote sensing image into a remote sensing image change detection model to carry out remote sensing image change detection based on the space-time difference capture network. The change area is highlighted through the spatial difference capturing module, so that the network pays more attention to the change sample. And meanwhile, a tense interaction attention mechanism is introduced, so that the interaction between the double-tense features is more deeply mined, and the extraction of the features with higher discrimination is facilitated. According to the invention, on the premise of low cost, simultaneous attention to local details and long-distance context is realized through a specific marking and window limiting strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of network detection, and in particular to a remote sensing image change detection method and related equipment. Background Art

[0002] Most existing change detection networks first use a two-stream feature extraction network to extract local features from bi-temporal remote sensing images. Then, feature fusion and enhancement of the bi-temporal image pairs are performed. Finally, the obtained multi-scale features are gradually fused from deep to shallow layers to obtain classification features, and then the classification features are input into the classifier to obtain the final change segmentation map. However, on the one hand, the class imbalance problem in change detection is not properly solved; on the other hand, most existing methods directly fuse bi-temporal features, ignoring the mining of potential information in the interaction of bi-temporal features.

[0003] In the change detection task, the changing region of interest in the bi-temporal image pair often only occupies a small part of the image, and the rest are unchanged regions. This leads to a very serious class imbalance problem in the change detection task, which greatly affects the performance of the model. Secondly, the feature interaction between bi-temporal remote sensing images contains potential change information, which can help to mine spatiotemporal clues and strengthen the feature representation of the changing region. However, this point has not received due attention, which reduces the network's ability to understand spatiotemporal correlation and suppress irrelevant interference.

[0004] Most of the existing technologies that have improved the class imbalance problem focus on the loss function, but this ignores the network's own attention to changing samples. As for the problem of bi-temporal feature interaction, some methods try to interact by exchanging the channel information of bi-temporal features, but such interaction is not thorough and cannot fully mine the change information. Summary of the invention

[0005] In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to provide a remote sensing image change detection method and related equipment to solve the technical problem that the prior art cannot solve the class imbalance and causes the network to tend to classify difficult samples into unchanged classes.

[0006] The present invention is achieved through the following technical solutions: In a first aspect, the present invention provides a remote sensing image change detection method, comprising: Acquisition of remote sensing images; Establish a remote sensing image change detection model; The acquired remote sensing images are input into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

[0007] Preferably, in the step of establishing a remote sensing image change detection model, the remote sensing image change detection model includes a feature extraction backbone, intra-scale feature enhancement, progressive cross-scale fusion, and a classifier; The feature extraction backbone model is ResNet18; the intra-scale feature enhancement model includes spatial difference extraction SDE and window-based feature interaction WFI; the progressive cross-scale fusion includes multiple groups of bilinear interpolation and 1×1 convolutional layers; the classifier includes two groups of 3×3 convolutional layers.

[0008] Preferably, the acquired remote sensing image is input into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network. The specific process is as follows: The remote sensing image is processed through the feature extraction backbone to obtain a local feature map; The temporal and spatial differences of the local feature map are captured to obtain enhanced features; The enhanced features are fused to obtain the final features; Classification is performed based on the final features to generate a change map and complete remote sensing image change detection.

[0009] Furthermore, in the step of capturing the spatiotemporal differences of the local feature map to obtain enhanced features, the specific process is as follows: Perform spatial difference extraction (SDE) on the local feature maps in sequence to obtain enhanced features; The local feature map and the enhanced feature are obtained through window-based feature interaction (WFI) to obtain the enhanced feature.

[0010] Furthermore, in the step of performing spatial difference extraction SDE output on the local feature map in sequence to obtain enhanced features, the specific process is as follows: The local feature map is passed through the spatial difference attention mechanism SDA to display the change clues hidden in the local features. The spatial difference attention mechanism SDA includes pixel-by-pixel subtraction operation, 3×3 depth-separable convolution layer and 1×1 convolution layer with sigmoid activation function. The expression of the spatial difference attention mechanism SDA is as follows:

[0011] in, and Represents a local feature map; Represents a difference feature map; Represents the spatial difference attention map SDA map; , , , , They represent absolute functions, Depthwise Separable Convolution, Convolutional layer and sigmoid activation function; Based on the spatial difference attention mechanism SDA, the spatial difference extraction SDE is performed on the change clues displayed in the local features to obtain enhanced features. Among them, the spatial difference extraction SDE includes an SDA branch and an information mining branch. The difference feature map is generated by the SDA branch; the information mining branch uses two 3×3 convolutional layers to combine the generated difference feature map with the change clues hidden in the local features to obtain enhanced features; among them, the expression of the spatial difference extraction SDE is as follows:

[0012] in, It represents the preliminary fusion characteristics of the two temporal states; and Represents channel-by-channel connection and convolution; To indicate enhanced characteristics.

[0013] Furthermore, the local feature map is combined with the enhanced feature through the window-based feature interaction WFI to obtain the enhanced feature. The specific process is as follows: The local feature map and enhanced features are reshaped into sequences through LayerNorm and Linear layers and mapped to obtain Query ( )、Key( ) and Value ( ) token, and Query ( )、Key( ) and Value ( ) token is input to the window-based temporal interaction attention layer W-TIA and divided into feature blocks and , the window size is a×a; based on the window being divided into n subspaces along the channel dimension, the expression is as follows:

[0014] in, Indicates the number of windows, represents the number of subspaces, , and The size is ; Represents the fusion feature block; Feature Block Based and Get the interactive and fusion features of information in each fixed window , the expression is as follows:

[0015] in, represents the scale factor, Represents learnable parameters containing relative position cues; The interaction and fusion features of the obtained information are obtained according to the location information Combined into the first intermediate feature , this feature is rich in global relationship and time information, and at the same time, it strengthens the feature Add to first intermediate feature In the process of enriching , generate the second intermediate feature ;Will Apply the second linear layer LayerNorm layer and the multi-layer perceptron layer MLP layer, combine residual connection and reshape operation to generate enhanced features .

[0016] In a second aspect, the present invention further provides a remote sensing image change detection system, comprising: An image acquisition module, used for acquiring remote sensing images; Model building module, used to build a remote sensing image change detection model; The image change detection module is used to input the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

[0017] Preferably, the image change detection module includes a feature extraction backbone module, an intra-scale feature enhancement module, a progressive cross-scale fusion module, and a classification module; The feature extraction backbone module is used to process the remote sensing image through the feature extraction backbone to obtain a local feature map; The intra-scale feature enhancement module is used to capture the temporal and spatial differences of the local feature map to obtain enhanced features; The progressive cross-scale fusion module is used to fuse the obtained enhanced features to obtain the final features; The classification module is used to generate change maps based on the final features and complete remote sensing image change detection.

[0018] In a third aspect, the present invention further provides a mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the remote sensing image change detection method as described above when executing the computer program.

[0019] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the remote sensing image change detection method as described above are implemented.

[0020] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention provides a remote sensing image change detection method. By establishing a remote sensing image change detection model and using a spatiotemporal difference capture network to perform remote sensing image change detection, a large amount of remote sensing image data can be efficiently processed and analyzed. The spatiotemporal difference capture network can accurately capture the temporal and spatial change characteristics in the image, thereby improving the accuracy of change detection. The present invention highlights the change area through the spatial difference capture module, so that the network itself pays more attention to the change samples. At the same time, the present invention introduces a temporal interactive attention mechanism to further explore the interaction between dual-temporal features, which is helpful to extract more discriminative features. The present invention achieves simultaneous attention to local details and long-distance contexts through specific marking and window restriction strategies at a low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of a remote sensing image change detection method in an embodiment of the present invention; Figure 2 A flow chart of remote sensing image change detection based on a spatiotemporal difference capture network in an embodiment of the present invention; Figure 3 This is a SDCNet framework diagram in an embodiment of the present invention; Figure 4 This is a schematic diagram of the spatial difference attention mechanism SDA in an embodiment of the present invention; Figure 5 This is a schematic diagram of the SDE structure for spatial difference extraction in an embodiment of the present invention; Figure 6 Schematic diagram of the window-based feature interaction WFI structure in an embodiment of the present invention; Figure 7 A schematic diagram of the W-TIA structure in an embodiment of the present invention; Figure 8 Schematic diagram of the remote sensing image change detection system in an embodiment of the present invention; Fig. 9 is an internal schematic diagram of an image change detection module in an embodiment of the present invention; In the figure: 1. Image acquisition module; 2. Model building module; 3. Image change detection module; 31. Feature extraction backbone module; 32. Intra-scale feature enhancement module; 33. Progressive cross-scale fusion module; 34. Classification module. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] The purpose of the present invention is to provide a remote sensing image change detection method and related equipment to solve the technical problem that the prior art cannot solve the class imbalance and causes the network to tend to classify difficult samples into unchanged classes.

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings: See also Figure 1 In one embodiment of the present invention, a remote sensing image change detection method is provided, comprising the following steps: Step 1, acquiring remote sensing images; Step 2, establishing a remote sensing image change detection model; Specifically, according to Figure 3 As shown, the remote sensing image change detection model includes a feature extraction backbone, intra-scale feature enhancement, progressive cross-scale fusion, and a classifier; The feature extraction backbone model is ResNet18; the intra-scale feature enhancement model includes spatial difference extraction SDE and window-based feature interaction WFI; the progressive cross-scale fusion includes multiple groups of bilinear interpolation and 1×1 convolutional layers; the classifier includes two groups of 3×3 convolutional layers.

[0025] Step 3: Input the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

[0026] Specifically, according to Figure 2 As shown, the specific process is as follows: S1, the remote sensing image is processed through the feature extraction backbone to obtain a local feature map; S2, captures the temporal and spatial differences of local feature maps to obtain enhanced features; Specifically, the specific process is as follows: S21, perform spatial difference extraction (SDE) on the local feature maps in sequence to obtain enhanced features; The specific process is as follows: The local feature map is passed through the spatial difference attention mechanism SDA to show the change clues hidden in the local features, where the spatial difference attention mechanism SDA includes pixel-by-pixel subtraction operation, 3×3 depth-wise separable convolution layer and 1×1 convolution layer with sigmoid activation function.

[0027] according to Figure 4 As shown, when a pair of local feature maps are input When the spatial difference attention mechanism SDA first applies a pixel-by-pixel subtraction operation to obtain the difference feature map This step aims to initially reveal the changes between dual-phase remote sensing images. Then, a progressive refinement strategy is used to further explore the clues of changes. First, we introduce Depthwise separable convolution Map to .choose The purpose of depthwise separable convolution is to effectively control the growth of the number of parameters while maintaining a large receptive field. The convolutional layer will Convert to SDA map Here, the spatial difference attention map SDA map can be regarded as a mask, in which the value ranges from 0 to 1, indicating the degree of attention the network needs to pay to these areas. The expression of the spatial difference attention mechanism SDA is as follows:

[0028] in, and Represents a local feature map; Represents a difference feature map; Represents the spatial difference attention map SDA map; , , , , They represent absolute functions, Depthwise Separable Convolution, Convolutional layer and sigmoid activation function; In this embodiment, on the one hand, the spatial difference attention mechanism SDA ensures that SDCNet can locate the change area more accurately because the false / real change area can be weakened / highlighted. On the other hand, since the spatial difference attention mechanism SDA assigns higher weights to the real change area, more attention can be paid to the limited number of positive samples when training SDCNet. In this way, the proposed method can alleviate the negative impact of the imbalance problem.

[0029] Based on the spatial difference attention mechanism SDA, the spatial difference extraction SDE of the change clues displayed in the local features is performed to obtain enhanced features, such as Figure 5 As shown, the spatial difference extraction SDE includes an SDA branch and an information mining branch. The SDA branch generates a difference feature map. The information mining branch uses two 3×3 convolutional layers to combine the generated difference feature map with the change clues hidden in the local features to obtain enhanced features.

[0030] In this embodiment, the two feature maps are connected in the channel dimension, and the first Convolution Generation , which contains detailed clues in the dual-time remote sensing imagery. Then, and Connect together to ensure that basic change information can be integrated. After convolution, we can generate , which contains the detail change information hidden in the two RS images. Next, the SDA map will with Multiply to get the final output This step can not only suppress the The irrelevant clues in the image can further strengthen the representation of the changed area. Possessing rich knowledge, it can help the model generate perfect variation graphs.

[0031] Among them, the expression of spatial difference extraction SDE is as follows:

[0032] in, It represents the preliminary fusion characteristics of the two temporal states; and Represents channel-by-channel connection and convolution; To indicate enhanced characteristics.

[0033] In this embodiment, spatial difference extraction can effectively guide SDCNet to focus on the changed areas and reduce the negative impact of the imbalance problem.

[0034] S22, the local feature map and the enhanced feature are obtained through the window-based feature interaction WFI to obtain the enhanced feature.

[0035] The specific process is as follows: The local feature map and enhanced features are reshaped into sequences through LayerNorm and Linear layers and mapped to obtain Query ( )、Key( ) and Value ( ) token, and Query ( )、Key( ) and Value ( ) token is input to the window-based temporal interaction attention layer W-TIA and divided into feature blocks and , the window size is a×a; based on the window, it is divided into n subspaces along the channel dimension.

[0036] In this example, the self-attention mechanism Transformer is introduced. However, when the Transformer is directly applied to When the self-attention mechanism Transformer is used, the detailed information will be ignored or blurred. Therefore, the window version Transformer is used. At the same time, considering the importance of analyzing the two time information for the RSCD task, in addition to In addition, local feature maps should also be considered and .

[0037] In this embodiment, the window-based feature interaction block WFI has a structure as follows: Figure 6 It consists of a LayerNorm with a linear layer, a Window-based Temporal Interaction Attention (W-TIA) layer, a LayerNorm layer, and a Multilayer Perceptron (MLP) layer (composed of two linear maps). They are connected in a residual way.

[0038] When the general When input into the WFI module, they are reshaped into sequences by the LayerNorm and Linear layers and mapped to the Query ( )、Key( ) and Value ( ) token. Then, they are sent to the W-TIA layer, which is the core of WFI, as shown in the diagram Figure 7 It is essentially a simplified version of the Swim Transformer. It uses a fixed gridding scheme instead of a shifting window to and Divided into various feature blocks, where the window size is . Instead of exchanging messages by moving individual windows, we only need to focus on the pixels that have changed within the window. In addition, this approach can reduce the parameter size of the entire model. At the same time, they will be divided into n subspaces along the channel dimension to help WFI capture contextual messages in terms of channels, as shown below:

[0039] in, Indicates the number of windows, represents the number of subspaces, , and The size is ; Represents the fusion feature block; Feature Block Based and Get the interactive and fusion features of information in each fixed window , the expression is as follows:

[0040] in, represents the scale factor, Represents learnable parameters containing relative position cues; In this embodiment, on the one hand, and The operation of realizes the interaction of time knowledge and helps WFI discover hidden change information. ,The change regions determined by SDE can also be fused, which helps WFI to identify the change regions more accurately based on the temporal cue analysis.

[0041] The interaction and fusion features of the obtained information are obtained according to the location information Combined into the first intermediate feature , the location information is rich in global relationship and time information, and at the same time, the enhanced features Add to first intermediate feature In the process of enriching , generate the second intermediate feature ;Will Apply the second linear layer LayerNorm layer and the multi-layer perceptron layer MLP layer, combine residual connection and reshape operation to generate enhanced features .

[0042] In this embodiment, the window-based feature interaction can focus on mining the interaction between bi-temporal features and deeply explore spatiotemporal clues to obtain a more robust feature representation.

[0043] S3, fusing the obtained enhanced features to obtain the final features; S4, classify and generate change maps based on the final features to complete remote sensing image change detection.

[0044] This embodiment first proposes a spatial difference attention mechanism, and then further proposes a spatial difference capture module, which can highlight the change area in the bi-temporal image pair and guide the network to pay more attention to the change target, thereby improving the accuracy of change detection. The window-based feature interaction module designed by the present invention deeply explores the interaction between bi-temporal features through temporal interaction attention, enhances the discriminability of spatiotemporal clues, and obtains richer feature representation.

[0045] In summary, the remote sensing image change detection method provided in this embodiment, compared with most methods that alleviate the class imbalance problem by weighting the loss function, the present invention highlights the change area through the spatial difference capture module, so that the network itself pays more attention to the positive samples of the change during the training process. Secondly, compared with other methods that simply exchange dual-temporal features, the present invention introduces a temporal interactive attention mechanism to further explore the interaction between dual-temporal features, which helps to extract more discriminative features. Finally, compared with other common Transformer and CNN combination methods in change detection tasks, the present invention achieves simultaneous attention to long-distance context and local details through specific labeling (implemented by SDE) and intra-window spatiotemporal exploration (implemented by WFI) strategies at a low cost.

[0046] Example 2 according to Figure 8 As shown, this embodiment provides a remote sensing image change detection system, including: Image acquisition module 1, used for acquiring remote sensing images; Model building module 2, used to build a remote sensing image change detection model; The image change detection module 3 is used to input the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

[0047] Specifically, according to Fig. 9 As shown, the image change detection module includes a feature extraction backbone module 31, an intra-scale feature enhancement module 32, a progressive cross-scale fusion module 33 and a classification module 34; The feature extraction backbone module 31 is used to process the remote sensing image through the feature extraction backbone to obtain a local feature map; The intra-scale feature enhancement module 32 is used to capture the temporal and spatial differences of the local feature map to obtain enhanced features; A progressive cross-scale fusion module 33 is used to fuse the obtained enhanced features to obtain the final features; The classification module 34 is used to perform classification based on the final features to generate a change map and complete remote sensing image change detection.

[0048] In the actual application process of this embodiment, the specific operations are as follows: When a pair of remote sensing images When input to SDCNet, the feature extraction backbone module 31 is first used to learn local information from it. Here, and Respectively represent the height, weight and channel of the remote sensing image. Thanks to the special structure of the feature extraction backbone ResNet18, four pairs of local feature maps corresponding to different scales can be obtained, denoted as ,in represents the level of feature maps. The size of these feature maps is ,in , For convenience, The local features are then fed into the intra-scale feature enhancement module 32, which aims to explore temporal cues and improve features at different single scales. After this module, we can obtain enhanced features with high attention to the changing area and rich representation. Next, we apply the progressive cross-scale fusion module33 to fuse them into , so that the relationship between different scales can be considered, and then the hidden change areas in the RS image can be detected from coarse to fine. , we use the classification module 34 to generate the change map .

[0049] Example 3 The present invention also provides a mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, such as a remote sensing image change detection management program.

[0050] When the processor executes the computer program, the steps of the remote sensing image change detection method are implemented, for example: Acquisition of remote sensing images; Establish a remote sensing image change detection model; The acquired remote sensing images are input into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

[0051] Among them, the acquired remote sensing image is input into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network. The specific process is as follows: The remote sensing image is processed through the feature extraction backbone to obtain a local feature map; The temporal and spatial differences of the local feature map are captured to obtain enhanced features; The enhanced features are fused to obtain the final features; Classification is performed based on the final features to generate a change map and complete remote sensing image change detection.

[0052] Alternatively, when the processor executes the computer program, the functions of each module in the above system are realized, for example: Image acquisition module 1, used for acquiring remote sensing images; Model building module 2, used to build a remote sensing image change detection model; The image change detection module 3 is used to input the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

[0053] The image change detection module includes a feature extraction backbone module 31, an intra-scale feature enhancement module 32, a progressive cross-scale fusion module 33, and a classification module 34; The feature extraction backbone module 31 is used to process the remote sensing image through the feature extraction backbone to obtain a local feature map; The intra-scale feature enhancement module 32 is used to capture the temporal and spatial differences of the local feature map to obtain enhanced features; A progressive cross-scale fusion module 33 is used to fuse the obtained enhanced features to obtain the final features; The classification module 34 is used to perform classification based on the final features to generate a change map and complete remote sensing image change detection.

[0054] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the mobile terminal.

[0055] For example, the computer program may be divided into an image acquisition module 1, a model building module 2 and an image change detection module 3; wherein the image change detection module 33 includes a feature extraction backbone module 31, an intra-scale feature enhancement module 32, a progressive cross-scale fusion module 33 and a classification module 34; The specific functions of each module are as follows: Image acquisition module 1, used for acquiring remote sensing images; Model building module 2, used to build a remote sensing image change detection model; The image change detection module 3 is used to input the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network. The feature extraction backbone module 31 is used to process the remote sensing image through the feature extraction backbone to obtain a local feature map; The intra-scale feature enhancement module 32 is used to capture the temporal and spatial differences of the local feature map to obtain enhanced features; A progressive cross-scale fusion module 33 is used to fuse the obtained enhanced features to obtain the final features; The classification module 34 is used to perform classification based on the final features to generate a change map and complete remote sensing image change detection.

[0056] The mobile terminal may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The mobile terminal may include, but is not limited to, a processor and a memory.

[0057] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the mobile terminal, and uses various interfaces and lines to connect various parts of the entire mobile terminal.

[0058] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the mobile terminal by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0059] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0060] Example 4 The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the remote sensing image change detection method are implemented.

[0061] If the module / unit integrated in the mobile terminal is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0062] Based on this understanding, the present invention implements all or part of the processes in the above method, and can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned aggregate reinforcement learning resource scheduling method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0063] The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0064] It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A remote sensing image change detection method, characterized in that: include: Acquisition of remote sensing images; Establish a remote sensing image change detection model; The acquired remote sensing images are input into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

2. A remote sensing image change detection method according to claim 1, characterized in that: In the step of establishing a remote sensing image change detection model, the remote sensing image change detection model includes a feature extraction backbone, intra-scale feature enhancement, progressive cross-scale fusion and a classifier; The feature extraction backbone model is ResNet18; the intra-scale feature enhancement model includes spatial difference extraction SDE and window-based feature interaction WFI; the progressive cross-scale fusion includes multiple groups of bilinear interpolation and 1×1 convolutional layers; the classifier includes two groups of 3×3 convolutional layers.

3. The remote sensing image change detection method according to claim 1, characterized in that: In the step of inputting the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network, the specific process is as follows: The remote sensing image is processed through the feature extraction backbone to obtain a local feature map; The temporal and spatial differences of the local feature map are captured to obtain enhanced features; The enhanced features are fused to obtain the final features; Classification is performed based on the final features to generate a change map and complete remote sensing image change detection.

4. A remote sensing image change detection method according to claim 3, characterized in that: In the step of capturing the spatiotemporal difference of the local feature map to obtain enhanced features, the specific process is as follows: Perform spatial difference extraction (SDE) on the local feature maps in sequence to obtain enhanced features; The local feature map and the enhanced feature are obtained through window-based feature interaction (WFI) to obtain the enhanced feature.

5. A remote sensing image change detection method according to claim 4, characterized in that: In the step of sequentially performing spatial difference extraction SDE on the local feature map to output enhanced features, the specific process is as follows: The local feature map is passed through the spatial difference attention mechanism SDA to display the change clues hidden in the local features. The spatial difference attention mechanism SDA includes pixel-by-pixel subtraction operation, 3×3 depth-separable convolution layer and 1×1 convolution layer with sigmoid activation function. The expression of the spatial difference attention mechanism SDA is as follows: in, and Represents a local feature map; Represents a difference feature map; Represents the spatial difference attention map SDA map; , , , , They represent absolute functions, Depthwise Separable Convolution, Convolutional layer and sigmoid activation function; Based on the spatial difference attention mechanism SDA, the spatial difference extraction SDE is performed on the change clues displayed in the local features to obtain enhanced features. Among them, the spatial difference extraction SDE includes an SDA branch and an information mining branch. The difference feature map is generated by the SDA branch; the information mining branch uses two 3×3 convolutional layers to combine the generated difference feature map with the change clues hidden in the local features to obtain enhanced features; among them, the expression of the spatial difference extraction SDE is as follows: in, It represents the preliminary fusion characteristics of the two temporal states; and Represents channel-by-channel connection and convolution; To indicate enhanced characteristics.

6. A remote sensing image change detection method according to claim 4, characterized in that: The specific process of combining the local feature map with the enhanced feature through the window-based feature interaction WFI to obtain the enhanced feature is as follows: The local feature map and enhanced features are reshaped into sequences through LayerNorm and Linear layers and mapped to obtain Query ( )、Key( ) and Value ( ) token, and Query ( )、Key( ) and Value ( ) token is input to the window-based temporal interaction attention layer W-TIA and divided into feature blocks and , the window size is a×a; based on the window being divided into n subspaces along the channel dimension, the expression is as follows: in, Indicates the number of windows, represents the number of subspaces, , and The size is ; Represents the fusion feature block; Feature Block Based and Get the interactive and fusion features of information in each fixed window , the expression is as follows: in, represents the scale factor, Represents learnable parameters containing relative position cues; The interaction and fusion features of the obtained information are obtained according to the location information Combined into the first intermediate feature , this feature is rich in global relationship and time information, and at the same time, it strengthens the feature Add to first intermediate feature In the process of enriching , generate the second intermediate feature ;Will Apply the second linear layer LayerNorm layer and the multi-layer perceptron layer MLP layer, combine residual connection and reshape operation to generate enhanced features .

7. A remote sensing image change detection system, characterized in that: include: An image acquisition module, used for acquiring remote sensing images; Model building module, used to build a remote sensing image change detection model; The image change detection module is used to input the acquired remote sensing image into the remote sensing image change detection model to perform remote sensing image change detection based on the spatiotemporal difference capture network.

8. A remote sensing image change detection system according to claim 1, characterized in that: The image change detection module includes a feature extraction backbone module, an intra-scale feature enhancement module, a progressive cross-scale fusion module and a classification module; The feature extraction backbone module is used to process the remote sensing image through the feature extraction backbone to obtain a local feature map; The intra-scale feature enhancement module is used to capture the temporal and spatial differences of the local feature map to obtain enhanced features; The progressive cross-scale fusion module is used to fuse the obtained enhanced features to obtain the final features; The classification module is used to generate change maps based on the final features and complete remote sensing image change detection.

9. A mobile terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the remote sensing image change detection method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the remote sensing image change detection method according to any one of claims 1 to 6 are implemented.

Images