Remote sensing image change detection method and related device

Abstract

The application discloses a remote sensing image change detection method and related device, the method comprises the following steps: acquiring first remote sensing image and second remote sensing image with the same area at different time, and respectively extracting features of the first remote sensing image and the second remote sensing image to obtain first feature map and second feature map; processing the first feature map by using the second feature map to obtain the processed first feature map, and processing the second feature map to obtain the processed second feature map; obtaining the difference area in the first remote sensing image and the second remote sensing image based on at least part of the first feature map, the second feature map, the processed first feature map and the processed second feature map. Through the above method, the application can improve the feature extraction efficiency, and further improve the accuracy of remote sensing image change detection.

Description

Technical Field

[0001] This application relates to the field of remote sensing image detection technology, and in particular to a method and related apparatus for detecting changes in remote sensing images. Background Technology

[0002] Remote sensing image change detection utilizes equipment such as remote sensing satellites, drones, and aircraft to acquire image data of the same area at multiple time points, and then employs certain technical methods to analyze and identify areas where changes in the land surface attributes have occurred. Existing remote sensing image change detection methods mainly rely on two twin feature extraction branches, which operate completely independently during the feature extraction process, resulting in low efficiency. Summary of the Invention

[0003] The main technical problem addressed by this application is to provide a method for detecting changes in remote sensing images, which can improve the efficiency of feature extraction and thus enhance the accuracy of change detection tasks.

[0004] To address the aforementioned technical problems, this application provides a method for detecting changes in remote sensing images, comprising: acquiring a first remote sensing image and a second remote sensing image with the same region at different times; extracting features from the first remote sensing image and the second remote sensing image to obtain a first feature map and a second feature map, respectively; processing the first feature map using the second feature map to obtain a processed first feature map; processing the second feature map to obtain a processed second feature map; and obtaining the difference regions in the first remote sensing image and the second remote sensing image based on at least a portion of the first feature map, the second feature map, the processed first feature map, and the processed second feature map.

[0005] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein the processor stores program instructions, and the processor is used to execute the program instructions to implement the remote sensing image change detection method described in the above embodiments.

[0006] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a storage device storing program instructions that can be executed by a processor, wherein the program instructions are used to implement the remote sensing image change detection method described in any of the above embodiments.

[0007] The beneficial effects of this application are as follows: Unlike the existing technology, this application proposes a remote sensing image change detection method, which optimizes the structure of the remote sensing image change detection network, increases the information exchange between the feature extraction branches in the remote sensing image change detection network, improves the efficiency of feature extraction, and thus improves the accuracy of remote sensing image change detection. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0009] Figure 1 This is a flowchart illustrating one implementation method of the remote sensing image change detection method of this application;

[0010] Figure 2 yes Figure 1 The middle step S102 corresponds to a topology diagram of one embodiment;

[0011] Figure 3 This is a flowchart illustrating one embodiment of obtaining the processed first feature map in step S102.

[0012] Figure 4 This is a flowchart illustrating one embodiment of obtaining the processed second feature map in step S102;

[0013] Figure 5 This is a schematic diagram of an implementation of the cross-processing network model proposed in this application;

[0014] Figure 6 This is a schematic diagram of the framework structure of one embodiment of the remote sensing image change detection device of this application;

[0015] Figure 7 This is a schematic diagram of the structure of one embodiment of the electronic device of this application;

[0016] Figure 8 This is a schematic diagram of one embodiment of the storage device of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] Please see Figure 1 , Figure 1 This is a flowchart illustrating one embodiment of the remote sensing image change detection method of this application. The method includes:

[0019] S101: Acquire a first remote sensing image and a second remote sensing image that have the same area at different times, and extract features from the first remote sensing image and the second remote sensing image respectively to obtain a first feature map and a second feature map.

[0020] Specifically, in this embodiment, the first remote sensing image and the second remote sensing image can be acquired from the same area using equipment such as remote sensing satellites, drones, and aircraft; and optionally, the acquisition time of the first remote sensing image can be earlier than the acquisition time of the second remote sensing image, and the time interval between the first remote sensing image and the second remote sensing image can be set according to actual conditions.

[0021] Furthermore, in implementing the above step S101, features can be extracted from the first remote sensing image and the second remote sensing image through a convolutional layer to obtain the corresponding first feature map and second feature map.

[0022] S102: Process the first feature map using the second feature map to obtain the processed first feature map, and process the second feature map to obtain the processed second feature map.

[0023] In one application scenario, when the process of obtaining the processed second feature map in step S102 involves processing the second feature map using the first feature map to obtain the processed second feature map, please refer to [the relevant documentation]. Figure 2 , Figure 2 for Figure 1 Step S102 corresponds to a topology diagram of one embodiment. For example... Figure 2 As shown, by simultaneously performing linear mapping processing on the first and second feature maps obtained in step S101, first, second, and third matrices of the first and second feature maps are obtained, respectively. Further, the first matrix of the second feature map is multiplied by the second matrix of the first feature map and normalized; the normalized result is multiplied by the third matrix of the first feature map, and the result is added to the first feature map to obtain the processed first feature map. Simultaneously, the first matrix of the first feature map is multiplied by the second matrix of the second feature map and normalized; the normalized result is multiplied by the third matrix of the second feature map, and the result is added to the second feature map to obtain the processed second feature map. In this application scenario, using the second feature map to process the first feature map to obtain the processed first feature map, and using the first feature map to process the second feature map to obtain the processed second feature map, increases the information exchange between the first and second feature maps, helping to identify the difference regions between them.

[0024] Specifically, the processes for obtaining the processed first feature map and the processed second feature map are described below. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 A flowchart illustrating one embodiment of obtaining the processed first feature map in step S102 is provided, the process including:

[0025] S201: Obtain the first weight matrix using the first matrix of the second feature map and the second matrix of the first feature map.

[0026] Specifically, the implementation process of step S201 includes: performing linear mapping processing on the first feature map and the second feature map respectively to obtain multiple feature matrices of the first feature map and the second feature map respectively, including: a first matrix, a second matrix, and a third matrix.

[0027] Furthermore, the first matrix of the second feature map and the second matrix of the first feature map are multiplied together and then normalized to obtain the first weight matrix. The normalization can be achieved by introducing a SoftMax algorithm. This process achieves the goal of finding features similar to the second feature map on the first feature map. Optionally, an additional neural network can be introduced to calculate the similarity between the first matrix of the second feature map and the second matrix of the first feature map, and then normalize the result to obtain the first weight matrix.

[0028] S202: Obtain the first weighted feature map using the first weight matrix and the third matrix of the first feature map.

[0029] Specifically, the implementation steps of step S202 include: weighting the third matrix of the first feature map with the first weight matrix obtained in step S201 (i.e., performing matrix multiplication) to generate a first weighted feature map. This step highlights the features in the first feature map that differ from the second feature map, and ignores similar features.

[0030] S203: Obtain the processed first feature map using the first weighted feature map and the first feature map.

[0031] Specifically, the implementation steps of step S203 include: adding the first weighted feature map obtained in step S202 to the first feature map to obtain the processed first feature map. This process further optimizes the first weighted feature map in step S202 and uses the result as the processed first feature map, which can effectively reduce the impact of errors.

[0032] Alternatively, the first weighted feature map obtained in step S202 can be used as the processed first feature map, that is, the process of adding the first weighted feature map and the first feature map is not performed.

[0033] Furthermore, the processed second feature map is obtained by processing the second feature map using the first feature map. Please refer to [link / reference]. Figure 4 , Figure 4 This is a flowchart illustrating one embodiment of obtaining the processed second feature map in step S102, the process including:

[0034] S301: Obtain the second weight matrix using the first matrix of the first feature map and the second matrix of the second feature map.

[0035] Specifically, the implementation process of step S301 includes: performing linear mapping processing on the first feature map and the second feature map respectively to obtain multiple feature matrices of the first feature map and the second feature map, including: a first matrix, a second matrix, and a third matrix. Further, multiplying the first matrix of the first feature map and the second matrix of the second feature map together and normalizing them, yields a second weight matrix.

[0036] S302: Obtain the second weighted feature map using the third matrix of the second weight matrix and the second feature map.

[0037] Specifically, the implementation process of step S302 includes: weighting the second weight matrix and the third matrix of the second feature map obtained in step S301 to generate a second weighted feature map.

[0038] S303: Obtain the processed second feature map using the second weighted feature map and the second feature map.

[0039] Specifically, the implementation process of step S303 includes: adding the second weighted feature map obtained in step S302 to the second feature map to obtain the processed second feature map.

[0040] Alternatively, the second weighted feature map obtained in step S302 can be used as the processed second feature map, that is, the process of adding the second weighted feature map and the second feature map is not performed.

[0041] The above steps S301 to S303, by processing the second feature map using the first feature map, can highlight the features in the second feature map that are different from the first feature map, and ignore similar features.

[0042] Optionally, in another application scenario, the step of processing the second feature map in step S102 to obtain the processed second feature map may also include: processing the second feature map using its own feature matrix to obtain the processed second feature map. Specifically, linear mapping is performed on the second feature map to obtain a first matrix, a second matrix, and a third matrix of the second feature map. The first matrix of the second feature map and the second matrix of the second feature map are multiplied and normalized to obtain a second weight matrix. The third matrix of the second feature map is weighted using the second weight matrix to generate a second weighted feature map. Further, the second weighted feature map is added to the second feature map. Optionally, the second weighted feature map can be used as the processed second feature map, or the feature map obtained by adding the second weighted feature map to the second feature map can be used as the processed second feature map. By processing the second feature map using its own information, feature points in the second feature map are identified and extracted to obtain the processed second feature map, which helps to combine the processed first feature map to obtain the difference region between the first remote sensing image and the second remote sensing image.

[0043] Optionally, in step S102, the first feature map and / or the second feature map may be linearly mapped to obtain multiple feature matrices, such as a first matrix, a second matrix, a third matrix, a fourth matrix, etc., and at least a portion of the feature matrices that best highlight the features in the first feature map and / or the second feature map may be selected to perform the process of processing the first feature map with the second feature map to obtain the processed first feature map, and processing the second feature map to obtain the processed second feature map.

[0044] Furthermore, in this embodiment, a set of processed first feature maps and processed second feature maps can be obtained after step S102; of course, in other embodiments, multiple sets of processed first feature maps and processed second feature maps can also be obtained after step S102; specifically, the above steps S102 and S103 may further include:

[0045] A. Output the processed first feature map and the processed second feature map; specifically, the processed first feature map and the processed second feature map can be output to step S103.

[0046] B. Perform feature extraction on the obtained processed first feature map to obtain a third feature map, and use the third feature map as the first feature map. Similarly, perform feature extraction on the processed second feature map to obtain a fourth feature map, and use the fourth feature map as the second feature map. Specifically, multiple different convolutional layers can be used to extract features from the processed first feature map and the processed second feature map respectively to obtain the corresponding third and fourth feature maps.

[0047] C. Repeat the steps of processing the first feature map with the second feature map to obtain the processed first feature map, and processing the second feature map to obtain the processed second feature map; that is, return to step S102 to obtain the next set of processed first feature maps and processed second feature maps.

[0048] By repeatedly executing the above process multiple times, multiple sets of processed first feature maps and multiple sets of processed second feature maps at different scales can be obtained. This method can improve the feature extraction capability of each branch network, making the difference regions extracted in the subsequent step S103 more accurate. The number of times the above process is repeated is consistent with the number of feature extraction modules in each branch network; and the parameters of the convolutional layer performing feature extraction can also be changed according to the number of repetitions.

[0049] S103: Obtain the difference region in the first remote sensing image and the second remote sensing image based on at least a portion of the first feature map, the second feature map, the processed first feature map, and the processed second feature map.

[0050] Specifically, the implementation process of step S103 includes: outputting the obtained first feature map, second feature map, all processed first feature maps, and all processed second feature maps to the feature fusion module, and fusing at least some of the feature maps to obtain a feature fusion map. Further, the decoder processes the feature fusion map to obtain the difference regions in the first and second remote sensing images. Optionally, methods such as overlay, subtraction, and feature distance calculation can be used to fuse at least some of the feature maps output to the feature fusion module to obtain the feature fusion map.

[0051] In the above embodiments, the remote sensing image change detection method provided in this application constructs a cross-processing network, which increases the information exchange between its feature extraction branches, makes full use of the correlation information between images of different time phases, improves the efficiency of feature extraction, and thus improves the accuracy of change detection.

[0052] In one application scenario, the remote sensing image detection method provided in this application can be implemented through a cross-processing network model. For example, when each branch of the cross-processing network contains four feature extraction modules, such as... Figure 5 As shown, Figure 5This is a schematic diagram of an embodiment of the cross-processing network model proposed in this application. The cross-processing network model includes two branch networks, and each branch network includes multiple feature extraction modules 10. The input of the first feature extraction module 10 in each branch network is a first remote sensing image or a second remote sensing image, and the output is a first feature map or a second feature map. The input of the remaining feature extraction modules other than the first feature extraction module 10 is the feature map output by the previous feature extraction module 10 on the adjacent branch network. Each feature extraction module 10 is used to obtain a processed first feature map or a processed second feature map based on the input feature map. That is, the aforementioned branch networks are used to implement some steps S101 and S102 in the above method.

[0053] Furthermore, the cross-processing network model may further include a feature fusion module 20 and a decoder 30. The feature fusion module 20 receives the outputs of all the feature extraction modules 10 described above, and is used to fuse at least some of the outputs of the feature extraction modules 10 to output a fused feature map. The decoder 30 receives the output of the feature fusion module 20 and is used to obtain the difference regions between the first and second remote sensing images from the fused feature map. That is, the feature fusion module 20 and the decoder 30 are used to implement step S103 in the above method.

[0054] It should be noted that, as Figure 5 As shown, Figure 5 In this diagram, only four feature extraction modules 10 are shown for each branch network. Those skilled in the art can design the number of feature extraction modules 10 for each branch network according to actual needs. For example, each branch network may include only one feature extraction module 10; or, for example, each branch network may include three, four, five, or other feature extraction modules 10.

[0055] by Figure 5 Taking the example of Siamese network model with cross-attention, the workflow includes:

[0056] 1) The first remote sensing image is input to the first feature extraction module 10 on branch network A. After processing by the first feature extraction module 10, a first feature map can be obtained. Similarly, the second remote sensing image is input to the first feature extraction module 10 on branch network B. After processing by the first feature extraction module 10, a second feature map can be obtained. The first feature map and the second feature map are transmitted to the feature fusion module 20.

[0057] 2) Input the first feature map into the second feature extraction module 10 on branch network A to obtain the corresponding processed first feature map; input the second feature map into the second feature extraction module 10 on branch network B to obtain the corresponding processed second feature map; transmit the processed first feature map or the processed second feature map obtained by the second feature extraction module 10 to the feature fusion module 20.

[0058] 3) Input the processed first feature map obtained by the second feature extraction module 10 on branch network A into the third feature extraction module 10 on branch network A to obtain a corresponding new processed first feature map; input the processed second feature map obtained by the second feature extraction module 10 on branch network B into the third feature extraction module 10 on branch network B to obtain a corresponding new processed second feature map; transmit the processed first feature map or the processed second feature map obtained by the third feature extraction module 10 to the feature fusion module.

[0059] 4) Repeat a process similar to step 3) above until the last feature extraction module 10 on the branch network obtains the corresponding processed first feature map or processed second feature map.

[0060] 5) The feature fusion module 20 fuses at least some of the received first feature map, second feature map, all processed first feature map and all processed second feature map to output a fused feature map.

[0061] 6) The decoder 30 obtains the difference regions in the first remote sensing image and the second remote sensing image from the above-mentioned fused feature map.

[0062] Please see Figure 6 , Figure 6 This is a schematic diagram of the framework structure of an embodiment of the remote sensing image change detection device of this application. The remote sensing image change detection device includes a first acquisition module 35, a second acquisition module 36, and a third acquisition module 37. The first acquisition module 35 is used to acquire a first remote sensing image and a second remote sensing image with the same region at different times, and to extract features from the first and second remote sensing images respectively to obtain a first feature map and a second feature map. The second acquisition module 36 is used to process the first feature map using the second feature map to obtain a processed first feature map, and to process the second feature map to obtain a processed second feature map. The third acquisition module 37 is used to obtain the difference regions in the first and second remote sensing images based on at least a portion of the first feature map, the second feature map, the processed first feature map, and the processed second feature map.

[0063] Please see Figure 7 , Figure 7This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device includes a memory 50 and a processor 40 coupled to each other. The memory 50 stores program instructions, and the processor 40 is used to execute the program instructions to implement the remote sensing image change detection method in any of the above embodiments. Specifically, the electronic device includes, but is not limited to, desktop computers, laptops, tablets, servers, etc., and is not limited thereto. In addition, the processor 40 may also be called a CPU (Central Processing Unit). The processor 40 may be an integrated circuit chip with signal processing capabilities. The processor 40 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor or any conventional processor. In addition, the processor 40 may be implemented by integrated circuit chips.

[0064] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a storage device according to an embodiment of the present application. The storage device 70 stores program instructions 80 that can be executed by a processor. The program instructions 80 are used to implement the steps in any of the remote sensing image change detection methods described above.

[0065] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting changes in remote sensing images, characterized in that, include: Acquire a first remote sensing image and a second remote sensing image that have the same area at different times, and extract features from the first remote sensing image and the second remote sensing image respectively to obtain a first feature map and a second feature map; The first feature map is processed using the second feature map to obtain a processed first feature map, and the second feature map is processed to obtain a processed second feature map. The difference regions in the first remote sensing image and the second remote sensing image are obtained based on at least a portion of the first feature map, the second feature map, the processed first feature map, and the processed second feature map; Prior to the step of obtaining the difference regions in the first remote sensing image and the second remote sensing image based on at least a portion of the first feature map, the second feature map, the processed first feature map, and the processed second feature map, the method includes: outputting the first feature map, the second feature map, the processed first feature map, and the processed second feature map; performing feature extraction on the processed first feature map to obtain a third feature map, and using the third feature map as the first feature map; performing feature extraction on the processed second feature map to obtain a fourth feature map, and using the fourth feature map as the second feature map; and repeatedly performing the steps of processing the first feature map using the second feature map to obtain the processed first feature map and processing the second feature map to obtain the processed second feature map.

2. The remote sensing image change detection method according to claim 1, characterized in that, The step of processing the first feature map using the second feature map to obtain the processed first feature map includes: Obtain the first matrix of the second feature map, and use the first matrix of the second feature map to process the first feature map to obtain the processed first feature map.

3. The remote sensing image change detection method according to claim 2, characterized in that, The step of processing the first feature map using the first matrix of the second feature map to obtain the processed first feature map includes: Obtain the second matrix of the first feature map, and use the first matrix of the second feature map and the second matrix of the first feature map to obtain the first weight matrix; Obtain the third matrix of the first feature map, and use the first weight matrix and the third matrix of the first feature map to obtain the first weighted feature map; The first feature map after processing is obtained using the first weighted feature map and the first feature map.

4. The remote sensing image change detection method according to claim 1, characterized in that, The step of processing the second feature map to obtain a processed second feature map includes: The second feature map is processed using the first feature map to obtain the processed second feature map.

5. The remote sensing image change detection method according to claim 1, characterized in that, The remote sensing image change detection method utilizes a cross-processing network, which includes: The system comprises two branch networks, each including multiple feature extraction modules. The first feature extraction module of each branch network takes a first remote sensing image or a second remote sensing image as input and outputs a first feature map or a second feature map. The inputs of the remaining feature extraction modules other than the first feature extraction module are the feature maps output by the previous feature extraction module on the adjacent branch network. Each feature extraction module is used to obtain the processed first feature map or the processed second feature map based on the input feature map.

6. The remote sensing image change detection method according to claim 1, characterized in that, The step of obtaining the difference regions in the first remote sensing image and the second remote sensing image based on at least a portion of the first feature map, the second feature map, the processed first feature map, and the processed second feature map includes: At least a portion of the first feature map, the second feature map, all the processed first feature maps, and all the processed second feature maps are fused to obtain a fused feature map; The difference regions in the first remote sensing image and the second remote sensing image are obtained from the fused feature map.

7. The remote sensing image change detection method according to claim 5, characterized in that, The cross-processing network also includes: The feature fusion module, whose input is the output of each feature extraction module, is used to fuse at least a portion of the first feature map, the second feature map, all the processed first feature maps, and all the processed second feature maps to obtain a fused feature map. The decoder, whose input is the output of the feature fusion module, is used to obtain the difference regions in the first remote sensing image and the second remote sensing image from the fused feature map.

8. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, wherein the processor stores program instructions and the processor is used to execute the program instructions to implement the remote sensing image change detection method according to any one of claims 1 to 7.

9. A storage device, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the remote sensing image change detection method according to any one of claims 1 to 7.

Images