A remote sensing image change detection method, system, device, medium and product based on a multi-task deep learning network model

Abstract

The application discloses a remote sensing image change detection method, system, device, medium and product based on a multi-task deep learning network model, relates to the field of remote sensing image processing, and comprises the following steps: acquiring double-time multi-source space data of a target area; the double-time multi-source space data comprises double-time optical images, double-time SAR images and a double-time digital elevation model; dividing the target area into multiple height levels according to the double-time digital elevation model and generating a height coding map; based on the double-time multi-source space data and the height coding map, a pre-trained multi-task deep learning network model is used to output a change detection map, a change type map and a height change map, so that remote sensing image change detection is realized; the multi-task deep learning network model comprises an encoder, an alignment and fusion module, a four-dimensional neighborhood difference convolution module and a stereo gradient enhancement decoder. The application can realize high-precision and quantifiable remote sensing image change detection in a stereo scene.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing, and in particular to a method, system, device, medium and product for detecting changes in remote sensing images based on a multi-task deep learning network model. Background Technology

[0002] With the rapid development of remote sensing satellite technology, the acquisition of multi-source aerospace data, such as high-resolution optical images, synthetic aperture radar (SAR) images, and digital elevation models (DEMs), has become increasingly convenient. Change detection, as one of the core tasks of remote sensing image analysis, has significant application value in urban planning, disaster emergency response, and environmental monitoring. Traditional change detection methods mainly process two-dimensional planar images, neglecting the three-dimensional spatial characteristics of ground features, leading to insufficient detection accuracy and high false detection rates in complex urban scenarios.

[0003] Currently, mainstream change detection technologies (such as Siamese networks and Transformer architecture) mainly have the following technological gaps: 1. Lack of stereo modeling capability: Existing methods such as FC-Siam and STANet treat images as two-dimensional planes, ignoring the height information of ground features. In scenarios involving only height changes, such as building additions and foundation construction, the detection capability is extremely weak.

[0004] 2. The difference mechanism is too simple: the traditional direct subtraction or channel splicing ignores the physical meaning of the features (such as the coupling between spectrum and altitude), and it is difficult to solve the spurious changes caused by different spectra of the same object.

[0005] 3. Insufficient heterogeneous data fusion: Existing methods often lack explicit geometric and semantic alignment when fusing optical, SAR and DEM data, resulting in multi-source data not only failing to complement each other, but also introducing noise.

[0006] 4. Lack of quantitative indicators in the results: Existing technologies usually only output binary masks (changed / unchanged), which cannot quantitatively give the height change value of ground features, making it difficult to meet the needs of refined surveying and mapping. Summary of the Invention

[0007] The purpose of this application is to provide a method, system, device, medium and product for detecting changes in remote sensing images based on a multi-task deep learning network model, which can achieve high-precision and quantifiable change detection in three-dimensional scenes.

[0008] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for detecting changes in remote sensing images based on a multi-task deep learning network model, including: Acquire dual-temporal multi-source aerospace data of the target area; the dual-temporal multi-source aerospace data includes dual-temporal optical images, dual-temporal SAR images, and dual-temporal digital elevation models; The target area is divided into multiple height levels based on the dual-temporal digital elevation model, and a height-coded map is generated. Based on the dual-temporal multi-source aerospace data and the height-coded map, a pre-trained multi-task deep learning network model is used to output a change detection map, a change type map, and a height change map to realize remote sensing image change detection. The multi-task deep learning network model includes an encoder, an alignment and fusion module, a four-dimensional neighborhood differential convolution module, and a stereo gradient enhancement decoder. Specifically, based on the dual-temporal multi-source aerospace data and the altitude coding map, a pre-trained multi-task deep learning network model is used to output a change detection map, a change type map, and an altitude change map, including: Based on the aforementioned dual-temporal multi-source aerospace data, an encoder is used to extract multi-source features; Based on the height coding map, an alignment and fusion module is used to adaptively align and weightedly fuse the multi-source features to obtain dual-temporal fusion features; Based on the dual-temporal fusion features and the height encoding map, a four-dimensional neighborhood difference convolution module is used to generate global difference features; Based on the dual-temporal fusion features and the global difference features, a stereo gradient enhancement decoder is used for decoding to obtain the decoded features; Based on decoding features, change detection of remote sensing images is achieved by simultaneously outputting change detection maps, change type maps, and height change maps through parallel multi-task heads.

[0009] Secondly, this application provides a remote sensing image change detection system based on a multi-task deep learning network model. The system is applied to the aforementioned remote sensing image change detection method based on a multi-task deep learning network model. The system includes: The data acquisition module is used to acquire dual-temporal multi-source aerospace data of the target area; the dual-temporal multi-source aerospace data includes dual-temporal optical images, dual-temporal SAR images, and dual-temporal digital elevation models; The height-coded map generation module is used to divide the target area into multiple height levels according to the dual-temporal digital elevation model and generate a height-coded map. The change detection module is used to output a change detection map, a change type map, and a height change map based on the dual-temporal multi-source aerospace data and the height encoding map, using a pre-trained multi-task deep learning network model to realize change detection in remote sensing images; the multi-task deep learning network model includes an encoder, an alignment and fusion module, a four-dimensional neighborhood differential convolution module, and a stereo gradient enhancement decoder. The change detection module specifically includes: The multi-source feature extraction unit is used to extract multi-source features based on the dual-temporal multi-source aerospace data using an encoder; The dual-temporal fusion feature generation unit is used to adaptively align and weightedly fuse the multi-source features based on the height coding map using an alignment and fusion module to obtain dual-temporal fusion features. A global difference feature generation unit is used to generate global difference features based on the dual-temporal fusion features and the height coding map using a four-dimensional neighborhood difference convolution module; The decoding unit is used to decode based on the dual-temporal fusion features and the global difference features using a stereo gradient enhancement decoder to obtain the decoded features; The change detection unit is used to simultaneously output change detection maps, change type maps, and height change maps based on decoding features through parallel multi-task heads, thereby realizing change detection in remote sensing images.

[0010] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described remote sensing image change detection method based on a multi-task deep learning network model.

[0011] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned remote sensing image change detection method based on a multi-task deep learning network model.

[0012] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned remote sensing image change detection method based on a multi-task deep learning network model.

[0013] According to the specific embodiments provided in this application, this application has the following technical effects: (1) This application introduces DEM data and generates a height-coded map, explicitly integrating the three-dimensional spatial features (height information) of ground objects into the model input; combined with a stereo gradient enhancement decoder, the network can perceive subtle changes in ground objects in the vertical direction. This not only solves the problem that traditional two-dimensional methods cannot identify pure height changes (such as the addition of floors), but also achieves a leap from qualitative detection to quantitative height change measurement, significantly improving the detection accuracy of three-dimensional ground object changes in complex urban scenarios; (2) This application utilizes four-dimensional neighborhood differential convolution, which can capture deep differences between features in a wider spatiotemporal neighborhood. This mechanism not only considers the changes in spectrum / intensity, but also implicitly models the joint spatial context and height gradient, thereby effectively distinguishing real physical changes from pseudo-changes caused by illumination, season or imaging angle, reducing the false detection rate and improving the robustness of change detection; (3) This application uses the generated height-coded map as a geometric constraint prior to guide the fusion process of multi-source features from optics, SAR, and DEM. Through an adaptive alignment and weighting mechanism, the model can automatically adjust the contribution weights of different source data according to the terrain height level and achieve accurate registration at the geometric level. This eliminates geometric misalignment and semantic conflict between multi-source data, realizes true complementary advantages (such as using SAR to penetrate clouds and fog, optics to provide texture, and DEM to provide structure), and significantly suppresses noise interference caused by the fusion of multi-source heterogeneous data; (4) This application uses a multi-task parallel learning architecture to produce multi-dimensional detection results at once. In addition to determining whether there is a change and the type of change, it can also directly output specific height change values. This makes the method not only serve macro-monitoring, but also directly meet the needs of urban planning, disaster assessment and other fields for refined quantitative analysis, which greatly improves the application value and decision support capabilities of remote sensing image change detection. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating a remote sensing image change detection method based on a multi-task deep learning network model, provided as an embodiment of this application; Figure 2 A schematic diagram of the overall architecture of a multi-task deep learning network model; Figure 3 This is a schematic diagram of a stereo gradient enhancement decoder. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] In one exemplary embodiment, such as Figure 1 As shown, a remote sensing image change detection method based on a multi-task deep learning network model is provided. This method is executed by a computer device, specifically by a computer device such as a terminal or a server alone, or by a terminal and a server together. In this embodiment, the method is described using a server as an example, including the following steps S1 to S3.

[0019] S1: Acquire dual-temporal multi-source aerospace data of the target area; the dual-temporal multi-source aerospace data includes dual-temporal optical images, dual-temporal SAR images and dual-temporal digital elevation models.

[0020] Specifically, dual-temporal optical imaging: Dual-temporal SAR imagery: Dual-phase digital elevation model: .in, , These represent the height and width of the image, respectively. The acquired data undergoes geometric registration (using SIFT feature point matching + RANSAC algorithm to register the SAR image to the optical image coordinate system, with registration error controlled within 0.3 pixels), radiometric correction, and normalization to ensure precise spatial alignment of the dual-temporal, multi-source aerospace data.

[0021] S2: Divide the target area into multiple height levels according to the dual-temporal digital elevation model and generate a height-coded map.

[0022] SegEncode is a height-level hierarchical encoding function that discretizes continuous elevation values ​​into... A high level (e.g.: 0-10m ground level 10-30m low-rise buildings 30-100m high-rise buildings (>100m super high-rise buildings), output height encoding map ∈ .

[0023] S3: Based on the aforementioned dual-temporal multi-source aerospace data and the aforementioned height coding map, a pre-trained multi-task deep learning network model is used to output a change detection map, a change type map, and a height change map, thereby achieving remote sensing image change detection. For example... Figure 2As shown, the multi-task deep learning network model includes an encoder (including an optical encoder, a SAR encoder, and a DEM encoder), an alignment and fusion module, a four-dimensional neighborhood differential convolution (4D-NDC) module, and a stereo gradient enhancement decoder.

[0024] Specifically, step S3 includes the following steps S31-S35.

[0025] S31: Based on the aforementioned dual-temporal multi-source aerospace data, an encoder is used to extract multi-source features.

[0026] The encoder (with ResNet-50 as the backbone network) consists of a three-branch Siamese network: an optical encoder, a SAR encoder, and a DEM encoder. The three-branch Siamese network is used to extract multi-source features corresponding to dual-temporal optical images, dual-temporal SAR images, and dual-temporal digital elevation models, respectively. in, It is an optical encoder. , Optical characteristics, For SAR encoder, , SAR characteristics For DEM encoder, , This is an elevation feature.

[0027] S32: Based on the height coding map, the alignment and fusion module is used to adaptively align and weightedly fuse the multi-source features to obtain dual-temporal fusion features.

[0028] Channel attention weight generation: .

[0029] The channel attention module is implemented as follows: first, global average pooling is performed on each source feature and the features are concatenated, then a fully connected layer and Softmax are applied to generate the channel attention module. . These are optical multi-source features, SAR multi-source features, and elevation multi-source features, respectively.

[0030] High-level guided spatial alignment: .

[0031] This represents a spatial transformation operation. These are the spatially aligned SAR features.

[0032] Specifically, through learnable convolutional networks Predicting the 2D offset field based on the height encoding map : Then, deformable convolution is applied to the SAR features to achieve highly guided spatial alignment: Where p represents the two-dimensional spatial location in the SAR feature. Represents the convolutional neighborhood The sampling offset position within, Represents the convolution weights. Indicates the location The predicted two-dimensional offset field, This represents the SAR feature after spatial alignment at position p.

[0033] Weighted fusion: .

[0034] in, Represents element-wise multiplication, merging features , Indicates the number of feature channels.

[0035] The above steps can be used to obtain... , and Dual-temporal fusion features after alignment and fusion , and, will , and Dual-temporal fusion features after alignment and fusion .

[0036] S33: Based on the aforementioned dual-temporal fusion features and the aforementioned height encoding map, a four-dimensional neighborhood differential convolution module is used to generate global differential features. Specifically, this includes: S331: Construct a four-dimensional feature tensor based on the dual-temporal fusion features and the height coding map.

[0037] Targeting the dual-temporal fusion characteristics Construct a four-dimensional feature tensor : in, , indicating the time dimension Indicates height dimension, This indicates a splicing operation.

[0038] S332: Based on the four-dimensional feature tensor, four-dimensional neighborhood difference convolution is used, combined with cross-difference modeling to obtain global difference features. Specifically, based on the height encoding map, cross-difference modeling is performed on the four-dimensional feature tensor to obtain a cross-stitched four-dimensional feature tensor; four-dimensional neighborhood difference convolution is performed on the cross-stitched four-dimensional feature tensor to obtain local difference features; and two-dimensional convolution aggregation is performed on the local difference features to generate global difference features.

[0039] To fully explore the deep differences in the characteristics of the two time phases, a highly perceptive cross-stitching strategy was designed. , This is a cross-stitched four-dimensional feature tensor. In the specific implementation, it is based on the height encoding map. Dynamically adjust the channel interleaving mode for height layers The channel index allocation rules are as follows: Phase 1 contributes to the channel: Phase 2 contribution channel: ; c represents the channel index, It is a modulo operation.

[0040] After cross-stitching, four-dimensional neighborhood difference convolution is applied to extract local difference features. : in, This represents the current position in four-dimensional space. It is a four-dimensional neighborhood with a size of (Channel × Time × Space × Height) The highly sensitive weighting function is defined as follows: exp(·) represents the exponential function, which maps the height difference to a weight that decreases as the difference increases, thereby enhancing the differential response of the same or similar height layers and suppressing irrelevant differences across height layers. This function makes the differential weight within the same height layer larger, and the difference between different height differences is suppressed. For height-scale parameters, For neighborhood location, It is located at the center.

[0041] Then, global differential representations are learned through 2D convolution to obtain global differential features. : This represents a 2D convolution operation. This represents the activation function.

[0042] S34: Based on the dual-temporal fusion features and the global difference features, a stereo gradient enhancement decoder is used for decoding to obtain the decoded features. Specifically, this includes: constructing shallow features based on the dual-temporal fusion features; performing stereo gradient detection based on the shallow features to determine gradient features; calculating the absolute difference of the dual-temporal digital elevation models and generating a height jump mask based on the absolute difference; generating channel attention weights for the global difference features and generating attention-weighted features based on the channel attention weights; generating enhanced features based on the gradient features, the height jump mask, and the attention-weighted features; and using a feature pyramid network to progressively upsample and fuse the enhanced features from deep to shallow layers to obtain the decoded features. Detailed procedures are as follows: Figure 3 As shown.

[0043] To preserve the 3D boundary details of the variable instances, a stereo gradient enhancement decoder (SGR-Decoder) is designed.

[0044] First, construct shallow features. , where (|) represents channel dimension splicing.

[0045] Subsequently, a 3D Laplacian operator is applied to the shallow fusion features to enhance the response to spatial edges and height abrupt changes. The continuous form of the 3D Laplacian operator is defined as follows: Where x and y represent planar spatial coordinates, and z represents the elevation dimension. This represents the three-dimensional feature field or scalar characteristic function to be processed. The Laplacian operator is defined as the sum of the second derivatives of the scalar characteristic function in the x, y, and z directions, used to characterize local edges and abrupt changes.

[0046] In this embodiment, the 3D Laplacian convolution can be achieved by combining bi-temporal DEM data with voxelized representation of shallow features. Specifically, a 3D feature volume is constructed. : This is the voxelization mapping function.

[0047] The three-dimensional boundary response features are obtained by convolution using a three-dimensional Laplacian convolution kernel. : This represents the 3D convolution operation corresponding to the 3D Laplacian discrete template. This represents the three-dimensional convolution kernel corresponding to the three-dimensional Laplacian discrete template.

[0048] Subsequently, the three-dimensional boundary response features are aggregated along the height dimension to obtain gradient enhancement features: .

[0049] This is a max pooling operation performed along the height dimension Z.

[0050] Then, height jump detection is performed. The absolute difference between the two-phase digital elevation models (i.e., DEM difference) is calculated. : Apply adaptive threshold detection to regions of height abrupt change (i.e. Figure 3 (Threshold detection in the middle) to obtain the high transition mask : Among them, adaptive threshold , and These are the mean and standard deviation, respectively. This is an indicator function.

[0051] Next, attention fusion (i.e.) Figure 3 Attention in the context of global differential features (i.e., Figure 3 (deep features in) Generate channel attention weights : in, Indicates global difference features; This represents a global pooling operation, used to aggregate features in a spatial dimension to obtain a global statistical description; This represents a fully connected mapping used to generate attention weights for each channel based on global difference features; V represents the generated channel attention weight vector, whose dimension is... The number of channels is consistent, which is used to adaptively weight the global differential features.

[0052] Calculate attention-weighted features : in, This is the Sigmoid function.

[0053] Fuse gradient features with height jump masks: in, This represents pointwise addition and outputs enhanced features. .

[0054] Finally, multi-scale feature aggregation is performed. A feature pyramid network structure is used to aggregate multi-scale differential features to obtain the decoded features. : This indicates a stepwise upsampling from deeper to shallower layers. Indicates an upsampling operation. This indicates a convolution operation.

[0055] S35: Based on decoding features, it simultaneously outputs change detection maps, change type maps (such as new construction, demolition, and reconstruction) and height change maps through parallel multi-task heads, thereby realizing change detection in remote sensing images.

[0056] Change detection chart: Classification of change types: Change types include: New ( ),tear down( ), renovation ( ), unchanged ( Quantification of high-level changes: Output the height change of each pixel (in meters). , and These represent three parallel methods used for binary change detection, change type classification, and high-change regression, respectively. Convolutional prediction head. It is used to map decoded features into binary classification responses, and output the prediction result of whether each pixel belongs to the changed or unchanged category; It is used to map decoded features into multi-class responses and output the prediction result of the change type corresponding to each pixel; It is used to map decoded features to height variation, and outputs the height variation regression value for each pixel.

[0057] Apply opening and closing operations to the change detection map to remove noise and fill holes: in, The structuring elements have sizes of [sizes]. and ; This indicates the opening operation. This indicates the closing operation.

[0058] The output includes the following information: Change Detection Map: Change type diagram: Altitude change map: .

[0059] This application constructs a multi-task deep learning network that includes an encoder, alignment and fusion module, a four-dimensional neighborhood differential convolution module, and a stereo gradient enhancement decoder, thereby achieving true three-dimensional stereo change detection. This significantly improves anti-interference capability and detection accuracy, and provides multi-dimensional quantitative analysis results.

[0060] The loss function used in the training process of the above multi-task deep learning network model is a joint loss for multiple tasks: in, This is a combined loss due to multiple tasks; , where is the cross-entropy loss, and i represents the pixel index. This represents the binary true label of the i-th pixel. This represents the probability that the i-th pixel, as predicted by the model, belongs to the change category. , where Y represents the set of pixels corresponding to the actual change region, P represents the set of pixels corresponding to the change region predicted by the model, |Y| represents the number of pixels in the actual change region, and |P| represents the number of pixels in the predicted change region; This represents the number of pixels in the overlapping area between the actual change region and the predicted change region. Let be the type classification loss, where q represents the index of the change type category. This represents the actual label value for the corresponding category. This represents the probability that the model predicts a change of type q. , where N represents the total number of pixels involved in the height regression loss calculation. This represents the actual height change value of the i-th pixel. This represents the predicted height change value of the i-th pixel. Weighting coefficients. . and The primary task has the highest weight. and To support the main task, reduce its weight appropriately to avoid interference with the main task.

[0061] Post-processing: Confidence assessment is performed, and the detection confidence level is calculated based on the consistency of multi-source data. : in, Confidence predictions are generated for the optical branch, and the same applies to the SAR and DEM branches. Output confidence plot: ∈ .

[0062] Quantitative evaluation was performed on the LEVIR-CD dataset, and the evaluation results are shown in Table 1.

[0063] Table 1

[0064] The F1 score of this application reached 93.82%, which is 2.18 percentage points higher than the second-best method Changer, and the intersection-union ratio (IoU) is 3.59 percentage points higher.

[0065] Ablation experiments were conducted to verify the contributions of each module, as shown in Table 2.

[0066] Table 2

[0067] Table 2 shows that the multi-source fusion, four-dimensional neighborhood differential convolution, height-aware cross-difference, and stereo gradient enhancement decoder proposed in this application can effectively improve change detection performance, and each module has good superposition gain. With the gradual introduction of each module, the model's F1 score increased from 89.45% to 93.82%, and the IoU increased from 80.93% to 88.35%. This indicates that this application can fully integrate multi-source remote sensing information, enhance the ability to model dual-temporal differences, and improve the accuracy of characterizing the boundaries and height changes of changing targets in complex scenes, thereby achieving better change detection results.

[0068] Based on the same inventive concept, this application also provides a system for implementing the remote sensing image change detection based on a multi-task deep learning network model as described above. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the remote sensing image change detection system based on a multi-task deep learning network model provided below can be found in the limitations of the remote sensing image change detection method based on a multi-task deep learning network model described above, and will not be repeated here.

[0069] In one exemplary embodiment, a remote sensing image change detection system based on a multi-task deep learning network model is provided, including the following modules.

[0070] The data acquisition module is used to acquire dual-temporal multi-source aerospace data of the target area; the dual-temporal multi-source aerospace data includes dual-temporal optical images, dual-temporal SAR images and dual-temporal digital elevation models.

[0071] The height-coded map generation module is used to divide the target area into multiple height levels according to the dual-temporal digital elevation model and generate a height-coded map.

[0072] The change detection module is used to output a change detection map, a change type map, and a height change map based on the dual-temporal multi-source aerospace data and the height encoding map, using a pre-trained multi-task deep learning network model to realize change detection in remote sensing images; the multi-task deep learning network model includes an encoder, an alignment and fusion module, a four-dimensional neighborhood differential convolution module, and a stereo gradient enhancement decoder.

[0073] The change detection module specifically includes: The multi-source feature extraction unit is used to extract multi-source features based on the dual-temporal multi-source aerospace data using an encoder; The dual-temporal fusion feature generation unit is used to adaptively align and weightedly fuse the multi-source features based on the height coding map using an alignment and fusion module to obtain dual-temporal fusion features. A global difference feature generation unit is used to generate global difference features based on the dual-temporal fusion features and the height coding map using a four-dimensional neighborhood difference convolution module; The decoding unit is used to decode based on the dual-temporal fusion features and the global difference features using a stereo gradient enhancement decoder to obtain the decoded features; The change detection unit is used to simultaneously output change detection maps, change type maps, and height change maps based on decoding features through parallel multi-task heads, thereby realizing change detection in remote sensing images.

[0074] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, memory, and I / O are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores data to be processed. The I / O interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with an external terminal via a network connection. When the computer program is executed by the processor, it implements the steps in the above-described method embodiments.

[0075] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0076] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0077] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0078] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0079] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting changes in remote sensing images based on a multi-task deep learning network model, characterized in that, include: Acquire dual-temporal multi-source aerospace data of the target area; the dual-temporal multi-source aerospace data includes dual-temporal optical images, dual-temporal SAR images, and dual-temporal digital elevation models; The target area is divided into multiple height levels based on the dual-temporal digital elevation model, and a height-coded map is generated. Based on the dual-temporal multi-source aerospace data and the height-coded map, a pre-trained multi-task deep learning network model is used to output a change detection map, a change type map, and a height change map to realize remote sensing image change detection. The multi-task deep learning network model includes an encoder, an alignment and fusion module, a four-dimensional neighborhood differential convolution module, and a stereo gradient enhancement decoder. Specifically, based on the dual-temporal multi-source aerospace data and the altitude coding map, a pre-trained multi-task deep learning network model is used to output a change detection map, a change type map, and an altitude change map, including: Based on the aforementioned dual-temporal multi-source aerospace data, an encoder is used to extract multi-source features; Based on the height coding map, an alignment and fusion module is used to adaptively align and weightedly fuse the multi-source features to obtain dual-temporal fusion features; Based on the dual-temporal fusion features and the height encoding map, a four-dimensional neighborhood difference convolution module is used to generate global difference features; Based on the dual-temporal fusion features and the global difference features, a stereo gradient enhancement decoder is used for decoding to obtain the decoded features; Based on decoding features, change detection of remote sensing images is achieved by simultaneously outputting change detection maps, change type maps, and height change maps through parallel multi-task heads.

2. The remote sensing image change detection method based on a multi-task deep learning network model according to claim 1, characterized in that, Based on the aforementioned dual-temporal fusion features and the aforementioned height encoding map, a four-dimensional neighborhood difference convolution module is used to generate global difference features, specifically including: A four-dimensional feature tensor is constructed based on the dual-temporal fusion features and the height coding map; Based on the aforementioned four-dimensional feature tensor, four-dimensional neighborhood difference convolution is used, combined with cross-difference modeling to obtain global difference features.

3. The remote sensing image change detection method based on a multi-task deep learning network model according to claim 2, characterized in that, Based on the aforementioned four-dimensional feature tensor, four-dimensional neighborhood difference convolution is employed, combined with cross-difference modeling, to obtain global difference features, specifically including: Based on the height encoding map, cross-difference modeling is performed on the four-dimensional feature tensor to obtain a cross-stitched four-dimensional feature tensor. Perform a four-dimensional neighborhood difference convolution operation on the cross-stitched four-dimensional feature tensor to obtain local difference features; The local differential features are aggregated by two-dimensional convolution to generate global differential features.

4. The remote sensing image change detection method based on a multi-task deep learning network model according to claim 1, characterized in that, Based on the dual-temporal fusion features and the global difference features, a stereo gradient enhancement decoder is used for decoding to obtain the decoded features, specifically including: Shallow features are constructed based on the dual-temporal fusion features; Based on the shallow features, perform stereo gradient detection to determine the gradient features; Calculate the absolute difference between the two-phase digital elevation models and generate a height jump mask based on the absolute difference; Channel attention weights are generated for the global difference features, and attention-weighted features are generated based on the channel attention weights; Enhanced features are generated based on the gradient features, the height jump mask, and the attention-weighted features; The enhanced features are upsampled and fused from deep to shallow layers using a feature pyramid network to obtain the decoded features.

5. The remote sensing image change detection method based on a multi-task deep learning network model according to claim 1, characterized in that, The loss function of the multi-task deep learning network model during training is: in, For multi-task joint losses, For cross-entropy loss, For Dice's loss, For type classification loss, For highly regressive loss, , , and All are weighting coefficients.

6. The remote sensing image change detection method based on a multi-task deep learning network model according to claim 1, characterized in that, The encoder includes a three-branch twin network.

7. A remote sensing image change detection system based on a multi-task deep learning network model, characterized in that, The system is applied to the remote sensing image change detection method based on a multi-task deep learning network model as described in any one of claims 1-6, and the system comprises: The data acquisition module is used to acquire dual-temporal multi-source aerospace data of the target area; the dual-temporal multi-source aerospace data includes dual-temporal optical images, dual-temporal SAR images, and dual-temporal digital elevation models; The height-coded map generation module is used to divide the target area into multiple height levels according to the dual-temporal digital elevation model and generate a height-coded map. The change detection module is used to output a change detection map, a change type map, and a height change map based on the dual-temporal multi-source aerospace data and the height encoding map, using a pre-trained multi-task deep learning network model to realize change detection in remote sensing images; the multi-task deep learning network model includes an encoder, an alignment and fusion module, a four-dimensional neighborhood differential convolution module, and a stereo gradient enhancement decoder. The change detection module specifically includes: The multi-source feature extraction unit is used to extract multi-source features based on the dual-temporal multi-source aerospace data using an encoder; The dual-temporal fusion feature generation unit is used to adaptively align and weightedly fuse the multi-source features based on the height coding map using an alignment and fusion module to obtain dual-temporal fusion features. A global difference feature generation unit is used to generate global difference features based on the dual-temporal fusion features and the height coding map using a four-dimensional neighborhood difference convolution module; The decoding unit is used to decode based on the dual-temporal fusion features and the global difference features using a stereo gradient enhancement decoder to obtain the decoded features; The change detection unit is used to simultaneously output change detection maps, change type maps, and height change maps based on decoding features through parallel multi-task heads, thereby realizing change detection in remote sensing images.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the remote sensing image change detection method based on a multi-task deep learning network model as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the remote sensing image change detection method based on a multi-task deep learning network model as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the remote sensing image change detection method based on a multi-task deep learning network model as described in any one of claims 1-6.

Images