Deep learning-based target object change detection method, device, equipment and medium

Abstract

This invention belongs to the field of image processing technology and provides a method, apparatus, device, and medium for detecting object changes based on deep learning. The method includes acquiring training samples of a pair of dual-temporal remote sensing images registered with the object and their corresponding pixel-level binary change annotation maps; constructing an object change detection model, including a feature extraction module for extracting multi-scale features from the two-temporal images, a bidirectional semantic alignment module for fusing the multi-scale features of the two-temporal images, and a decoder module for restoring spatial details; training the object change detection model using the training samples to obtain a trained object change detection model; inputting the dual-temporal remote sensing image pair registered with the object to be detected into the trained object change detection model, and outputting the corresponding binary change prediction map. This invention achieves accurate and robust segmentation of changed object regions from dual-temporal low-altitude images.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method, apparatus, device, and medium for detecting changes in target objects based on deep learning. Background Technology

[0002] With the rapid development of Earth observation technology, low-altitude airspace has become an indispensable data source for monitoring urban development, assessing disaster losses, and updating geographic information databases. Among these, target change detection, as a core task of Earth image interpretation, aims to automatically identify and locate areas of change in targets by analyzing low-altitude image data acquired at different times. It has wide-ranging applications in urban planning, illegal construction supervision, disaster assessment, and rescue. Accurate and automated detection of target changes is crucial for achieving dynamic perception and refined management in smart cities.

[0003] Object change detection, especially of buildings and structures, is essentially a pixel-level or object-level classification problem of multi-temporal low-altitude images, that is, determining whether each pixel or each image object belongs to the "changed" or "unchanged" category. However, due to differences in imaging conditions (such as lighting, season, sensor angle, and weather), as well as the complexity of the building's structure and interference from the surrounding environment (such as shadows and vegetation obstruction), how to robustly extract discriminative change features and effectively suppress "spurious changes" (such as changes caused by lighting, vehicle movement, and vegetation growth) has long been a serious challenge for this technology.

[0004] Traditional change detection methods mainly rely on image algebraic methods (such as image differencing and ratio methods) and image classification methods (such as principal component analysis and change vector analysis). These methods usually require complex preprocessing (such as radiometric correction and precise registration), and their feature representation capabilities are limited, heavily relying on manually designed features. When faced with complex scenes and various interferences, their generalization ability and detection accuracy often fail to meet the needs of practical applications.

[0005] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has brought revolutionary breakthroughs to the field of change detection due to its powerful ability to automatically extract and represent hierarchical features. Some studies employ a pixel-level comparison paradigm, first extracting deep features from bi-temporal images using independent or weight-shared CNN branches, then constructing difference maps through feature differencing, feature concatenation, or convolution operations, and finally classifying the difference maps to obtain change detection maps. Others have proposed a Siamese network-based architecture, ensuring consistency in feature extraction through a dual-branch network with shared weights, and combining attention mechanisms to focus on significant change regions. Still others have explored fully convolutional networks and U-Net structures, fusing deep semantic information with shallow detail information through encoder-decoder paths and skip connections to achieve end-to-end pixel-level accurate segmentation. Furthermore, to address the imbalanced sample problem, some researchers have introduced weighted cross-entropy loss, Dice loss, or adversarial loss to optimize model training. Still other studies treat change detection as a spatiotemporal sequence analysis problem, attempting to combine recurrent neural networks or Transformer models to capture long-range dependencies between multi-temporal images.

[0006] Despite significant progress in deep learning-based methods, they still face numerous challenges: how to design efficient network architectures to better integrate dual-temporal and multi-level features; how to effectively distinguish between real building (structure) changes and various pseudo-changes; how to reduce the model's dependence on large amounts of finely labeled data; and how to improve the model's generalization ability across regions and sensor scenarios. Therefore, developing a building (structure) change detection algorithm that can balance high accuracy, strong robustness, and good generalization remains a key problem that urgently needs to be solved in the field of low-altitude imagery.

[0007] Insufficient dual-temporal feature fusion capability: Many existing methods employ simple feature concatenation or difference operations for dual-temporal feature fusion. This shallow fusion strategy is insufficient to fully capture the complex spatiotemporal dependencies and interaction information between images from different periods, resulting in incomplete semantic representation of changes, easy loss of subtle change features, or introduction of fusion noise.

[0008] Limited ability to suppress "spurious changes": "Spurious changes" caused by non-target factors such as changes in illumination, seasonal changes, vehicle movement, and vegetation growth are common in low-altitude imagery. Existing models often lack targeted mechanisms to distinguish changes in the essential features of ground features from these apparent disturbances, resulting in a high false alarm rate, especially in complex urban scenes and cross-seasonal data, where detection accuracy drops significantly.

[0009] Poor representation of multi-scale buildings: Buildings in cities vary greatly in scale, from large factories to small residences. Traditional encoder-decoder structures may lose detailed information about small-scale buildings during feature extraction or fail to effectively model the overall structure of large-scale buildings, leading to issues such as missed detection of small buildings or incomplete boundaries of large buildings in the detection results. Summary of the Invention

[0010] To address the problems of insufficient fusion of dual-temporal features, sensitivity to "pseudo-changes," weak multi-scale feature representation capabilities, and strong dependence on large amounts of labeled data in existing deep learning change detection methods, this invention provides a deep learning-based method, apparatus, device, and medium for detecting changes in target objects.

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] On the one hand, a deep learning-based method for detecting changes in target objects is provided, comprising the following steps:

[0013] Obtain a training dataset, wherein the training samples in the training dataset include dual-temporal remote sensing image pairs registered with the target object and their corresponding pixel-level binary variation annotation maps;

[0014] A target change detection model is constructed, which includes a feature extraction module for extracting multi-scale features of two temporal images in a mid-image pair, a bidirectional semantic alignment module for fusing multi-scale features of two temporal images, and a decoder module for restoring spatial details.

[0015] A target object change detection model is trained using a training dataset. During the training process, the target object change detection model is optimized using a constructed loss function to obtain a well-trained target object change detection model.

[0016] The dual-temporal remote sensing image pairs registered with the target object to be detected are input into the trained target object change detection model, and the corresponding binary change prediction map is output.

[0017] On the other hand, a target change detection device based on deep learning is provided, comprising:

[0018] The first module is used to acquire a training dataset, wherein the training samples in the training dataset include dual-temporal remote sensing image pairs registered with the target object and their corresponding pixel-level binary change annotation maps.

[0019] The second module is used to build a target change detection model. The model includes a feature extraction module for extracting multi-scale features of the two temporal images in the mid-phase remote sensing image pair, a bidirectional semantic alignment module for fusing multi-scale features of the two temporal images, and a decoder module for restoring spatial details.

[0020] The third module is used to train the target object change detection model using the training dataset. During the training process, the constructed loss function is used to optimize the target object change detection model to obtain a well-trained target object change detection model.

[0021] The fourth module is used to input the dual-temporal remote sensing image pairs registered with the target object to be detected into the trained target object change detection model, and output the corresponding binary change prediction map.

[0022] On the other hand, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described deep learning-based target change detection method.

[0023] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described deep learning-based target change detection method.

[0024] On the other hand, the present invention provides a computer program product stored on a computer-readable storage medium and including computer instructions that, when executed by a processor, cause an electronic device to implement the steps of the above-described deep learning-based target change detection method.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0026] This invention provides a target change detection method based on deep learning. By constructing an end-to-end target change detection model, the deep learning network model includes a feature extraction module for extracting multi-scale features of two temporal images in a dual-temporal remote sensing image pair, a bidirectional semantic alignment module for fusing multi-scale features of two temporal images, and a decoder module for restoring spatial details, thereby achieving accurate and robust segmentation of changed target areas from dual-temporal low-altitude images.

[0027] Specifically, the target object change detection model first utilizes a shared-weight feature extraction backbone network to extract deep features from dual-temporal images. Second, a bidirectional semantic alignment module is introduced to perform deep feature interaction and fusion at multiple scales, rather than simply stitching or differencing, thereby more fully capturing spatiotemporal dependencies. Then, the decoder module gradually recovers spatial details and integrates contextual information from different scales through upsampling, cross-layer connections, and feature aggregation to simultaneously detect changes in target objects (such as buildings) at both large and small scales. Furthermore, this invention optimizes model training through a strategy combining edge enhancement loss function and online hard sample mining, effectively suppressing spurious changes and improving the segmentation accuracy of target object boundaries in the target object change detection model. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of a target change detection method based on deep learning in one embodiment;

[0030] Figure 2 This is a block diagram of an electronic device in one embodiment. Detailed Implementation

[0031] The technical solution of the present invention will now be clearly and completely described through specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0032] In one embodiment, reference is made to Figure 1 This paper provides a deep learning-based method for detecting changes in target objects, comprising the following steps:

[0033] Obtain a training dataset, wherein the training samples in the training dataset include dual-temporal remote sensing image pairs registered with the target object and their corresponding pixel-level binary variation annotation maps;

[0034] A target change detection model is constructed, which includes a feature extraction module for extracting multi-scale features of two temporal images in a mid-image pair, a bidirectional semantic alignment module for fusing multi-scale features of two temporal images, and a decoder module for restoring spatial details.

[0035] A target object change detection model is trained using a training dataset. During the training process, the target object change detection model is optimized using a constructed loss function to obtain a well-trained target object change detection model.

[0036] The dual-temporal remote sensing image pairs registered with the target object to be detected are input into the trained target object change detection model, and the corresponding binary change prediction map is output.

[0037] This invention is applicable to a variety of application scenarios. The target objects described in this invention include, but are not limited to, buildings, structures, lakes, and topography.

[0038] Furthermore, the registered dual-temporal remote sensing image pair is input into the feature extraction module, which includes two feature extraction backbone networks with shared weights. The two temporal images in the registered dual-temporal remote sensing image pair are respectively input into the two shared weight feature extraction backbone networks for multi-scale feature extraction, generating multi-scale feature map groups corresponding to the two temporal images. Each multi-scale feature map group contains feature maps of different levels from shallow to deep.

[0039] In one embodiment, the two temporal images in the registered dual-temporal remote sensing image pair and Inputting each feature extraction backbone network with shared weights (such as ResNet-50 / 101) generates multi-scale feature maps. , , , , } and{ , , , , }, where superscripts 1 to 5 represent different feature levels from shallow to deep. In practical applications, feature levels... =1,2,..., N, N The settings are unlimited.

[0040] The bidirectional semantic alignment module is key to solving the problems of insufficient feature fusion and pseudo-changes. The bidirectional semantic alignment module is used to perform bidirectional spatiotemporal context fusion of feature maps of the same level in multi-scale feature map groups of two temporal images to obtain fused feature maps.

[0041] In one embodiment, the bidirectional semantic alignment module performs a first-order semantic alignment on the input two-temporal images. First phase feature map of the hierarchy Second phase feature map Perform the following operations:

[0042] For the first of the two-phase images First phase feature map of the hierarchy Second phase feature map , =1,2,..., N ;

[0043] Generate the first phase feature map respectively The first query feature and the first key value feature, and the second temporal feature map. The second query feature and the second key-value feature. Specifically, for example, the first phase feature map Second phase feature map The features are respectively input into at least one convolutional layer for processing to obtain the first query feature and the first key value feature of the first temporal feature map, and the second query feature and the second key value feature of the second temporal feature map.

[0044] Based on the first query feature and the second key-value feature, calculate the forward attention weights that represent the dependency relationship from the first time phase to the second time phase. Based on the second query features and the first key-value features, calculate the backward attention weights that represent the dependency relationship from the second time phase to the first time phase. ;

[0045] ;

[0046] ;

[0047] in, Represents forward attention weights The weight in the p-th row and q-th column represents the temporal image. Features and temporal images of position p The similarity of features at position q in the middle; Represents the backward attention weights The weight of the p-th row and q-th column; · represents the dot product; , These represent the p-th values ​​of the first query feature and the second query feature, respectively. , Let q represent the q-th value of the first key-value feature and the second key-value feature, respectively. This represents the predicted value of the binary change during forward propagation.

[0048] Utilizing forward attention weights For the first phase feature map Weighting is performed to obtain the first enhanced feature map enhanced by the second phase information. ;

[0049] Using backward attention weights For the second phase feature map Weighting is performed to obtain the second enhanced feature map enhanced by the first phase information. ;

[0050] ;

[0051] ;

[0052] The first enhanced feature map Second Enhanced Feature Map First phase characteristic diagram Second phase feature map The images are stitched together and fused using a 1x1 convolutional layer to obtain the first image from the two temporal phases. Hierarchical fusion feature map ;

[0053] ;

[0054] The bidirectional semantic alignment module can dynamically establish the correspondence between two temporal images by calculating bidirectional, dense pixel-level correlations, thereby enhancing areas of real change and suppressing pseudo-changes caused by appearance differences during fusion.

[0055] The decoder module is responsible for restoring the low-resolution, high-semantic fused features of the encoded output to a high-resolution variation map.

[0056] In one embodiment, the decoder module starts from the first... Hierarchical fusion feature map Initially, spatial details are recovered from deep to shallow layers to generate an initial probability map of change. For the initial change probability map Threshold segmentation is performed to obtain a binary change prediction map. ,include:

[0057] S101, No. Hierarchical fusion feature map As the current level input features of the decoder module In the first iteration, the current level = N；

[0058] S102, input features of the current level Perform convolution and upsampling; for example, use a 3x3 convolution to reduce the number of channels and extract features, and then perform bilinear interpolation upsampling to increase the spatial size by 2 times.

[0059] S103, For the current fused feature map at the same level Perform global average pooling to generate channel attention weights, and simultaneously fuse the current feature maps at the same level. Spatial attention weights are computed through a convolutional layer, and the current fused feature map at the same level is then analyzed based on channel attention weights and spatial attention weights. We perform weighting to obtain the weighted skip connection features. This allows the decoder to focus more on the fused feature maps at the same level. The channels and locations that contain important information about changes and the edges of the target object.

[0060] S104, weighted skip connection features The features are added element-wise to the upsampled features in S102, and then fused through a residual convolutional block to output the decoded features of the current stage. And use it as the input feature for the next level. Update the current level Return to step S102 until all levels of fused feature maps have been processed, ultimately obtaining multiple decoded features at different scales. , , ,..., };

[0061] S105 upsamples decoding features at different scales to the same spatial size and then stitches them together;

[0062] S106, the concatenated features are sequentially subjected to convolutional fusion, channel dimensionality reduction, and nonlinear activation to output the initial change probability map. If the concatenated features are fused using a 3x3 convolutional layer, followed by a 1x1 convolution and a sigmoid activation function, the initial change probability map will be output. This design ensures that the model can utilize both deep semantic information (which is helpful for identifying changes in large areas) and shallow detailed information (which is helpful for accurately locating small targets and boundaries).

[0063] S107, Initial Change Probability Map Threshold segmentation is performed to obtain a binary change prediction map. In this context, a pixel value of 1 represents a change, and 0 represents no change. For example, consider the initial change probability map. The binary change prediction map is obtained by comparing it with the threshold T (default is 0.5). .

[0064] To address class imbalance (where changed pixels are typically far fewer than unchanged pixels) and improve boundary quality, a composite loss function is designed in one embodiment of this invention. The loss function... Dice loss term Weighted binary cross-entropy loss term Edge consistency loss term constitute, , where α, β∈(1,0), α + β=1.

[0065] Specifically, Dice loss item This formula is used to address the issue of unbalanced foreground / background pixel counts.

[0066] ;

[0067] in, A binary transformation annotation map for the training samples. This is a binary change prediction map for the training samples, where ε is the smoothing term;

[0068] Weighted binary cross-entropy loss term This provides a foundation for mining difficult samples, and its formula is:

[0069] ;

[0070] in, It is a weight that is dynamically calculated based on the ratio of positive to negative samples in the training dataset. ∈ (1, 0), used to increase the loss contribution of pixels of the change class.

[0071] Using the Sobel operator to annotate the binary transformation of the training samples Binary change prediction chart The edges extracted from each are used to calculate the edge consistency loss term between them. , The edge consistency loss term forces the model to learn clear object boundaries, which aligns the predicted boundaries of the change region as closely as possible to the true boundaries.

[0072] During training, in each batch, a binary transformation prediction map is generated based on the training samples. The model calculates the loss for each sample and automatically identifies difficult samples that are misclassified by the model (i.e., high confidence but incorrect prediction). During backpropagation, gradients are backpropagated only to the top K% (e.g., 30%) of samples with the highest loss values, forcing the model to focus on optimizing these difficult samples and improving its discriminative ability.

[0073] For a dual-temporal remote sensing image registered with the target object to be detected, inputting it into a trained target object change detection model will yield its binary change prediction map, thus completing the target object change detection. Specifically, the trained target object change detection model performs the following steps:

[0074] Multi-scale feature extraction is performed on the two temporal images in the dual-temporal remote sensing image pair to generate multi-scale feature map groups corresponding to the two temporal images. Each multi-scale feature map group contains feature maps of different levels from low to high.

[0075] By performing bidirectional spatiotemporal context fusion on feature maps of the same level from two temporal images, fused feature map groups at each level are obtained.

[0076] Spatial details are recovered from the fused feature maps of each level through upsampling, cross-layer connections, and feature aggregation operations to generate an initial change probability map;

[0077] The initial change probability map is binarized to generate a binary change prediction map.

[0078] Furthermore, it also includes prediction maps of binary changes. First, an opening operation is performed to remove noise points and small areas of spurious change. Then, a closing operation is performed to fill small holes within the change areas, making the detected target area more complete. Finally, area filtering is performed: all connected regions with an area smaller than a preset threshold are removed. (For example, a patch corresponding to an actual ground area of ​​30 square meters) is further filtered to remove minor changes that are not the target object, and the final smooth and complete binary change prediction map of the target object is output.

[0079] In another embodiment, a target change detection device based on deep learning is provided, comprising:

[0080] The first module is used to acquire a training dataset, wherein the training samples in the training dataset include dual-temporal remote sensing image pairs registered with the target object and their corresponding pixel-level binary change annotation maps.

[0081] The second module is used to build a target change detection model. The model includes a feature extraction module for extracting multi-scale features of the two temporal images in the mid-phase remote sensing image pair, a bidirectional semantic alignment module for fusing multi-scale features of the two temporal images, and a decoder module for restoring spatial details.

[0082] The third module is used to train the target object change detection model using the training dataset. During the training process, the constructed loss function is used to optimize the target object change detection model to obtain a well-trained target object change detection model.

[0083] The fourth module is used to input the dual-temporal remote sensing image pairs registered with the target object to be detected into the trained target object change detection model, and output the corresponding binary change prediction map.

[0084] The object change detection method and apparatus based on deep learning provided by this invention achieves significant technical progress through a series of innovative designs, including the introduction of a bidirectional semantic alignment module, a multi-scale detail recovery decoder, and an edge enhancement composite loss function. The beneficial effects are specifically reflected in the following aspects:

[0085] (1) Significantly improved the accuracy and robustness of change detection:

[0086] The core of this invention, the bidirectional semantic alignment module, achieves deep-level spatiotemporal feature interaction and adaptive fusion by calculating dense, bidirectional pixel-level correlations between two temporal images. Compared with traditional simple feature stitching or difference methods, this module can more accurately capture the semantic information of real building changes, while effectively suppressing "spurious change" interference caused by changes in lighting, seasonal changes, vehicle movement, and vegetation growth. Experimental results show that on public datasets, the algorithm of this invention significantly outperforms mainstream methods such as FC-EF, STANet, and BIT in key metrics such as F1 score and intersection-over-union ratio, especially in complex urban scenes and cross-seasonal data, where the false alarm rate is significantly reduced.

[0087] (2) Enhanced the ability to perform detailed inspections of buildings at multiple scales:

[0088] This invention designs a multi-scale detail recovery decoder that combines a feature pyramid structure with a residual attention upsampling module. This structure can effectively fuse features from different levels of the encoder, making full use of the high-resolution detail information of shallow networks and the rich semantic information of deep networks. This enables the invention to simultaneously and accurately detect the complete outline of large-scale buildings and the fine structure of small-scale buildings, effectively solving the common problems of missed detection of small targets and fragmented boundaries of large targets in traditional methods. The final output transformation map has more complete and clearer building boundaries.

[0089] (3) It reduces the dependence on a large amount of finely labeled data and improves the generalization ability of the algorithm:

[0090] This invention employs an edge-enhancing composite loss function (combining Dice loss, weighted cross-entropy, and edge consistency loss) with an online hard sample mining strategy to optimize the model's training process. This loss function, by emphasizing boundary accuracy and dynamically adjusting sample weights, guides the model to learn more discriminative features, thus maintaining high performance even with limited labeled data. Furthermore, the BSA module learns a general spatiotemporal relationship modeling capability, rather than overfitting to the apparent features of a specific dataset. This makes the algorithm of this invention exhibit stronger adaptability and generalization ability when facing new geographical regions or image data from different sensors.

[0091] (4) It provides an end-to-end, high-performance automation solution:

[0092] This invention integrates complex feature fusion, multi-scale processing, and fine segmentation into a unified deep learning network, achieving end-to-end automated processing from raw dual-temporal image input to final binary change map output. The entire process requires no complex manual intervention or tedious feature engineering, significantly improving change detection efficiency. It can meet the needs of smart city management, disaster emergency response, and other application scenarios for rapid and automated analysis of large-scale low-altitude image data. Its modular design also allows for easy deployment on various cloud or edge computing platforms, possessing high practical value and promising industrialization prospects.

[0093] In summary, this invention, through multiple innovations at the algorithm level, effectively overcomes the technical bottlenecks of existing deep learning change detection methods in terms of feature fusion, pseudo-change suppression, multi-scale detection, and generalization ability, and provides a building (structure) change detection scheme with higher accuracy, stronger robustness, and better practicality.

[0094] Figure 2 The figure shows a block diagram of an electronic device according to an embodiment, such as Figure 2 As shown, the electronic device includes one or more processors and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the deep learning-based target change detection method provided in any of the above embodiments. The processor may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions. The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0095] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0096] Of course, for the sake of simplicity, Figure 2 Only some of the components of the electronic device relevant to this application are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0097] Embodiments of the present invention may also be computer-readable storage media storing a computer program thereon, which, when executed by a processor, implements the steps of the deep learning-based target change detection method provided in any of the above embodiments. The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0098] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not restrict the application from being implemented using the specific details described above.

[0099] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0100] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0101] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application should not be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0102] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A target change detection method based on deep learning, characterized in that, Includes the following steps: Obtain a training dataset, wherein the training samples in the training dataset include dual-temporal remote sensing image pairs registered with the target object and their corresponding pixel-level binary variation annotation maps; A target change detection model is constructed, comprising a feature extraction module for extracting multi-scale features from two temporal remote sensing images, a bidirectional semantic alignment module for fusing multi-scale features from two temporal images, and a decoder module for restoring spatial details. The bidirectional semantic alignment module performs bidirectional spatiotemporal context fusion of feature maps of the same level in the multi-scale feature map group of the two temporal images to obtain the first feature map from the two temporal images. Hierarchical fusion feature map ,include: For the first of the two-phase images First phase feature map of the hierarchy Second phase feature map , =1,2,..., N ; Generate the first phase feature map respectively The first query feature and the first key value feature, and the second temporal feature map. The second query feature and the second key value feature; Based on the first query feature and the second key-value feature, calculate the forward attention weights that represent the dependency relationship from the first time phase to the second time phase. Based on the second query features and the first key-value features, calculate the reverse attention weights that represent the dependency relationship from the second time phase to the first time phase. ; in, Represents forward attention weights The weight in the p-th row and q-th column represents the temporal image. Features and temporal images of position p The similarity of features at position q in the middle; Represents the backward attention weights The weight of the p-th row and q-th column; · represents the dot product; , These represent the p-th values ​​of the first query feature and the second query feature, respectively. Let q represent the q-th value of the first key-value feature and the second key-value feature, respectively. This represents the predicted value of the binary change during forward propagation; Utilizing forward attention weights For the first phase feature map Weighting is performed to obtain the first enhanced feature map enhanced by the second phase information. ; Using backward attention weights For the second phase feature map Weighting is performed to obtain the second enhanced feature map enhanced by the first phase information. ; The first enhanced feature map Second Enhanced Feature Map First phase characteristic diagram Second phase feature map By stitching and merging the two temporal images, the first phase image is obtained. Hierarchical fusion feature map ; The decoder module from the first Hierarchical fusion feature map Initially, spatial details are recovered from deep to shallow layers to generate an initial change probability map. This initial change probability map is then thresholded to obtain a binary change prediction map, including: S101, No. Hierarchical fusion feature map As the current level input features of the decoder module In the first iteration, the current level = N； S102, input features of the current level Perform convolution and upsampling; S103, For the current fused feature map at the same level Perform global average pooling to generate channel attention weights, and simultaneously fuse the current feature maps at the same level. Spatial attention weights are computed through a convolutional layer, and the current fused feature map at the same level is then analyzed based on channel attention weights and spatial attention weights. We perform weighting to obtain the weighted skip connection features. ; S104, weighted skip connection features The features are added element-wise to the upsampled features in S102, and then fused through a residual convolutional block to output the decoded features of the current stage. And use it as the input feature for the next level. Update the current level Return to step S102 until all levels of fused feature maps have been processed, ultimately obtaining multiple decoded features at different scales. , , ,..., }; S105 upsamples decoding features at different scales to the same spatial size and then stitches them together; S106, the concatenated features are sequentially subjected to convolutional fusion, channel dimensionality reduction and nonlinear activation, and the initial change probability map is output; S107, Threshold segmentation is performed on the initial change probability map to obtain a binary change prediction map; A target object change detection model is trained using a training dataset. During training, the model is optimized using a constructed loss function to obtain a well-trained target object change detection model. The loss function... Dice loss term Weighted binary cross-entropy loss term Edge consistency loss term constitute, β , where α, β∈(1,0), α + β=1; The dual-temporal remote sensing image pairs registered with the target object to be detected are input into the trained target object change detection model, and the corresponding binary change prediction map is output.

2. The target change detection method based on deep learning according to claim 1, characterized in that, The target object is a building or structure.

3. The target change detection method based on deep learning according to claim 1 or 2, characterized in that, The dual-temporal remote sensing image pairs registered with the target object are input into the feature extraction module. The feature extraction module includes two feature extraction backbone networks with shared weights. The two temporal images in the dual-temporal remote sensing image pairs are respectively input into the two feature extraction backbone networks with shared weights for multi-scale feature extraction, generating multi-scale feature map groups corresponding to the two temporal images. Each multi-scale feature map group contains feature maps of different levels from shallow to deep.

4. The target change detection method based on deep learning according to claim 1, characterized in that, Dice Loss Items The formula is: in, A binary transformation annotation map for the training samples. The binary change prediction plot for the training samples is given, where ε is the smoothing term. Weighted binary cross-entropy loss term The formula is: in, It is a weight that is dynamically calculated based on the ratio of positive to negative samples in the training dataset. ∈ (1, 0); Using the Sobel operator to annotate the binary transformation of the training samples Binary change prediction chart The edges extracted from each are used to calculate the edge consistency loss term between them. , .

5. A target change detection device based on deep learning, used to implement the target change detection method based on deep learning as described in claim 1, characterized in that, include: The first module is used to acquire a training dataset, wherein the training samples in the training dataset include dual-temporal remote sensing image pairs registered with the target object and their corresponding pixel-level binary change annotation maps. The second module is used to build a target change detection model. The model includes a feature extraction module for extracting multi-scale features of the two temporal images in the mid-phase remote sensing image pair, a bidirectional semantic alignment module for fusing multi-scale features of the two temporal images, and a decoder module for restoring spatial details. The third module is used to train the target object change detection model using the training dataset. During the training process, the constructed loss function is used to optimize the target object change detection model to obtain a well-trained target object change detection model. The fourth module is used to input the dual-temporal remote sensing image pairs registered with the target object to be detected into the trained target object change detection model, and output the corresponding binary change prediction map.

6. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the target change detection method based on deep learning as described in claim 1.

7. A computer storage medium storing computer-executable instructions thereon, characterized in that: When the computer-executable instructions are executed by the processor, they implement the steps of the deep learning-based target change detection method as described in claim 1.

Images