Remote sensing image change detection method of affected area and storage medium thereof

Abstract

The invention provides a remote sensing image change detection method for a disaster area and a storage medium thereof, and the method comprises the steps: obtaining a pre-disaster image and a post-disaster image of the disaster area, generating an image detection training model based on a visual language architecture, and carrying out the parameter fine adjustment of the image detection training model based on an LoRA technology and an AdaLN technology, thereby achieving the remote sensing image change detection of the disaster area. And finally, through the image detection target model, carrying out image change detection on the pre-disaster image and the post-disaster image to obtain a change mask image of the affected area, so that efficient fine adjustment of model parameters can be realized in a mode of training a low-rank matrix by introducing a LoRA module into the model, and meanwhile, the accuracy of the model is improved. In addition, the feature scale of the model is effectively controlled by introducing an AdaLN module, the stability of fine tuning training of the model is remarkably improved, the gradient explosion or disappearance phenomenon of the model is prevented, and then the generalization ability and reasoning efficiency of the model in the post-disaster change detection task can be remarkably improved under the condition of a very small number of labeled samples.

Description

Technical Field

[0001] The present application relates to the field of intelligent processing of remote sensing images, and in particular to a method for detecting changes in remote sensing images of disaster-stricken areas and a storage medium thereof. Background Art

[0002] When disaster strikes, existing technologies typically use drone oblique photography or satellite multispectral imaging systems to obtain high-resolution orthophotos of the same geographic area before and after the disaster. These images are then used to identify areas of disaster change using large-scale image recognition models. However, actual data collection faces three core challenges: First, cloud cover and sensor parameter drift in disaster response scenarios can lead to radiometric differences and geometric misalignment in multi-temporal images, which can lead to biased model detection of post-disaster changes. Second, labeled data in disaster-affected areas is extremely scarce, requiring the establishment of weak supervision relationships using a small number of manually labeled image pairs (typically only tens to hundreds). This increases the data processing time of the image recognition model to accurately detect images. Third, the 72-hour post-disaster rescue window requires rapid training iterations using limited data. However, existing image recognition models require high data processing capacity, making it difficult to deploy image recognition models in disaster-affected areas for real-time analysis. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a remote sensing image change detection method for disaster-stricken areas and its storage medium, which can significantly improve the model generalization ability and reasoning efficiency in post-disaster change detection tasks under the condition of a very small number of labeled samples, improve the model's emergency deployment capability for cross-regional remote sensing images, and improve the model's real-time analysis of changes in disaster-stricken areas.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present application provides a method for detecting changes in remote sensing images of a disaster-stricken area, comprising: Obtain pre-disaster and post-disaster images of the affected area, as well as an image detection training model generated based on a visual language architecture; Based on LoRA technology and AdaLN technology, the parameters of the image detection training model are fine-tuned to obtain the image detection target model; Image change detection is performed on pre-disaster and post-disaster images using an image detection target model to obtain a change mask image of the disaster-stricken area, where each pixel in the change mask image is used to indicate whether a semantic change has occurred.

[0005] Furthermore, in some embodiments, image change detection is performed on pre-disaster images and post-disaster images using an image detection target model to obtain a change mask image of the disaster-stricken area, including: Inputting the pre-disaster image and the post-disaster image into the image encoder of the image detection target model to extract features of the pre-disaster image and the post-disaster image to obtain a pre-disaster feature image and a post-disaster feature image; Through the difference modeling module of the image detection target model, multi-dimensional difference modeling is performed on the pre-disaster feature image and the post-disaster feature image to obtain the difference image; Determine the voice prompt template based on the detection task scenario in the disaster-stricken area; Input the language prompt template into the text encoder of the image detection target model to extract multi-scale semantic embedding of the speech prompt template and obtain the semantics to be embedded; Through the multi-scale decoder of the image detection target model, the difference image and the semantics to be embedded are cross-modally fused to obtain the change mask image.

[0006] Furthermore, in some embodiments, based on the LoRA technology and the AdaLN technology, the image detection training model is fine-tuned to obtain an image detection target model, including: Determine a target layer containing a Transformer architecture in the image encoder backbone and the text encoder backbone, and determine a module region containing a multi-head attention module in the target layer; Insert the dimension reduction matrix of the LoRA module on the input side of the linear mapping in the module area, and insert the dimension increase matrix of the LoRA module on the output side of the linear mapping in the module area; The AdaLN module is introduced at the input side of the dimension-raising matrix to perform adaptive normalization adjustment on the parameters input to the dimension-raising matrix. A constraint loss function is added on the output side of the LoRA module to reparameterize the output parameters of the LoRA module; Get multiple sample images for fine-tuning the model and freeze the backbone network of the image encoder and the backbone network of the text encoder; Each sample image is input into the image detection training model to iteratively adjust the parameters of the image detection training model to obtain the image detection target model.

[0007] Furthermore, in some embodiments, each sample image is input into an image detection training model to iteratively adjust parameters of the image detection training model to obtain an image detection target model, including: Input each sample image into the input layer of the image detection training model so that the sample image is forward propagated to the image encoder to obtain multiple feature sample images; Input each characteristic sample image into the difference modeling module to perform multi-dimensional difference modeling to obtain multiple difference sample images between the sample images; Input multiple difference sample images and the semantics to be embedded into the multi-scale decoder for cross-modal fusion to obtain multiple change mask sample images; The loss function is used to calculate the difference between each change mask sample image and the corresponding sample image to obtain multiple loss values; Based on multiple loss values, backpropagation updates are performed on the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder using the Adam optimization algorithm. If the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder meet the preset parameter adjustment conditions, the iterative parameter adjustment of the image detection training model is terminated to obtain the image detection target model.

[0008] Furthermore, in some embodiments, each sample image is input into an image detection training model to iteratively adjust parameters of the image detection training model to obtain an image detection target model, including: Input each sample image into the input layer of the image detection training model so that the sample image is forward propagated to the image encoder to obtain multiple feature sample images; Input each characteristic sample image into the difference modeling module to perform multi-dimensional difference modeling to obtain multiple difference sample images between the sample images; Input multiple difference sample images and the semantics to be embedded into the multi-scale decoder for cross-modal fusion to obtain multiple change mask sample images; The loss function is used to calculate the difference between each change mask sample image and the corresponding sample image to obtain multiple loss values; Based on multiple loss values, backpropagation updates are performed on the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder using the Adam optimization algorithm. If the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder meet the preset parameter adjustment conditions, the iterative parameter adjustment of the image detection training model is terminated to obtain the image detection target model.

[0009] Furthermore, in some embodiments, the image encoder includes a first image encoder and a second image encoder, and the first image encoder and the second image encoder share weights. The pre-disaster image and the post-disaster image are input into the image encoder of the image detection target model to extract features of the pre-disaster image and the post-disaster image to obtain the pre-disaster feature image and the post-disaster feature image, including: Inputting the pre-disaster image into a first image encoder to extract semantic features to obtain a pre-disaster feature image; The post-disaster image is input into the second image encoder for semantic feature extraction to obtain a post-disaster feature image.

[0010] Furthermore, in some embodiments, the pre-disaster feature image includes a plurality of pre-disaster semantic feature images of different scales, and the post-disaster feature image includes a plurality of post-disaster semantic feature images of different scales, and the scales of each pre-disaster semantic feature image and each post-disaster semantic feature image correspond one to one, and the difference modeling module includes a first differential path, a second differential path, a third differential path, and a channel attention module, and the channel attention module is communicatively connected to the first differential path, the second differential path, and the third differential path, respectively; The method includes the following steps: performing multi-dimensional difference modeling on the pre-disaster feature image and the post-disaster feature image through the difference modeling module of the image detection target model to obtain a difference image. Inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into a first differential path to perform feature element subtraction processing to obtain a first feature difference image; Inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into the second difference path to perform cosine similarity measurement to obtain a second feature difference image; Inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into the third differential path for feature channel dimension splicing to obtain a third feature difference image; Inputting the first feature difference image, the second feature difference image, and the third feature difference image into the channel attention module for weighted fusion to obtain feature difference images between pre-disaster images and post-disaster images at the same scale; The feature difference images of different scales are superimposed and output through the output layer of the difference modeling module to obtain a difference image.

[0011] Furthermore, in some embodiments, the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale are input into a first differential path to perform feature element subtraction processing to obtain a first feature difference image, including: Subtracting the feature elements of the pre-disaster semantic feature image from the feature elements of the post-disaster semantic feature image with the same dimensions to obtain a first feature difference image; Taking the absolute value of each feature difference in the first feature difference image to obtain a second feature difference image; Inputting the second feature difference image into the convolution layer of the first difference path to perform threshold capture on the structural changes of the second feature difference image through a nonlinear activation function to obtain a first feature difference image; The first feature difference image is used to determine a boundary area or a structural difference area between a pre-disaster image and a post-disaster image.

[0012] Furthermore, in some embodiments, the difference image includes a plurality of feature difference images at different scales, and the multi-scale decoder includes a picture-text fusion module, a hierarchical connection module, and a low-rank adaptation module, wherein the picture-text fusion module is communicatively connected to the hierarchical connection module, and the hierarchical connection module is communicatively connected to the low-rank adaptation module; The multi-scale decoder of the image detection target model is used to cross-modally fuse the difference image with the semantics to be embedded to obtain a change mask image of the affected area, including the following steps: Inputting the semantics to be embedded and each feature difference image into the image-text fusion module, so as to inject the semantics to be embedded into each feature difference image through the cross attention mechanism, and obtain multiple first image-text difference images of different scales; Inputting each image-text difference image into a hierarchical connection module to spatially connect each image-text difference image in a residual connection manner to obtain a multi-scale fused second image-text difference image; Inputting the second image-text difference image into a low-rank adaptation module to enhance the cross-modal fusion expression of the second image-text difference image between the image and the text, thereby obtaining a third image-text difference image; The spatial resolution of the third image-text difference image is reconstructed through the upsampling layer of the difference modeling module, and the reconstructed third image-text difference image is convolved and output through the convolution layer of the difference modeling module to obtain a change mask image, wherein the spatial resolution of the change mask image is consistent with the spatial resolution of the post-disaster image.

[0013] Furthermore, in some embodiments, the low-rank adaptation module includes a low-rank dimensionality reduction layer, a low-rank dimensionality increase layer, and an output constraint layer, the low-rank dimensionality reduction layer is communicatively connected to the low-rank dimensionality increase layer, and the low-rank dimensionality increase layer is communicatively connected to the output constraint layer; The second image-text difference image is input into the low-rank adaptation module to enhance the cross-modal fusion expression of the second image-text difference image between the image and the text, thereby obtaining a third image-text difference image, including the following steps: The second image-text difference image is input to the low-rank dimensionality reduction layer to map and project the image features and text vector of the second image-text difference image respectively to obtain an image mapping matrix, a first text mapping matrix, and a second text mapping matrix; Inputting the image mapping matrix and the first text mapping matrix into a low-rank dimension-raising layer for fusion and sharing, thereby obtaining a low-rank adjusted residual image of the second image-text difference image; The low-rank adjusted residual image is input into the output constraint layer, and the loss constraint of the low-rank adjusted residual image is reparameterized by the second text mapping matrix to obtain the third image-text difference image.

[0014] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes a computer-readable storage medium, which stores a program executable by a processor. When the computer program is executed by the processor, the remote sensing image change detection method of the above-mentioned first aspect embodiment is implemented.

[0015] The embodiments of the first aspect of the present application have the following beneficial effects: by obtaining pre-disaster images and post-disaster images of the disaster-stricken area, and an image detection training model generated based on a visual language architecture, and then based on the LoRA technology and AdaLN technology, the image detection training model is fine-tuned to obtain an image detection target model, and finally, through the image detection target model, image change detection is performed on the pre-disaster images and post-disaster images to obtain a change mask image of the disaster-stricken area, and then the model parameters can be efficiently fine-tuned by introducing the LoRA module into the model to train a low-rank matrix. At the same time, in order to alleviate the problems of output instability and feature drift in the LoRA module, the present application also effectively controls the feature scale of the model by introducing the AdaLN module, significantly improving the stability of the model fine-tuning training, and preventing the model from experiencing gradient explosion or disappearance. Therefore, under the condition of a very small number of labeled samples, the generalization ability and reasoning efficiency of the model in the post-disaster change detection task can be significantly improved, the emergency deployment capability of the model for cross-regional remote sensing images is improved, and the real-time performance of the model for change analysis in the disaster-stricken area is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an optional flow chart of the method for detecting changes in remote sensing images of disaster-stricken areas provided in an embodiment of the present application; Figure 2 This embodiment of the present application provides Figure 1 An optional flowchart of step S103; Figure 3 This embodiment of the present application provides Figure 1 An optional flowchart of step S102; Figure 4 This embodiment of the present application provides Figure 3 An optional flowchart of step S306; Figure 5 This embodiment of the present application provides Figure 2 An optional flowchart of step S201; Figure 6 This embodiment of the present application provides Figure 2 An optional flowchart of step S202; Figure 7 This embodiment of the present application provides Figure 6 An optional flowchart of step S601; Figure 8This embodiment of the present application provides Figure 2 An optional flowchart of step S205; Figure 9 This embodiment of the present application provides Figure 2 An optional visual flowchart of step S205; Figure 10 This embodiment of the present application provides Figure 8 An optional flowchart of step S803; Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0018] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0019] It should also be noted that, in the description of this application, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0021] In the description of this application, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0022] To facilitate understanding of the technical solutions provided by the embodiments of the present disclosure, some key terms used in the embodiments of the present disclosure are explained here: When disaster strikes, existing technologies typically use drone oblique photography or satellite multispectral imaging systems to obtain high-resolution orthophotos of the same geographic area before and after the disaster. These images are then used to identify areas of disaster change using large-scale image recognition models. However, actual data collection faces three core challenges: First, cloud cover and sensor parameter drift in disaster response scenarios can lead to radiometric differences and geometric misalignment in multi-temporal images, which can lead to biased model detection of post-disaster changes. Second, labeled data in disaster-affected areas is extremely scarce, requiring the establishment of weak supervision relationships using a small number of manually labeled image pairs (typically only tens to hundreds). This increases the data processing time of the image recognition model to accurately detect images. Third, the 72-hour post-disaster rescue window requires rapid training iterations using limited data. However, existing image recognition models require high data processing capacity, making it difficult to deploy image recognition models in disaster-affected areas for real-time analysis.

[0023] Based on this, the embodiment of the present application provides a remote sensing image change detection method for disaster-stricken areas and its storage medium, which can significantly improve the model generalization ability and reasoning efficiency in post-disaster change detection tasks under the condition of a very small number of labeled samples, improve the model's emergency deployment capability for cross-regional remote sensing images, and improve the model's real-time analysis of changes in disaster-stricken areas.

[0024] The embodiments of the present application provide a method for detecting changes in remote sensing images of disaster-stricken areas and a storage medium thereof, which are specifically described through the following embodiments.

[0025] First, refer to Figure 1 As shown, Figure 1 This is an optional flowchart of the method for detecting changes in remote sensing images of disaster-stricken areas provided in an embodiment of the present application. The method may include but is not limited to steps S101 to S103.

[0026] Step S101: obtaining pre-disaster images and post-disaster images of the disaster-stricken area, and an image detection training model generated based on a visual language architecture.

[0027] Among them, the image detection training model includes an image encoder, a text encoder, a difference modeling module and a multi-scale decoder. The image encoder is communicated with the difference modeling module, and the text encoder and the difference modeling module are both communicated with the multi-scale decoder. The image encoder is used to extract image features, the text encoder is used to extract the text semantics to be embedded, the difference modeling module is used to extract the difference features between two images, and the multi-scale decoder is used to cross-modally fuse image and text semantics.

[0028] It should be noted that in the task of remote sensing image change detection, the model needs to understand and compare the difference areas between pre-disaster and post-disaster images, and combine the task semantics to determine whether it is a "meaningful change." However, traditional image feature-based models have difficulty accurately identifying "what objects the changes occur on," such as buildings, roads, or vegetation. Therefore, this application introduces the architecture of a visual language model. By combining multimodal input construction with images and language prompts, it enhances the model's semantic recognition and language guidance capabilities, providing rich semantic information support for subsequent feature extraction and change judgment.

[0029] Step S102: Based on the LoRA technology and the AdaLN technology, fine-tune the parameters of the image detection training model to obtain an image detection target model.

[0030] Step S103: performing image change detection on the pre-disaster image and the post-disaster image using the image detection target model to obtain a change mask image of the disaster-stricken area.

[0031] Each pixel of the change mask image is used to indicate whether a semantic change occurs.

[0032] It should be noted that in steps S101 to S103, the LoRA module is introduced into the model to train the low-rank matrix to achieve efficient fine-tuning of the model parameters. At the same time, in order to alleviate the problems of output instability and feature drift in the LoRA module, the present application also introduces the AdaLN module to effectively control the feature scale of the model, significantly improving the stability of the model fine-tuning training and preventing the model from experiencing gradient explosion or disappearance. As a result, under the condition of a very small number of labeled samples, the generalization ability and reasoning efficiency of the model in the post-disaster change detection task can be significantly improved, the model's emergency deployment capability for cross-regional remote sensing images is improved, and the model's real-time analysis of changes in disaster-stricken areas is improved.

[0033] Reference Figure 2 As shown, Figure 2 This embodiment of the present application provides Figure 1 An optional flowchart of step S103 in the method may include but is not limited to steps S201 to S205.

[0034] Step S201: inputting the pre-disaster image and the post-disaster image into the image encoder of the image detection target model to perform feature extraction on the pre-disaster image and the post-disaster image to obtain a pre-disaster feature image and a post-disaster feature image.

[0035] In some embodiments, the input data consists of a pair of pre-disaster and post-disaster images, which can be sized to standard remote sensing image sizes (e.g., 256×256 or 512×512) and maintain consistent spatial resolution. The two images form a dual-branch input structure and are fed into a weight-shared image encoder for semantic feature extraction, resulting in pre-disaster and post-disaster feature images.

[0036] Step S202: Perform multi-dimensional difference modeling on the pre-disaster feature image and the post-disaster feature image through the difference modeling module of the image detection target model to obtain a difference image.

[0037] Specifically, to effectively model the regions of change in remote sensing imagery between pre- and post-disaster time points, this application employs a difference modeling module (DFC) to perform multi-dimensional difference modeling on the visual semantic features of bi-temporal imagery, constructing a highly expressive description of the regions of change. This module, combined with the multi-scale feature maps modulated by ADALNLORA in the application's backbone network, exploits differential responses while preserving semantic consistency.

[0038] This application outputs four layers of backbone visual features in the image encoder, which are recorded as follows: (1) in and Represents the pre-disaster image and post-disaster image in the Based on the characteristics of deep features having stronger semantic expression capabilities and shallow features having higher spatial resolution, the difference modeling module performs difference modeling and fusion on four groups of features at different scales to construct a multi-scale difference representation.

[0039] In order to fully explore the significant change features between pre-disaster and post-disaster remote sensing images, while taking into account the consistency of spatial position alignment and semantic categories, this application proposes a difference feature compensation mechanism. This mechanism is based on the multi-scale semantic features extracted by the aforementioned image encoder module, and extracts semantic feature maps of four stages for pre-disaster images and post-disaster images, respectively, denoted as and , where subscripts a and b denote pre-disaster and post-disaster images, respectively. This type of multi-scale feature captures both edge details and overall semantic distribution of features, playing an important role in accurately locating the affected area.

[0040] Step S203: Determine a voice prompt template according to the detection task scenario of the disaster-stricken area.

[0041] Specifically, this application identifies a set of voice prompt templates (PromptTemplate) based on the disaster area detection task scenario. These voice prompt templates are used to construct language input. Typical template forms include: Foreground Prompt: such as "{target category} in remote sensing image"; Background Prompt: such as "Natural area in image" or "Unchanged area".

[0042] In some possible embodiments, the aforementioned target categories can be preset to common semantic labels for remote sensing features, such as "buildings," "roads," "vegetation," and "water bodies," depending on the task, and can be flexibly expanded. Foreground and background cues together constitute a language description set that guides the model to focus on areas of change with practical semantics. For example, in the task of detecting building damage changes, the following example text can be used as input: Foreground: "building" in remote sensing images; Background: "background area" or "unchanged area" in remote sensing images.

[0043] Step S204: Input the language prompt template into the text encoder of the image detection target model to perform multi-scale semantic embedding extraction on the speech prompt template to obtain the semantics to be embedded.

[0044] Step S205: The difference image is cross-modally fused with the semantics to be embedded through the multi-scale decoder of the image detection target model to obtain a change mask image.

[0045] Reference Figure 3 As shown, Figure 3 This embodiment of the present application provides Figure 1 An optional flowchart of step S102 in the method may include but is not limited to steps S301 to S306.

[0046] Step S301: Determine a target layer containing a Transformer architecture in the backbone network of the image encoder and the backbone network of the text encoder, and determine a module area containing a multi-head attention module in the target layer.

[0047] Step S302: inserting the dimension reduction matrix of the LoRA module on the input side of the linear mapping in the module area, and inserting the dimension increase matrix of the LoRA module on the output side of the linear mapping in the module area.

[0048] It's important to note that the core concept of this application's LoRA module is to freeze the main structure of the pre-trained model and only add low-rank matrix adjustments at specific locations to achieve lightweight training. It is suitable for a variety of tasks, including remote sensing image classification, object detection, and change detection.

[0049] In this application, the LoRA module is inserted into the Query, Key, and Value (QKV) linear transformation positions of the multi-head attention module of the Transformer architecture. The original attention weight is , the input is the feature vector , then the conventional forward calculation is as follows: (2) After the introduction of the LoRA model, the above linear mapping is replaced by: (3) in. .

[0050] This structure is also applicable to K and V paths, that is, the linear mapping structure of the Key and Value paths is similar to the linear mapping structure of the above-mentioned Query path, and will not be repeated here. In addition, in actual applications, it can be selectively inserted according to resource conditions, and this application will not be specifically limited.

[0051] Step S303: introducing an AdaLN module at the input side of the dimension-raising matrix to perform adaptive normalization adjustment on the parameters input to the dimension-raising matrix.

[0052] In step S303, in the existing technology, low-rank adaptation technology (LoRA module) is widely used for efficient parameter fine-tuning. Its basic approach is to introduce a low-rank matrix pair (reduced dimension matrix A) and (increased dimension matrix B) on the basis of freezing the main parameters of the model, so as to adjust the model output by adding a low-rank residual AB. Although the traditional LoRA module has achieved good results in many natural image tasks, it has obvious limitations in small sample remote sensing application scenarios. First, the LoRA module reduces the dimension of the input value matrix A and obtains the feature Directly inputting the resulting matrix B without any normalization or stabilization can easily lead to feature scale drift during training. Furthermore, under extremely small sample sizes (such as in post-disaster remote sensing change detection with only a small number of labeled samples), the output features of traditional LoRa technology rely heavily on random initialization, leading to initial instability and large gradient fluctuations. This can easily lead to the model falling into local optima or training divergence. Furthermore, remote sensing data inherently suffers from multi-source heterogeneity and complex imaging conditions, exacerbating fluctuations in the distribution of small sample features and further amplifying the instability of traditional LoRa technology.

[0053] In response to the above problems, this application combines the needs of small sample remote sensing applications in actual disaster-stricken areas and proposes an improved low-rank adaptation method, which introduces adaptive normalization processing (called AdaLN module) into the LoRA module. Specifically, this application uses the feature of the original LoRA module after the dimension reduction matrix A is output. , does not directly enter the dimension-raising matrix B, but first passes through the AdaLN module, where the AdaLN module includes a small multi-layer perceptron (MLP) that is used to generate dynamic affine parameters, and then the dynamic affine parameters are used to adjust the features. Perform normalization adjustment. The output result of the normalization adjustment of the AdaLN module is processed by the following calculation formula: (4) in, and They are The mean and standard deviation of 、 Generated by the multi-layer perceptron (MLP) perceiving the weights of the model, 、 For features Perform adaptive scaling and translation. Then, complete the normalized features Then input it into the dimension-raising matrix B for fusion to form the final low-rank adjusted residual.

[0054] It should be noted that, through step S303 of this application, the overall LoRA path is transformed from the traditional becomes .

[0055] This application introduces the AdaLN module to significantly improve the training stability under small sample training conditions. At the same time, the features of the output of the dimensionality reduction matrix A are Normalization can effectively control the feature scale and prevent the model from experiencing gradient explosion or vanishing phenomena. The dynamic affine parameters generated by the multi-layer perceptron (MLP) further enhance the model's adaptability to different input distributions, making it particularly suitable for small-sample change detection tasks with complex land object categories and diverse disaster patterns in remote sensing images. In addition, in order to maintain training stability while not deviating from the original LoRA modeling path, this application also designs an additional constrained loss function in the LoRA module through step S304, so that the output parameters after processing by the AdaLN module are as close as possible to the output parameters of the unnormalized path in Euclidean space. Through this design, this application ensures that the LoRA module after the normalization has both distribution stability and does not excessively distort the original low-rank adjustment direction.

[0056] Overall, compared to the traditional LoRA method, the method combining Adaln and LoRA technologies proposed in this application has the following outstanding advantages: First, it significantly improves training stability and convergence speed in extremely small sample remote sensing tasks; second, it enhances the model's adaptability to different feature distributions, improving the robustness of tasks such as change detection; and third, by designing a reasonable constraint mechanism, it ensures training stability while maintaining the compactness and efficiency of the inference phase. This method is particularly suitable for remote sensing intelligent processing scenarios such as post-disaster emergency change detection and land use change monitoring, where samples are scarce but high precision is required.

[0057] Step S304: adding a constraint loss function on the output side of the LoRA module to reparameterize the output parameters of the LoRA module.

[0058] Specifically, in the AdalnLora method proposed in this application, in order to enhance training stability and introduce pre-training structure perception capabilities, a normalization module (AdaLN) is inserted between the dimensionality reduction matrix A and the dimensionality increase matrix B of the original LoRA module. The normalization parameters γ and β of this module are not the input parameters of the LoRA module. Instead, the structural parameters of the backbone network of the image detection training model are determined The input is dynamically generated into a small multi-layer perceptron (MLP), so that the normalization process has the ability to perceive the original model (i.e., the image detection training model) a priori.

[0059] However, since the nonlinear operation of the AdaLN module depends on the structural parameters , resulting in the residual term in the overall fine-tuning path no longer being a simple AB form (i.e., low-rank residual It is no longer AB), but becomes the result obtained by the following structural formula: (5) It should be noted that the result obtained by this structural formula will destroy the reparameterization property of the traditional LoRA module, that is, it is no longer possible to directly merge the trained dimensionality reduction matrix A and dimensionality increase matrix B back into the original weights during the inference phase. , the complete AdaLN module must be retained during inference, which increases the system complexity and deployment burden.

[0060] To address this issue, this application introduces an auxiliary training objective, which aims to make the output after AdaLN processing as close as possible to the standard output of the unprocessed path during the training phase, so that the approximate reduced-dimensionality matrix A and the approximate increased-dimensionality matrix B replace the path output of the AdaLN module during the inference phase. The constraint loss function is defined as follows: (6) By minimizing this error term during training, the model implicitly learns a "normalized output" equivalent to the reduced-dimensional matrix A and the increased-dimensional matrix B, thereby restoring the reparameterized form of the traditional LoRA module during the inference phase. The output parameters of the reparameterized LoRA module are as follows: (7) Although the nonlinear normalization module (AdaLN) is introduced during parameter adjustment and training in step S304 of this application, the loss term derived from the upper-constrained loss function allows the model to degenerate into an equivalent structure during inference, thus minimizing inference costs. This design is particularly suitable for practical scenarios such as remote sensing missions with small sample sizes, where deployment efficiency and structural stability are clearly required.

[0061] Step S305: Acquire multiple sample images for fine-tuning the model, and freeze the backbone network of the image encoder and the backbone network of the text encoder.

[0062] Step S306: Input each sample image into the image detection training model to iteratively adjust the parameters of the image detection training model to obtain the image detection target model.

[0063] It should be noted that in steps S301 to S306, to fully leverage the potential of large-scale image detection training models generated based on visual language architectures (such as CLIP and DINOv2) for remote sensing tasks, this application first freezes the main structure of the image detection training model (i.e., freezes the backbone network of the image encoder and the backbone network of the text encoder), and only inserts the LoRA module and AdaLN module at key locations for fine-tuning. The main reason for this strategy is that the image detection training model has already learned a wealth of common visual features from massive amounts of data and possesses strong feature extraction and generalization capabilities. Directly fine-tuning all parameters in small-sample remote sensing scenarios can easily lead to catastrophic forgetting, causing the original model to lose its original knowledge, and also resulting in severe overfitting due to insufficient sample size. Furthermore, large-scale models typically have a huge number of parameters (tens of millions to hundreds of millions), and fine-tuning the entire model is not only expensive in terms of training resources, but also takes a long time to train and is difficult to deploy. Therefore, this application freezes the main network of the model and only trains a very small number of new parameters (such as less than 1% of the total parameters). On the basis of maintaining the original capabilities, it achieves rapid adaptation to specific remote sensing tasks at a low cost.

[0064] In this way, on the one hand, the computational and storage overheads during the training process are significantly reduced, lowering the threshold for practical applications; on the other hand, since the main structure of the model is frozen, the model output is more stable during the parameter adjustment process, which is conducive to the convergence and robustness of the small sample learning process. At the same time, freezing the main structure also keeps the final model reasoning structure simple, which is convenient for rapid deployment in actual post-disaster emergency response, aerial monitoring and other scenarios with extremely high requirements for timeliness. This strategy of freezing the backbone and local adaptation takes into account multiple advantages such as maintaining the advantages of prior training, avoiding overfitting of small samples and improving training reasoning efficiency, and is the key basis for achieving the technical effects of this application.

[0065] Reference Figure 4 As shown, Figure 4 This embodiment of the present application provides Figure 3 An optional flowchart of step S306 in the method may include but is not limited to steps S401 to S406.

[0066] Step S401: Input each sample image into the input layer of the image detection training model so that the sample image is forward propagated to the image encoder to obtain multiple feature sample images.

[0067] Step S402: Input each characteristic sample image into a difference modeling module to perform multi-dimensional difference modeling to obtain a plurality of difference sample images between the sample images.

[0068] Step S403: inputting the multiple difference sample images and the semantics to be embedded into a multi-scale decoder for cross-modal fusion to obtain multiple change mask sample images.

[0069] Step S404: Calculate the difference between each change mask sample image and the corresponding sample image using a loss function to obtain multiple loss values.

[0070] Step S405: Based on the multiple loss values, backpropagation updates are performed on the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder based on the Adam optimization algorithm.

[0071] Step S406: If the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder both meet the preset parameter adjustment conditions, terminate the iterative parameter adjustment of the image detection training model to obtain the image detection target model.

[0072] It should be noted that, in steps S401 to S406, by inputting the sample image into the input layer of the image detection training model and forward propagating it to the image encoder to obtain multiple feature sample images, the features of the sample image can be effectively extracted, providing a basis for subsequent difference modeling and fusion. Each feature sample image is input into the difference modeling module for multi-dimensional difference modeling to obtain multiple difference sample images between the sample images, which helps to capture the differences and change information between the sample images and provides a key basis for accurately detecting targets in the image. The multiple difference sample images and the semantics to be embedded are then input into the multi-scale decoder for cross-modal fusion to obtain multiple change mask sample images, thereby realizing the fusion of multimodal information, making full use of image and semantic information, and improving the accuracy and robustness of the model for image detection. The difference value between each change mask sample image and the corresponding sample image is calculated through the loss function to obtain multiple loss values, which provides a clear direction and indicator for the optimization of model parameters. Based on multiple loss values, the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder are back-propagated and updated using the Adam optimization algorithm. This effectively adjusts the model parameters, allowing the model to be continuously optimized during training, improving the model's performance and generalization capabilities. Finally, when the non-backbone network parameters of the image encoder and the text encoder meet the preset parameter adjustment conditions, the iterative parameter adjustment of the image detection training model is terminated, and the image detection target model is obtained. This ensures that the training process has a clear termination condition, ensuring that the model stops training after reaching the predetermined performance indicators, avoiding overtraining or undertraining, and thus improving the practicality of the model.

[0073] In some embodiments, the image encoder includes a first image encoder and a second image encoder, and the first image encoder and the second image encoder share weights. Figure 5 As shown, Figure 5 This embodiment of the present application provides Figure 2 An optional flowchart of step S201 in the method may include but is not limited to steps S501 to S502.

[0074] Step S501: inputting the pre-disaster image into a first image encoder to extract semantic features to obtain a pre-disaster feature image.

[0075] Step S502: inputting the post-disaster image into a second image encoder to extract semantic features to obtain a post-disaster feature image.

[0076] It should be noted that the shared weights between the first image encoder and the second image encoder indicate that the two branches use the same set of parameters of the image encoder (such as ResNet, ViT). The dual-branch shared image encoder architecture of this application demonstrates high efficiency and robustness in image comparison tasks through parameter sharing and feature consistency constraints.

[0077] In some embodiments, the pre-disaster feature image includes a plurality of pre-disaster semantic feature images of different scales, and the post-disaster feature image includes a plurality of post-disaster semantic feature images of different scales, and the scales of each pre-disaster semantic feature image and each post-disaster semantic feature image correspond one to one. In some embodiments, considering that the change area behaves differently at different scales, using only deep or shallow features may result in information loss. The present application performs difference modeling on the feature maps of the dual-phase image at each scale, constructs three complementary difference paths, and the difference modeling module includes a first difference path, a second difference path, a third difference path and a channel attention module. The channel attention module is respectively connected to the first difference path, the second difference path and the third difference path. Figure 6 As shown, Figure 6 This embodiment of the present application provides Figure 2 An optional flowchart of step S202 in the method may include but is not limited to steps S601 to S605.

[0078] Step S601: inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into a first differential path to perform feature element subtraction processing to obtain a first feature difference image.

[0079] Step S602: inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into a second differential path to perform cosine similarity measurement to obtain a second feature difference image.

[0080] Specifically, this application introduces a cosine similarity path (i.e., the second differential path) to measure the degree of similarity between pre- and post-disaster features in directional space. Unlike changes in amplitude or intensity, subtle shifts in semantic levels often manifest as changes in vector direction. Therefore, calculating the cosine distance between pre- and post-disaster semantic feature images can more sensitively capture these semantic drifts. The calculation method is: (8) in, is the sigmoid function, is the difference mask from the cosine path, is the semantic feature image before the disaster at the same scale, is the semantic feature image before the disaster at the same scale, for A substitution parameter of the function, for A substitution parameter of the function, The function is expressed as , The function is to take Product and The maximum value between It is a preset threshold, which is used to prevent the denominator from being too small, resulting in unstable numerical calculations, or to adjust the similarity measure in certain specific scenarios.

[0081] Step S603: inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into a third differential path for feature channel dimension splicing to obtain a third feature difference image.

[0082] Specifically, this application directly inputs the pre-disaster semantic feature image and the post-disaster semantic feature image into the third differential path for channel dimension splicing. Then, the spliced ​​feature image is input into the Feature Pyramid Network (FPN) module in the third differential path for further fusion processing. The result of the third feature difference image is shown below: (9) in, is the semantic feature image before the disaster at the same scale, is the semantic feature image before the disaster at the same scale, is the channel dimension splicing function, It's important to note that the Feature Pyramid Network (FPN) module integrates cross-layer information, enhancing the contextual understanding of fused features and enabling the model to perceive differences by simultaneously considering both low-level textures and high-level semantics. Taking a simple FPN as an example, assume the input features after concatenation consist of three feature maps of different scales (obtained through downsampling). During FPN processing, high-level features (e.g., smaller feature maps after multiple convolutions and pooling) are fused with low-level features (larger feature maps) through operations such as upsampling. This fusion approach ensures that the final features contain both detailed spatial information (from low-level features) and rich semantic information (from high-level features), facilitating more accurate disaster analysis, such as more precisely locating disaster area boundaries and identifying disaster types.

[0083] Step S604: Input the first feature difference image, the second feature difference image, and the third feature difference image into the channel attention module for weighted fusion to obtain feature difference images between the pre-disaster image and the post-disaster image at the same scale.

[0084] Specifically, after obtaining the above three types of difference features (first feature difference image, second feature difference image, and third feature difference image), this application uses the channel attention mechanism to fuse and weight the three types of difference features to highlight the significant change areas and suppress invalid redundant information. The final fusion expression is as follows: (10) in, represents the final difference compensation output feature, is the difference mask from the cosine path, Used to guide information channel selection, is the eigenvalue of the first feature difference image. It should be noted that this channel structure can effectively guide the model to focus on the spatial regions where changes have actually occurred while preserving semantic category information. It is an important module for identifying subtle changes in remote sensing change detection.

[0085] Step S605: The feature difference images of different scales are superimposed and outputted through the output layer of the difference modeling module to obtain a difference image.

[0086] In steps S601 to S605, by constructing a multi-scale differential modeling process and combining the three strategies of explicit difference (first differential path), directional drift (second differential path) and context fusion (third differential path), this application significantly enhances the model's change perception ability in complex scenarios, and is particularly suitable for remote sensing mission scenarios with large disaster diversity and weak change signals.

[0087] Reference Figure 7 As shown, Figure 7 This embodiment of the present application provides Figure 6 An optional flowchart of step S601 in the method may include but is not limited to steps S701 to S703.

[0088] Step S701: subtracting feature elements of the pre-disaster semantic feature image from feature elements of the post-disaster semantic feature image corresponding to the same dimension to obtain a first feature difference image.

[0089] Step S702: taking the absolute value of each feature difference in the first feature difference image to obtain a second feature difference image.

[0090] Step S703: Input the second feature difference image into the convolution layer of the first differential path to perform threshold capture on the structural changes of the second feature difference image through a nonlinear activation function to obtain a first feature difference image.

[0091] The first feature difference image is used to determine a boundary area or a structural difference area between a pre-disaster image and a post-disaster image.

[0092] In steps S701 to S703, the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale are subtracted one by one through the first difference path, and the absolute value is taken. A nonlinear activation function is then applied to capture the most direct structural change response. The calculation result of the first feature difference image is: (11) in, is the convolution module, is the semantic feature image before the disaster at the same scale, This is a pre-disaster semantic feature image at the same scale. This first differential path can efficiently extract boundary regions or obvious structural differences, and is particularly effective in typical disaster scenarios such as collapsed buildings and road interruptions.

[0093] After completing differential feature compensation through the difference modeling module, this application designs a visual language-driven multi-scale decoder structure to further fuse the extracted significant change information with language semantic embedding to generate the final change mask. This module focuses on image-text collaborative enhancement and effectively combines low-level spatial resolution with high-level semantic representation through a layer-by-layer decoding fusion strategy, thereby improving the model's performance in terms of boundary clarity and category consistency.

[0094] In some embodiments, the difference image includes a plurality of feature difference images of different scales, and the multi-scale decoder includes a picture-text fusion module, a hierarchical connection module, and a low-rank adaptation module. The picture-text fusion module is communicatively connected to the hierarchical connection module, and the hierarchical connection module is communicatively connected to the low-rank adaptation module. Figure 8 and 9 As shown, Figure 8 This embodiment of the present application provides Figure 2 An optional flow chart of step S205 in FIG. Figure 9 As shown, Figure 9 This embodiment of the present application provides Figure 2 An optional visual flowchart of step S205 in the method may include but is not limited to steps S801 to S804.

[0095] Step S801: Input the semantics to be embedded and each feature difference image into the image-text fusion module to inject the semantics to be embedded into each feature difference image through the cross-attention mechanism to obtain multiple first image-text difference images of different scales.

[0096] Specifically, the present application decodes by fusing the multi-scale structure layer by layer through the image-text fusion module. The input image is the four-layer feature difference image output by the difference modeling module. , corresponding to different scale levels of pre-disaster images and post-disaster images respectively. At the same time, in order to ensure the effective transmission of difference information, the embedded semantics T output by the text encoder is introduced. The embedded semantics are derived from the above-mentioned language prompt template. The embedded semantics T contains the global understanding ability of the image change semantics. The embedded semantics T is combined with the four-layer feature difference image Joint interaction can effectively enhance the model's ability to discriminate "what category the change occurs in".

[0097] Step S802: Input each image-text difference image into a hierarchical connection module to perform spatial hierarchical connection on each image-text difference image in a residual connection manner to obtain a multi-scale fused second image-text difference image.

[0098] Specifically, the multi-scale decoder uses a hierarchical connection mechanism to connect the image after the image and text fusion at each level using a residual connection. The specific residual hierarchical connection method is as follows: (12) in, The image-text fusion module is implemented using a lightweight cross-attention mechanism to inject the semantics to be embedded, T, into each feature of the feature difference image at each layer. Through the layer-by-layer propagation in step S802 and the fusion structure in step S801, the model can gradually recover the fine edges, spatial layout, and category consistency of the changed regions in the difference image.

[0099] Step S803: inputting the second image-text difference image into a low-rank adaptation module to enhance the cross-modal fusion expression of the second image-text difference image between image and text, thereby obtaining a third image-text difference image.

[0100] Step S804: reconstruct the spatial resolution of the third image-text difference image through the upsampling layer of the difference modeling module, and convolve the reconstructed third image-text difference image through the convolution layer of the difference modeling module to obtain a change mask image.

[0101] Among them, the spatial resolution of the change mask image is consistent with the spatial resolution of the post-disaster image.

[0102] Specifically, the third image-text difference image is restored to the same spatial resolution as the post-disaster image through an upsampling layer, and then output through a convolutional layer to create a final change mask. This entire process ensures that change detection results not only have pixel-level boundary accuracy but also retain language-guided semantic consistency. This allows the model to identify semantically relevant change types, such as "whether a building has collapsed" or "whether a body of water has expanded," rather than simply segmenting changes at the structural level.

[0103] In summary, in steps S801 to S804, the visual language-driven multi-scale decoding module not only improves the interpretability of the changed area by introducing semantic guidance (step S801), hierarchical interaction fusion (step S802) and cross-modal attention mechanism (step S803), but also enhances the model's adaptability to the semantics of different scene tasks, providing key technical support for realizing remote sensing intelligent change detection.

[0104] In some embodiments, the low-rank adaptation module includes a low-rank dimensionality reduction layer, a low-rank dimensionality increase layer, and an output constraint layer, wherein the low-rank dimensionality reduction layer is communicatively connected to the low-rank dimensionality increase layer, and the low-rank dimensionality increase layer is communicatively connected to the output constraint layer. Figure 10 As shown, Figure 10 This embodiment of the present application provides Figure 8 An optional flowchart of step S803 in the method may include but is not limited to steps S901 to S903.

[0105] Step S901: The second image-text difference image is input to the low-rank dimensionality reduction layer to map and project the image features and text vectors of the second image-text difference image respectively to obtain an image mapping matrix, a first text mapping matrix and a second text mapping matrix.

[0106] Specifically, the image features and text vectors of the second image-text difference image are mapped and projected (three linear positions of Query, Key, and Value) respectively, and calculated as follows: , (13) in, , , are projection matrices, is the image feature of the second image-text difference image, The text vector of the second image-text difference image, (i.e., Query linear position) is the image matrix of the second image-text difference image, (i.e., Key linear position) is the first text mapping matrix of the second image-text difference image, (ie, Value linear position) is the second text mapping matrix of the second image-text difference image.

[0107] Step S902: input the image matrix and the first text mapping matrix into a low-rank dimensionality raising layer for fusion and sharing, and obtain a low-rank adjusted residual image of the second image-text difference image.

[0108] Specifically, the low-rank adjusted residual image of the second image-text difference image is calculated as follows: (14) in, The image features or text vector of the second image-text difference image are linearly projected , The target dimension mapped to, specifically, the first text mapping matrix mapped by the Key linear position and the image mapping matrix of the Query linear position are fused and shared to obtain a low-rank adjusted residual image.

[0109] It should be noted that, through step S901 and step S902, the image features of the second image-text difference image are Vector with text The projections are fused into a shared interactive space, thereby achieving efficient fusion of cross-modal content.

[0110] Step S903: Input the low-rank adjusted residual image to the output constraint layer, and perform loss constraint on the low-rank adjusted residual image through the second text mapping matrix to obtain a third image-text difference image.

[0111] Furthermore, in order to preserve the information structure of the second image-text difference image, a low-rank approximate image is obtained by the following formula to reduce the computational complexity and meet the efficiency requirements of remote sensing large-scale image reasoning: (15) in, It is the image output by the output constraint layer. Specifically, in the output constraint layer, the low-rank adjusted residual image is linearly constrained by the second text mapping matrix mapped by the Value linear position, so that the low-rank adjusted residual image is re-parameterized to obtain an image that is approximate to the second image-text difference image, that is, the third image-text difference image, thereby achieving the purpose of not adding additional parameter calculations during the fine-tuning fusion process.

[0112] The present application also provides an electronic device comprising a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program, implements the remote sensing image change detection method described in the first embodiment. The electronic device can be any smart terminal, such as a mobile phone, a tablet computer, or an in-vehicle computer.

[0113] See also Figure 11 , Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application, the electronic device comprising: The processor 1101 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the remote sensing image change detection method of the first embodiment provided in the embodiments of the present application; The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called by the processor 1101 to execute the remote sensing image change detection method of the first embodiment provided in the embodiments of this application. Input / output interface 1103, used to implement information input and output; Communication interface 1104, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); Bus 1105 , which transmits information between various components of the device (e.g., processor 1101 , memory 1102 , input / output interface 1103 , and communication interface 1104 ); The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via a bus 1105 .

[0114] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the remote sensing image change detection method of the above-mentioned first embodiment provided in the embodiment of the present application is provided.

[0115] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0116] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0117] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0118] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0119] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0120] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0121] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0123] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

[0126] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for detecting changes in remote sensing images of disaster-stricken areas, characterized in that: include: Obtain pre-disaster and post-disaster images of the affected area, as well as an image detection training model generated based on a visual language architecture; Based on LoRA technology and AdaLN technology, the parameters of the image detection training model are fine-tuned to obtain an image detection target model; Image change detection is performed on the pre-disaster image and the post-disaster image using the image detection target model to obtain a change mask image of the disaster-stricken area, wherein each pixel of the change mask image is used to indicate whether a semantic change occurs.

2. The remote sensing image change detection method according to claim 1, characterized in that: The method of performing image change detection on the pre-disaster image and the post-disaster image using the image detection target model to obtain a change mask image of the disaster-stricken area includes: Inputting the pre-disaster image and the post-disaster image into the image encoder of the image detection target model to perform feature extraction on the pre-disaster image and the post-disaster image to obtain a pre-disaster feature image and a post-disaster feature image; Performing multi-dimensional difference modeling on the pre-disaster feature image and the post-disaster feature image through the difference modeling module of the image detection target model to obtain a difference image; Determining a voice prompt template according to the detection task scenario of the disaster-stricken area; Inputting the language prompt template into the text encoder of the image detection target model to perform multi-scale semantic embedding extraction on the speech prompt template to obtain semantics to be embedded; The difference image is cross-modally fused with the semantics to be embedded through a multi-scale decoder of the image detection target model to obtain the change mask image.

3. The remote sensing image change detection method according to claim 2, characterized in that: The image detection training model is fine-tuned based on the LoRA technology and the AdaLN technology to obtain the image detection target model, including: Determining a target layer containing a Transformer architecture in the backbone network of the image encoder and the backbone network of the text encoder, and determining a module region containing a multi-head attention module in the target layer; Inserting the dimension reduction matrix of the LoRA module on the input side of the linear mapping in the module area, and inserting the dimension increase matrix of the LoRA module on the output side of the linear mapping in the module area; An AdaLN module is introduced at the input side of the dimension-raising matrix to perform adaptive normalization adjustment on the parameters input to the dimension-raising matrix; Adding a constraint loss function on the output side of the LoRA module to reparameterize the output parameters of the LoRA module; Acquire multiple sample images for fine-tuning the model, and freeze the backbone network of the image encoder and the backbone network of the text encoder; Each of the sample images is input into the image detection training model to iteratively adjust the parameters of the image detection training model to obtain the image detection target model.

4. The remote sensing image change detection method according to claim 3, characterized in that: Inputting each of the sample images into the image detection training model to iteratively adjust the parameters of the image detection training model to obtain the image detection target model includes: Inputting each of the sample images into the input layer of the image detection training model so that the sample image is forward propagated to the image encoder to obtain a plurality of feature sample images; Inputting each of the characteristic sample images into the difference modeling module to perform multi-dimensional difference modeling to obtain a plurality of difference sample images between the sample images; Inputting the plurality of difference sample images and the semantics to be embedded into the multi-scale decoder for cross-modal fusion to obtain a plurality of change mask sample images; Calculating the difference between each of the change mask sample images and the corresponding sample image using a loss function to obtain a plurality of loss values; According to the multiple loss values, backpropagation updates are performed on the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder based on the Adam optimization algorithm; If the non-backbone network parameters of the image encoder and the non-backbone network parameters of the text encoder both meet the preset parameter adjustment conditions, the iterative parameter adjustment of the image detection training model is terminated to obtain the image detection target model.

5. The remote sensing image change detection method according to claim 2, wherein: The image encoder includes a first image encoder and a second image encoder, wherein the first image encoder and the second image encoder share a weight. The inputting the pre-disaster image and the post-disaster image into the image encoder of the image detection target model to extract features of the pre-disaster image and the post-disaster image to obtain a pre-disaster feature image and a post-disaster feature image includes: Inputting the pre-disaster image into the first image encoder to perform semantic feature extraction to obtain the pre-disaster feature image; The post-disaster image is input into the second image encoder to perform semantic feature extraction to obtain the post-disaster feature image.

6. The remote sensing image change detection method according to claim 2, characterized in that: The pre-disaster feature image includes a plurality of pre-disaster semantic feature images of different scales, and the post-disaster feature image includes a plurality of post-disaster semantic feature images of different scales, and the scales of each pre-disaster semantic feature image and each post-disaster semantic feature image correspond one to one. The difference modeling module includes a first differential path, a second differential path, a third differential path, and a channel attention module, and the channel attention module is communicatively connected to the first differential path, the second differential path, and the third differential path, respectively; The method of performing multi-dimensional difference modeling on the pre-disaster feature image and the post-disaster feature image through the difference modeling module of the image detection target model to obtain a difference image includes the following steps: Inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into the first differential path to perform feature element subtraction processing to obtain a first feature difference image; Inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into the second differential path to perform cosine similarity measurement to obtain a second feature difference image; Inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into the third differential path to perform feature channel dimension splicing to obtain a third feature difference image; Inputting the first feature difference image, the second feature difference image, and the third feature difference image into the channel attention module for weighted fusion to obtain a feature difference image between the pre-disaster image and the post-disaster image at the same scale; The feature difference images of different scales are superimposed and outputted through the output layer of the difference modeling module to obtain the difference image.

7. The remote sensing image change detection method according to claim 6, characterized in that: The step of inputting the pre-disaster semantic feature image and the post-disaster semantic feature image at the same scale into the first differential path to perform feature element subtraction processing to obtain a first feature difference image includes: Subtracting feature elements of the pre-disaster semantic feature image from feature elements of the post-disaster semantic feature image corresponding to the same dimension to obtain a first feature difference image; Taking the absolute value of each feature difference in the first feature difference image to obtain a second feature difference image; Inputting the second feature difference image into the convolution layer of the first difference path to perform threshold capture on the structural changes of the second feature difference image through a nonlinear activation function to obtain the first feature difference image; The first feature difference image is used to determine a boundary area or a structural difference area between the pre-disaster image and the post-disaster image.

8. The remote sensing image change detection method according to claim 2, characterized in that: The difference image includes a plurality of feature difference images of different scales, and the multi-scale decoder includes a picture-text fusion module, a hierarchical connection module and a low-rank adaptation module, wherein the picture-text fusion module is communicatively connected to the hierarchical connection module, and the hierarchical connection module is communicatively connected to the low-rank adaptation module; The method of cross-modally fusing the difference image with the semantics to be embedded through the multi-scale decoder of the image detection target model to obtain a change mask image of the disaster-stricken area includes the following steps: Inputting the semantics to be embedded and each of the feature difference images into the image-text fusion module, so as to inject the semantics to be embedded into each of the feature difference images through a cross attention mechanism, thereby obtaining a plurality of first image-text difference images of different scales; Inputting each of the image-text difference images into the hierarchical connection module to spatially connect each of the image-text difference images in a residual connection manner to obtain a multi-scale fused second image-text difference image; Inputting the second image-text difference image into the low-rank adaptation module to enhance the cross-modal fusion expression between the image and the text in the second image-text difference image, thereby obtaining a third image-text difference image; The spatial resolution of the third image-text difference image is reconstructed through the upsampling layer of the difference modeling module, and the reconstructed third image-text difference image is convolved and output through the convolution layer of the difference modeling module to obtain the change mask image, wherein the spatial resolution of the change mask image is consistent with the spatial resolution of the post-disaster image.

9. The remote sensing image change detection method according to claim 8, characterized in that: The low-rank adaptation module includes a low-rank dimensionality reduction layer, a low-rank dimensionality increase layer and an output constraint layer, wherein the low-rank dimensionality reduction layer is communicatively connected to the low-rank dimensionality increase layer, and the low-rank dimensionality increase layer is communicatively connected to the output constraint layer; The step of inputting the second image-text difference image into the low-rank adaptation module to enhance the cross-modal fusion expression of the second image-text difference image between image and text to obtain the third image-text difference image includes the following steps: The second image-text difference image is input to the low-rank dimensionality reduction layer to respectively perform mapping projection on the image features and the text vector of the second image-text difference image to obtain an image mapping matrix, a first text mapping matrix, and a second text mapping matrix; Inputting the image mapping matrix and the first text mapping matrix into the low-rank dimensionality raising layer for fusion and sharing, thereby obtaining a low-rank adjusted residual image of the second image-text difference image; The low-rank adjusted residual image is input into the output constraint layer, and the loss constraint of the low-rank adjusted residual image is re-parameterized by the second text mapping matrix to obtain the third image-text difference image.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program executable by a processor, and when the program executable by the processor is executed by the processor, the remote sensing image change detection method according to any one of claims 1 to 9 is implemented.

Images