A remote sensing image change detection method based on self-supervised network

Through the change detection method based on the self-supervised network, the sample prompt is constructed using the local knowledge vector library, which solves the problem of insufficient generalization ability and training efficiency in remote sensing image change detection, and realizes efficient change detection model training and generalization reasoning.

CN118470563BActive Publication Date: 2024-11-01BEIJING DATA INTELLIGENCE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410926416.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-01
Estimated Expiration
2044-07-11

AI Technical Summary

Technical Problem

The existing remote sensing image change detection methods have shortcomings in generalization ability and training efficiency, especially the lack of effective solutions for the change detection of specific land types, and the model training process is complex and inefficient.

Method used

Using a change detection method based on a self-supervised network, the change detection model is trained to improve its generalization reasoning ability and simplify the training process by constructing an initial change detection model and using the local knowledge vector library to construct sample prompts.

Benefits of technology

The general semantic visual feature extraction of the change detection model is realized, the training process is simplified, the network convergence speed is accelerated, the model training efficiency is improved, and the generalization reasoning of the model is realized without changing the model parameters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118470563B_ABST
    Figure CN118470563B_ABST
Patent Text Reader

Abstract

The present invention provides a remote sensing image change detection method based on a self-supervised network, comprising: constructing an initial change detection model, including a change region detection network, a change feature extraction network and an initial recognition network, wherein the change feature extraction network is a self-supervised network; constructing sample prompts and using the sample prompts to train the initial change detection model to obtain a change detection model; using the change detection model to perform change detection on the previous phase image and the next phase image, and outputting the change detection result. By using the change feature extraction network based on the self-supervised network, the change detection model has visual features with universal semantics; by using the sample prompts constructed by the local knowledge vector library to train the initial change detection model to obtain the change detection model, the change detection model can use the prompt information to realize the generalized reasoning of the model without changing the model parameters when performing reasoning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of change detection, and in particular to a remote sensing image change detection method based on a self-supervisory network. Background Art

[0002] Remote sensing images have the characteristics of wide coverage, high resolution, and fast change cycle. Remote sensing image change detection is the process of obtaining dynamic changes of surface objects or phenomena by observing the same area at different times. The identification of changes in remote sensing images is of great significance in the fields of urban and rural planning, environmental monitoring, land management, disaster assessment, and ecological protection. With the development of science and technology, the combination of deep learning technology and high-resolution images can obtain more refined change detection results. However, most of the research work on existing change detection methods focuses on the detection of change areas in multi-temporal remote sensing images, but rarely pays attention to the changes in specific types of objects. In practical applications, it has become one of the important needs to clarify what type of land use the objects in the changed area have changed from.

[0003] In the related art, dual-phase images are input into a neural network model to obtain change type detection results, but the neural network model used usually has poor generalization ability. In addition, the training of the neural network model in the related art requires a large amount of remote sensing image data, the model training process is complicated, and the effect and efficiency are poor. Summary of the invention

[0004] The present invention provides a remote sensing image change detection method based on a self-supervised network. By constructing a change detection model based on a self-supervised network and using a local knowledge vector library to construct sample prompts to train the change detection model, the change detection model has the ability of generalized reasoning.

[0005] A remote sensing image change detection method based on a self-supervised network is provided, the method comprising:

[0006] S1: acquiring a front phase image and a back phase image, and preprocessing the front phase image and the back phase image;

[0007] S2: constructing an initial change detection model, wherein the initial change detection model includes a change region detection network, a change feature extraction network and an initial recognition network, wherein the change region detection network and the change feature extraction network are pre-trained, and the change feature extraction network is a self-supervised learning network based on a Transformer architecture;

[0008] S3: constructing sample prompts through the change region detection network and the change feature extraction network;

[0009] S4: training the initial change detection model using the sample prompt to obtain a change detection model;

[0010] S5: Perform change detection on the previous phase image and the next phase image using the change detection model, and output the change detection result.

[0011] Furthermore, the change area detection network includes a feature extraction layer, a feature contrast layer and an MLP, the feature extraction layer includes a pair of U-net networks; the change feature extraction network is a visual Transformer network based on label-free self-distillation training, including an embedding layer, a Transformer encoder and a projection head; the initial recognition network is a Transformer network.

[0012] The step S3 comprises:

[0013] S31: Acquire the front phase sample and the back phase sample;

[0014] S32: performing change region detection on the front phase sample and the back phase sample through the change region detection network to obtain a plurality of front sample change pattern masks and a plurality of back sample change pattern masks;

[0015] S33: constructing a plurality of sample labels using the plurality of the front sample change pattern masks and the plurality of the rear sample change pattern masks, wherein the sample labels include a front sample pattern label, a rear sample pattern label, and a change authenticity label;

[0016] S34: extracting and processing the features of the front phase sample and the back phase sample respectively through the change feature extraction network to obtain a plurality of front sample spot features and a plurality of back sample spot features;

[0017] S35: splicing the plurality of the front sample spot features, the plurality of the rear sample spot features and the plurality of corresponding sample labels to obtain a plurality of the sample prompts.

[0018] The step S4 comprises:

[0019] S41: randomly selecting i sample prompts from the n sample prompts and replacing the corresponding sample labels with masks to obtain i half-masked prompts, and taking the sample labels before the selected i sample prompts are replaced as true labels;

[0020] S42: Selecting a number of the sample prompts from the ni number and cross-joining them with the i number of the half-mask prompts according to a set ratio to obtain a combined prompt;

[0021] S43: Freeze the parameters of the change region detection network and the change feature extraction network, input the combined prompt into the initial recognition network, and obtain corresponding recognition labels, wherein the recognition labels include a front spot recognition label, a rear spot recognition label, and an authenticity recognition label;

[0022] S44: Calculating a recognition loss value of the initial recognition network based on the recognition label and the true label;

[0023] S45: Adjusting the parameters of the initial recognition network according to the recognition loss value to obtain the recognition network and the change detection model.

[0024] Furthermore, the step S44 includes:

[0025] Step a: Construct a type matrix from all land feature types;

[0026] Step b: performing L2 normalization on the identification label and the type matrix, and respectively calculating the cosine similarity between the front spot identification label, the rear spot identification label and the type matrix to obtain a similarity value;

[0027] Step c: scaling the similarity value and calculating the probability value of each land feature type according to a normalized exponential function;

[0028] Step d: Calculate the cross entropy loss value between the probability value of each ground feature type and the true label to obtain the recognition loss value.

[0029] The step S33 comprises:

[0030] By using a connected region labeling method, each independent sample change spot is extracted from the corresponding front phase sample or the back phase sample using the plurality of front sample change spot masks and the plurality of back sample change spot masks, to obtain and save a plurality of front sample change spots and a plurality of back sample change spots;

[0031] According to the plurality of the front sample change spots and the plurality of the rear sample change spots, corresponding plurality of the front sample spot labels, the rear sample spot labels, and the change authenticity labels are sequentially produced to obtain corresponding plurality of the sample labels.

[0032] The step S34 comprises:

[0033] By using a variation feature extraction network, feature extraction is performed on the front phase sample and the back phase sample respectively to obtain a front sample feature map and a back sample feature map;

[0034] The corresponding feature point sets are screened out from the front sample feature map and the rear sample feature map according to the range of the front sample change pattern mask and the range of the rear sample change pattern mask, and the average value of the feature point sets is calculated to obtain the corresponding front sample pattern feature and the rear sample pattern feature.

[0035] The step S32 comprises:

[0036] Input the front phase sample and the back phase sample into the feature extraction layer, and extract the front sample feature map and the back sample feature map respectively;

[0037] Input the feature map of the previous sample and the feature map of the next sample into the feature comparison layer, calculate the feature difference and perform absolute value processing to obtain the sample change feature map;

[0038] The change feature map is judged by MLP to obtain the front sample change pattern mask and the back sample change pattern mask.

[0039] Furthermore, the pre-phase image and the post-phase image are multispectral remote sensing images.

[0040] The step S5 comprises:

[0041] S51: performing change region detection on the front-phase image and the back-phase image through the change region detection network to obtain a front image spot mask and a back image spot mask;

[0042] S52: extracting features from the front phase image and the back phase image respectively through the change feature extraction network, and performing feature screening and calculation according to the front image spot mask and the back image spot mask respectively, to obtain the front image spot feature and the back image spot feature;

[0043] S53: predicting the ground object types of the front image spot features and the rear image spot features according to the sample prompts through the recognition network, obtaining image recognition labels, and outputting corresponding change detection results.

[0044] The beneficial effects of the present invention include at least:

[0045] 1. By using a change feature extraction network based on a self-supervised network, the change detection model is equipped with visual features of universal semantics;

[0046] 2. The change detection model is obtained by training the initial change detection model using sample prompts constructed by the local knowledge vector library, so that the change detection model can use the prompt information to achieve generalized reasoning of the model without changing the model parameters during reasoning;

[0047] 3. Using sample prompts to train the initial change detection model can simplify the training process, accelerate network convergence, and improve the efficiency of model training while ensuring quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A schematic diagram of a process of a remote sensing image change detection method based on a self-supervised network according to the present invention;

[0050] Figure 2 A schematic diagram of a process for constructing a sample prompt of the present invention;

[0051] Figure 3 A schematic diagram of a process of training an initial change detection model using sample prompts according to the present invention;

[0052] Figure 4 A schematic flow chart of a method for calculating an identification loss value of the present invention;

[0053] Figure 5 It is a schematic diagram of a process of performing change detection on a front-phase image and a back-phase image by using a change detection model of the present invention;

[0054] Figure 6 This is an example diagram of the change detection results output after the present invention performs change detection on the previous phase image and the next phase image through the change detection model. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0056] Figure 1 The flow chart of an embodiment of a remote sensing image change detection method based on a self-supervised network provided by the present invention is shown. The method comprises:

[0057] Step S1: Acquire a front-phase image and a rear-phase image, and pre-process the front-phase image and the rear-phase image.

[0058] Specifically, the pre-phase image and the post-phase image are obtained, and pre-processing such as radiation correction, atmospheric correction, geometric correction, image registration and mosaicking is performed. The pre-phase image and the post-phase image can be three-band remote sensing images or multispectral remote sensing images. Multispectral remote sensing images usually refer to four-band remote sensing images, including four multispectral bands of red, green, blue and near infrared.

[0059] Remote sensing imaging is affected by many factors, such as sensor attitude changes, satellite platform motion, optical system distortion, etc., which causes the captured remote sensing images to be distorted, offset, squeezed, stretched, and other types of geometric distortion relative to the actual ground position. Before using high-resolution remote sensing images for change detection, the remote sensing images need to be preprocessed. Preprocessing includes radiation correction, atmospheric correction, geometric correction, image registration, and mosaicking.

[0060] Step S2: construct an initial change detection model, which includes a change area detection network, a change feature extraction network and an initial recognition network, wherein the change area detection network and the change feature extraction network are pre-trained, and the change feature extraction network is a self-supervised learning network based on the Transformer architecture.

[0061] Based on an embodiment of the present invention, an initial change detection model is constructed, which includes a change area detection network, a change feature extraction network and an initial recognition network. Among them, the change area detection network includes a feature extraction layer, a feature comparison layer and a multilayer perceptron (Multilayer Perceptron, MLP), and the feature extraction layer uses two U-net networks with the same structure. MLP includes a 3*3 convolutional layer. The change area detection network is used to detect the area where the remote sensing images of the previous and next phases have changed, but it cannot detect the content of the change. The change area detection network is pre-trained. The initial change area detection network is pre-trained using a remote sensing image data set to obtain a change area detection network.

[0062] The variation feature extraction network is based on a network based on Self-Distillation with NoLabels (DINO). DINO is a self-supervised learning network based on the Transformer architecture, which learns visual feature representations by performing self-supervised training on unlabeled images. The overall DINO framework contains two identical architectures, namely the teacher network and the student network, both of which include an embedding layer, a Transformer encoder, and a projection head. The Transformer encoder includes L repeatedly stacked encoding modules, which include a linear layer (Layer Norm), a self-attention layer, a feedforward neural network layer, and a jump connection. The projection head is a multi-layer perceptron. Batch normalization is not used in the DINO network. The DINO network first uses its own output to generate pseudo labels, and then uses these pseudo labels to retrain the model, thereby further improving the performance and generalization ability of the model.

[0063] It should be noted that the change feature extraction network of the present invention uses DINOv2, an enhanced version of DINO. DINOv2 is trained on the basis of DINO with more data and more skills. DINOv2 uses unsupervised learning methods to train image encoders on large image datasets to obtain visual features with universal semantics, which are applicable to various image distributions and tasks, and no fine-tuning is required in downstream tasks.

[0064] The change feature extraction network of the present invention is pre-trained. Specifically, a Vision Transformer (ViT) model with about 4 million parameters is trained based on DINOv2 using a remote sensing image dataset, and it can be compressed into a series of small models that can be used for different tasks through an unsupervised distillation method to obtain the change feature extraction network.

[0065] In particular, the remote sensing image datasets used for the above pre-training include three-band remote sensing image data and multispectral remote sensing image data. For example, a change feature extraction network is obtained by training the DINOv2 based on multispectral remote sensing image data, and a change region detection network is obtained by training the initial change region detection network using multispectral image data.

[0066] The initial recognition network is a standard Transformer network, which consists of multiple stacked residual blocks. The residual blocks include linear layers, multi-head attention layers, and MLP layers.

[0067] The present invention constructs an initial change detection model by utilizing a change feature extraction network based on a self-supervisory network, so that the initial change detection model has visual features with universal semantics and has good generalization ability.

[0068] Step S3: constructing sample prompts through the change region detection network and the change feature extraction network.

[0069] According to one embodiment of the present invention, Figure 2 As shown, constructing sample prompts through the change region detection network and the change feature extraction network includes:

[0070] Step S31: Acquire the front phase sample and the back phase sample.

[0071] A portion of pre-phase remote sensing images and post-phase remote sensing images in which the ground object contents actually change are selected from the pre-phase images and post-phase images that have been preprocessed as pre-phase samples and post-phase samples.

[0072] Step S32: performing change region detection on the front phase samples and the back phase samples through the change region detection network to obtain a plurality of front sample change pattern masks and a plurality of back sample change pattern masks.

[0073] First, the samples of the previous phase and the samples of the next phase are input into the feature extraction layer to extract the feature map of the previous sample and the feature map of the next sample, respectively. Then, the feature map of the previous sample and the feature map of the next sample are input into the feature comparison layer to calculate the feature difference and perform absolute value processing to obtain the sample change feature map. Finally, the change feature map is judged by MLP to obtain the change pattern mask of the previous sample and the change pattern mask of the next sample.

[0074] The pre-sample change pattern mask and the post-sample change pattern mask are binary vectors used to mark the areas where the image content of the pre-phase sample and the post-phase sample has changed. Assuming that the number of areas where a pair of pre-phase samples and post-phase samples have changed is t, then t pre-phase sample change pattern masks and t post-phase sample change pattern masks are output, where t is a positive integer.

[0075] Step S33: constructing a plurality of sample labels using the plurality of the front sample change pattern masks and the plurality of the rear sample change pattern masks, wherein the sample labels include a front sample pattern label, a rear sample pattern label, and a change authenticity label.

[0076] The specific steps are as follows:

[0077] (i) By means of a connected region labeling method, a plurality of the front sample change pattern masks and a plurality of the rear sample change pattern masks are used to extract each independent sample change pattern from the corresponding front phase sample or rear phase sample, thereby obtaining a plurality of front sample change patterns and a plurality of rear sample change patterns and saving them.

[0078] In step S32, a pair of front phase samples and rear phase samples are detected to obtain t front sample change pattern masks and t rear sample change pattern masks, which correspond to t front sample change patterns and t rear sample change patterns, respectively. In order to make independent labels for each sample change pattern, it is necessary to extract the sample change patterns separately. In this embodiment, the independent sample change patterns are extracted by the connected region labeling method.

[0079] The processing object of the connected component labeling algorithm is usually a binary image. The function of the algorithm is to separate the foreground object of interest from the image background so that related image processing operations can be performed on the foreground object of interest later.

[0080] Taking the previous phase sample as an example, the specific steps of extracting the previous sample change spots one by one through the connected region labeling technology include:

[0081] 1) For the previous phase sample, the pixels in the mask area with the previous sample change pattern are set as foreground pixels, and the pixel value is set to 1, and the pixels in the mask area without the previous sample change pattern are set as background pixels, and the pixel value is set to 0;

[0082] 2) Scan the pixels of the previous phase sample until the scanned pixel value is 1, put the pixel point as the starting seed into a set marked as A, and push all foreground pixels adjacent to the seed into the stack; if the pixel point has been scanned, continue scanning, otherwise go to step c);

[0083] 3) Pop the top pixel of the stack, put it in the set marked as A, and push all foreground pixels adjacent to the top pixel of the stack into the stack;

[0084] 4) Repeat step 3) until the stack is empty; at this point, a connected region in the image is extracted and the connected region is marked as A;

[0085] 5) Repeat steps 2) to 4) until the entire image is traversed and i independent previous sample change patches are obtained.

[0086] Correspondingly, the same steps are adopted to process the later phase samples to obtain i independent later sample change patterns.

[0087] (ii) According to the plurality of the front sample change spots and the plurality of the rear sample change spots, a corresponding plurality of the front sample spot labels, the rear sample spot labels, and the change authenticity labels are sequentially produced to obtain a corresponding plurality of the sample labels.

[0088] First, an initial label is generated based on the front sample change pattern and the back sample change pattern, and then the initial label is vectorized to obtain the sample label.

[0089] Specifically, the initial label is obtained by manually marking the type of each front sample change map and the back sample change map in turn. The initial label includes the initial label of the front phase sample, the initial label of the back phase sample and the initial change authenticity label. Since the initial label is in text format, it needs to be converted into a vector format. For the vectorization of the initial label, you can choose to use a Transformer-based deep bidirectional language representation network (Bidirectional Encoder Representations from Transformers, BERT) or a generative pre-trained network (Generative Pre-trained Transformer, GPT) or other networks. This embodiment gives priority to the BERT network. The initial label is input into the BERT network, and the initial label in text form is converted into a vector label through the BERT network to obtain the final sample label: ,in, represents the previous sample patch label, represents the label of the sample patch, and v represents the change authenticity label. The change authenticity label is used to mark whether the sample change patch is a true change or a false change. The false change patch can be removed by the change authenticity label, thus improving the accuracy of remote sensing image change detection.

[0090] Step S34: extracting and processing the features of the front phase samples and the back phase samples respectively through the change feature extraction network to obtain a plurality of front sample spot features and a plurality of back sample spot features.

[0091] Specifically, step S34 includes:

[0092] (1) Extracting features of the front-phase sample and the back-phase sample respectively through a variation feature extraction network to obtain a front-sample feature map and a back-sample feature map.

[0093] Taking the previous phase sample as an example, the previous phase sample is divided into N×N image blocks, where N is 16 or 8, and the image blocks are input into the change feature extraction network. Specifically, the image blocks are mapped into a linear vector sequence after passing through the linear mapping layer. The linear vector sequence is input into the embedding layer, and a learnable position code is added and a special classification tag is added. The classification tag is a learnable tag used to aggregate information from the entire linear vector sequence. It is not attached to any label or supervision. The embedding layer concatenates the position code and the classification tag and inputs it into the Transformer encoder. The Transformer encoder uses the self-attention mechanism to update the classification tag and outputs the classification tag to the projection head to obtain the previous sample feature map. The next phase sample is input into the change feature extraction network, and the next sample feature map is obtained by adopting the same steps as above.

[0094] (2) Selecting corresponding feature point sets from the front sample feature map and the rear sample feature map according to the range of the front sample change pattern mask and the range of the rear sample change pattern mask, respectively, and calculating the average value of the feature point sets to obtain the front sample pattern feature and the rear sample pattern feature.

[0095] Specifically, taking the previous phase sample as an example, according to the range of the previous sample change pattern mask, the corresponding feature point set is screened out from the previous sample feature map, and the average value of the feature point set is calculated to obtain the corresponding previous sample pattern feature. .

[0096] Alternatively, it is also possible to select high-quality feature points from the feature point set to form a high-quality feature point subset, and calculate the average value of the high-quality feature point subset as the previous sample spot feature. For each previous sample change spot, the number of high-quality feature points selected from the corresponding feature point set can be the same or different. For example, it can be set to select 10 high-quality feature points from each feature point set, or it can be set to adaptively set the number of selected high-quality feature points according to the size of the previous sample change spot.

[0097] In the same way, the mask range of the rear sample change pattern is used to select the corresponding feature point set from the rear feature map, and the average value of the feature point set is calculated to obtain the corresponding rear sample pattern feature. .

[0098] Step S35: splicing the plurality of the front sample spot features, the plurality of the rear sample spot features and the plurality of corresponding sample labels to obtain a plurality of the sample prompts.

[0099] The previous sample pattern feature , the sample pattern features With the previous sample spot label , the sample patch label , the authenticity label of the change Perform splicing to obtain the sample prompt .

[0100] Assuming that there are n regions where all the previous phase samples and the next phase samples have changed, there are n previous sample change pattern masks and n next sample change pattern masks, and the corresponding sample prompts are n, where n is a positive integer.

[0101] The present invention constructs sample prompts based on a local knowledge vector library and uses the sample prompts to pre-train the initial change detection model to obtain a change detection model, which can simplify the training process, accelerate network convergence, and enable the final change detection model to have powerful generalization reasoning capabilities.

[0102] Step S4: using the sample prompts to train the initial change detection model to obtain a change detection model.

[0103] The change region detection network and change feature extraction network of the present invention are pre-trained, and the training of the initial change detection model mainly trains the initial recognition network. In the process of training the initial recognition network, the parameters of the change region detection network and the change feature extraction network are frozen.

[0104] Specifically, Figure 3 As shown in Figure 2, the process of training the initial change detection model using sample prompts is:

[0105] S41: randomly selecting i sample prompts from the n sample prompts and replacing the corresponding sample labels with masks to obtain i half-masked prompts, and taking the sample labels before the selected i sample prompts are replaced as true labels.

[0106] Specifically, i sample prompts are randomly selected from n sample prompts. , replace the front sample spot label, the back sample spot label and the change authenticity label with a mask to obtain i half-mask prompts in Represents the mask mask, and replaces the sample label before with the i selected sample prompts is the true label, and x is a positive integer not greater than n.

[0107] S42: Select a number of the sample prompts from the ni number, and cross-combine them with the i number of half-mask prompts according to a set ratio to obtain a combined prompt.

[0108] Select several sample prompts from ni sample prompts other than the selected i sample prompts as reference prompts, and splice the several reference prompts with i half-mask prompts according to a set ratio relationship to obtain a combined prompt. The set ratio relationship can be set as needed, for example: splicing one half-mask prompt with one reference prompt, or splicing one half-mask prompt with multiple reference prompts, or splicing one reference prompt with multiple half-mask prompts, or splicing multiple half-mask prompts with multiple reference prompts.

[0109] In this embodiment, the half-mask prompt and the reference prompt are cross-joined in a ratio of 1:2 to obtain a combined prompt. With 2 reference tips , Cross-stitching, the combination tips after splicing . Half-mask hint The corresponding original sample prompt is .

[0110] S43: Freeze the parameters of the change region detection network and the change feature extraction network, input the combined prompt into the initial recognition network, and obtain corresponding recognition labels, which include a front spot recognition label, a rear spot recognition label, and an authenticity recognition label.

[0111] The combined prompt is input into the initial recognition network to train the initial recognition network. The initial recognition network predicts the sample spot label in the half-mask prompt based on the sample spot features in the half-mask prompt and the sample spot features and sample spot labels in the reference prompt to obtain the corresponding recognition label. The recognition label includes the front spot recognition label, the back spot recognition label and the authenticity recognition label. Input the initial recognition network, which predicts the mask label in the combined prompt to obtain the recognition label , where the identification tag The corresponding true label is .

[0112] After the initial recognition network predicts the front spot recognition label and the back spot recognition label, it judges the authenticity. When the difference between the current spot recognition label and the back spot recognition label is greater than the set threshold, it is considered to be a true change, and the corresponding authenticity recognition label represents "true". When it is less than or equal to the preset threshold, it is considered to be a false change, and the corresponding authenticity recognition label represents "false".

[0113] By combining the masked sample prompts with the original labeled sample prompts, the initial recognition network is trained using the combined prompts, making the initial recognition network learn faster.

[0114] S44: Calculate a recognition loss value of the initial recognition network based on the recognition label and the true label.

[0115] According to one embodiment of the present invention, Figure 4 As shown, the method for calculating the recognition loss value is as follows:

[0116] Step a: Construct a type matrix from all land feature types.

[0117] Assuming that there are k types of all features, and each feature type is a one-dimensional vector of size c, then all feature types constitute a type matrix For identification tags , front spot identification label , rear spot recognition tag and authenticity identification label are all one-dimensional vectors of size c.

[0118] Step b: Perform L2 normalization on the identification label and the type matrix, and calculate the cosine similarity between the front spot identification label, the back spot identification label and the type matrix respectively to obtain the similarity value.

[0119] Perform L2 normalization on the identification label and type matrix, and calculate the previous spot identification label in the identification label With type matrix The cosine similarity between all the land object types in the image is calculated, and the patch recognition label is calculated. With type matrix The cosine similarity between all the feature types in is used to obtain the similarity value, including k front similarity values ​​and k back similarity values.

[0120] Step c: Scale the similarity value and calculate the probability value of each feature type according to the normalized exponential function.

[0121] Specifically, the k front similarity values ​​and the k back similarity values ​​are scaled, for example, by 100 times, and the normalized exponential function (SoftMax) is used to calculate the corresponding probability values ​​of each feature type, including the k front feature type probability values ​​and the k back feature type probability values.

[0122] Step d: Calculate the cross entropy loss value between the probability value of each feature type and the true label to obtain the recognition loss value.

[0123] Specifically, calculate the k previous feature type probability values, k subsequent feature type probability values ​​and the true label The corresponding previous sample patch label in , and then sample patch labels The cross-entropy loss (CE Loss) between them is used to obtain the corresponding pre-recognition loss value and post-recognition loss value, and the recognition loss value is obtained based on the pre-recognition loss value and the post-recognition loss value. The recognition loss value obtained based on the pre-recognition loss value and the post-recognition loss value can be the sum of the pre-recognition loss value and the post-recognition loss value as the recognition loss value, or one of them can be selected alternately as the recognition loss value.

[0124] The calculation formula of the cross entropy loss value is:

[0125]

[0126] Where k represents the number of landform types. represents the true label of the j-th type, It represents the probability that the change detection model predicts that this sample belongs to the jth feature type.

[0127] The present invention uses cross entropy loss as recognition loss, and cross entropy loss is used to measure the similarity between the recognition label and the true label.

[0128] The method of calculating the recognition loss of the present invention makes the feature type scalable. In practical applications, when the feature type changes, only the type matrix needs to be changed. The corresponding feature type in is easy to operate and has strong scalability. For example, when a feature type is added, a one-dimensional vector of size c is added, and the type matrix becomes .

[0129] S45: Adjusting the parameters of the initial recognition network according to the recognition loss value to obtain a recognition network and a change detection model.

[0130] The parameters of the change region detection network and the change feature extraction network are frozen, and the parameters of the initial recognition network are adjusted according to the recognition loss value. The process is iterated continuously until the initial recognition network converges, thereby obtaining a recognition network and a corresponding change detection model.

[0131] When the sample prompt is constructed based on multispectral remote sensing image data, the change detection model trained using the sample prompt can be used to perform change detection on the multispectral remote sensing image to obtain the corresponding multispectral remote sensing image detection result.

[0132] The present invention uses sample prompts to train an initial change detection model to obtain a change detection model, so that when the change detection model is used for reasoning, the prompt information can be used to achieve generalized reasoning of the model without changing the model parameters.

[0133] Step S5: Perform change detection on the previous phase image and the next phase image using the change detection model, and output the change detection result.

[0134] Specifically, in the inference stage, the change detection model is used to perform change detection on the previous phase image and the next phase image. Figure 5 As shown, the change detection of the previous phase image and the next phase image by the change detection model includes:

[0135] S51: Perform change region detection on the front-phase image and the back-phase image through the change region detection network to obtain a front image spot mask and a back image spot mask.

[0136] The previous phase image and the next phase image are input into the change detection model. First, the change spots of the previous phase image and the next phase image are detected through the change region detection network, and marked with masks, and the previous image spot mask and the next image spot mask are output.

[0137] Step S52: extracting features from the front phase image and the back phase image respectively through the change feature extraction network, and performing feature screening and calculation according to the front image spot mask and the front image spot mask respectively to obtain corresponding front image spot features and back image spot features.

[0138] Specifically, the feature extraction network is used to extract features from the front phase image and the back phase image respectively, to obtain a front image feature map and a back image feature map. According to the range of the front image spot mask, a corresponding image feature point set is selected from the front image feature map, and the average value of the image feature point set is calculated to obtain the corresponding front image spot feature; according to the range of the back image spot mask, a corresponding image feature point set is selected from the back image feature map, and the average value of the image feature point set is calculated to obtain the corresponding back image spot feature.

[0139] S53: predicting the ground object types of the front image spot features and the rear image spot features according to the sample prompts through the recognition network, obtaining image recognition labels, and outputting corresponding change detection results.

[0140] The front image spot features, the rear image spot features and the sample prompts are input into the recognition network. The recognition network predicts the ground object type of the front image spot features and the rear image spot features according to the sample prompts, predicts the corresponding ground object type, and determines the authenticity of the change to obtain the image recognition label. The image recognition label includes the front image spot label, the rear image spot label and the image change authenticity label. Figure 6 As shown, the final output includes the front phase image, the back phase image and the corresponding mask, and the change detection result of the image recognition label. Figure 6 In the image, it is detected that the type of land features in the previous phase image and the next phase image has changed, and the land feature type has changed from non-agricultural land to cultivated land, and the authenticity is identified as "true". The changed area is marked with a corresponding mask, and the identification label is marked on the image and output together with the image.

[0141] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A remote sensing image change detection method based on a self-supervised network, characterized in that: The method includes: S1: acquiring a front phase image and a back phase image, and preprocessing the front phase image and the back phase image; S2: constructing an initial change detection model, wherein the initial change detection model includes a change region detection network, a change feature extraction network and an initial recognition network, wherein the change region detection network and the change feature extraction network are pre-trained, and the change feature extraction network is a self-supervised learning network based on a Transformer architecture; S3: constructing sample prompts through the change region detection network and the change feature extraction network; S4: training the initial change detection model using the sample prompt to obtain a change detection model; S5: performing change detection on the previous phase image and the next phase image by using the change detection model, and outputting the change detection result; The step S3 comprises: S31: Acquire the front phase sample and the back phase sample; S32: performing change region detection on the front phase sample and the back phase sample through the change region detection network to obtain a plurality of front sample change pattern masks and a plurality of back sample change pattern masks; S33: constructing a plurality of sample labels using the plurality of the front sample change pattern masks and the plurality of the rear sample change pattern masks, wherein the sample labels include a front sample pattern label, a rear sample pattern label, and a change authenticity label; S34: extracting and processing the features of the front phase sample and the back phase sample respectively through the change feature extraction network to obtain a plurality of front sample spot features and a plurality of back sample spot features; S35: splicing the plurality of the front sample spot features, the plurality of the rear sample spot features and the plurality of corresponding sample labels to obtain a plurality of the sample prompts.

2. The method according to claim 1, characterized in that The change region detection network includes a feature extraction layer, a feature contrast layer and an MLP, wherein the feature extraction layer includes a pair of U-net networks; the change feature extraction network is a visual Transformer network based on label-free self-distillation training, including an embedding layer, a Transformer encoder and a projection head; the initial recognition network is a Transformer network.

3. The method according to claim 1, characterized in that: The step S4 comprises: S41: randomly selecting i sample prompts from the n sample prompts and replacing the corresponding sample labels with masks to obtain i half-masked prompts, and taking the sample labels before the selected i sample prompts are replaced as true labels; S42: Selecting a number of the sample prompts from the ni number and cross-joining them with the i number of the half-mask prompts according to a set ratio to obtain a combined prompt; S43: Freeze the parameters of the change region detection network and the change feature extraction network, input the combined prompt into the initial recognition network, and obtain corresponding recognition labels, wherein the recognition labels include a front spot recognition label, a rear spot recognition label, and an authenticity recognition label; S44: Calculating a recognition loss value of the initial recognition network based on the recognition label and the true label; S45: Adjusting the parameters of the initial recognition network according to the recognition loss value to obtain the recognition network and the change detection model.

4. The method according to claim 3, characterized in that The step S44 comprises: Step a: Construct a type matrix of all land feature types; Step b: performing L2 normalization on the identification label and the type matrix, and respectively calculating the cosine similarity between the front spot identification label, the rear spot identification label and the type matrix to obtain a similarity value; Step c: scaling the similarity value and calculating the probability value of each land feature type according to a normalized exponential function; Step d: Calculate the cross entropy loss value between the probability value of each feature type and the true label to obtain the recognition loss value.

5. The method according to claim 1, characterized in that The step S33 comprises: By using a connected region labeling method, each independent sample change spot is extracted from the corresponding front phase sample or the back phase sample using the plurality of front sample change spot masks and the plurality of back sample change spot masks, to obtain and save a plurality of front sample change spots and a plurality of back sample change spots; According to the plurality of the front sample change spots and the plurality of the rear sample change spots, corresponding plurality of the front sample spot labels, the rear sample spot labels, and the change authenticity labels are sequentially produced to obtain corresponding plurality of the sample labels.

6. The method according to claim 1, characterized in that The step S34 comprises: The feature extraction network is used to extract features of the front phase sample and the back phase sample respectively, so as to obtain a front sample feature map and a back sample feature map; The corresponding feature point sets are screened out from the front sample feature map and the rear sample feature map according to the range of the front sample change pattern mask and the range of the rear sample change pattern mask, and the average value of the feature point sets is calculated to obtain the corresponding front sample pattern feature and the rear sample pattern feature.

7. The method according to claim 2, characterized in that The step S32 comprises: Inputting the front phase sample and the back phase sample into the feature extraction layer, extracting the front sample feature map and the back sample feature map respectively; Inputting the front sample feature map and the rear sample feature map into the feature comparison layer, calculating the feature difference and performing absolute value processing to obtain a sample change feature map; The MLP is used to perform change determination on the change feature map to obtain a front sample change pattern mask and a rear sample change pattern mask.

8. The method according to claim 1, characterized in that The pre-phase image and post-phase image are multi-spectral remote sensing images.

9. The method according to claim 3, characterized in that: The step S5 comprises: S51: performing change region detection on the front-phase image and the back-phase image through the change region detection network to obtain a front image spot mask and a back image spot mask; S52: extracting features from the front phase image and the back phase image respectively through the change feature extraction network, and performing feature screening and calculation according to the front image spot mask and the back image spot mask respectively, to obtain the front image spot feature and the back image spot feature; S53: predicting the ground object types of the front image spot features and the rear image spot features according to the sample prompts through the recognition network, obtaining image recognition labels, and outputting corresponding change detection results.