Contour precision and positioning precision collaborative optimization instance level change detection method and system

Abstract

The invention provides a contour precision and positioning precision collaborative optimization instance level change detection method and system. The method comprises the following steps: making a change detection label for preprocessed satellite data; training a change detection model by adopting joint loss formed by pixel-level loss and instance-level loss with positioning precision constraint significance; the instance-level evaluation index AP10 is adapted to change detection result evaluation; evaluating the trained change detection model by adopting the adapted evaluation index AP10; and inputting the initial change pattern spot and the double-time-phase image obtained by the trained change detection model into an SAM change detection algorithm to obtain a change pattern spot. According to the scheme provided by the invention, double requirements of high positioning precision and high contour precision can be met, the integrity of large pattern spots is improved, and false detection and missing detection of small pattern spots are reduced. The boundary integrating degree of a detected area and a real change area is enhanced, and the contour precision and the positioning precision of a change detection result in a complex scene are improved.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing change detection, and in particular relates to an instance-level change detection method and system for collaboratively optimizing contour accuracy and positioning accuracy. Background Art

[0002] Remote sensing change detection can help us understand the objects at the geographic spatial scale and time scale, explore the dynamic development process or future trend of the objects under the influence of natural or human factors, and is widely used in land change survey (Li Qimei, 2022), post-disaster damage assessment (Ma Jianwen et al., 2004), urban development planning (Peng Shunxi, 2007) and other fields. At present, a large number of remote sensing change detection applications (such as illegal construction inspection, ecological protection area monitoring, etc.) require algorithms to accurately detect changes and obtain fine change patches, which means that change detection algorithms with high positioning accuracy and contour accuracy are important requirements for the current algorithm implementation. Among them, contour accuracy represents the degree of fit between the contour of the patch and the boundary of the actual change area, that is, the degree of refinement of the evaluation contour. Positioning accuracy represents the degree of intersection between the patch and the real change area in space, that is, the false detection and missed detection of the evaluation patch. Figure 2 Some positioning accuracy and contour accuracy evaluation scenarios are shown, where the blue spots are the output of the change detection algorithm and the red spots are the change labels.

[0003] Change detection has gone through a long development process. As remote sensing enters the era of big data (Zhang Bing, 2017, Zhang Bing et al., 2022), deep learning change detection algorithms have made great progress. A large number of research results show that in the change detection task of large-scale high-resolution remote sensing images, the performance of deep learning methods is better than traditional pixel-oriented and object-oriented change detection algorithms (Wang Yihao, 2024, Hou et al., 2021, Lukang Wang et al., 2024). Traditional methods (such as vector analysis) are difficult to handle complex texture information in high-resolution images, and too much manual intervention leads to insufficient generalization ability of the model over a large range. In contrast, data-driven deep learning change detection algorithms bring less manual intervention, higher accuracy, stronger robustness, and better generalization ability, and have gradually become a huge breakthrough in the application of change detection services. In the deep learning change detection algorithm, the algorithm can be divided into three types according to the granularity of the detection results: scene-level change detection (Scene-Level Change Detection, SLCD), object-level change detection (Object-Level Change Detection, OLCD), and pixel-level change detection (Pixel-Level Change Detection, PLCD). The schematic diagrams of different detection results are shown in the figure. Figure 3As shown. Among them, SLCD and OLCD lack contour accuracy in detection granularity, and have difficulty detecting changes in complex structure objects (such as roads, rivers, construction sites, etc.), so they are difficult to apply to high-precision change detection in large-scale high-resolution scenes. PLCD completes the judgment of change and invariance for each pixel, and simultaneously realizes the positioning of the changed area and fine contour extraction, achieving relatively good results. However, the current PLCD has the problem of lack of positioning accuracy constraints. Even if high contour accuracy can be obtained, the problems of missed detection and false detection of small spots are still serious. As shown Figure 2 As shown in the figure, although PLCD has achieved fine detection results for obvious large-change spots, it has missed a large number of small spots, resulting in low positioning accuracy. The positioning accuracy of PLCD is insufficiently constrained, which is largely caused by the lack of attention to small spots in loss calculation and evaluation indicators. PLCD mainly constrains model training through pixel-level loss functions (such as binary cross entropy loss and dice loss), and evaluates model accuracy with pixel-level evaluation indicators (such as kappa coefficient and F1 score). Among them, all loss calculations and accuracy evaluations are completed on a pixel-by-pixel basis. During the calculation process, both large and small spot pixels enjoy the same attention, which also leads to more obvious large spots with more pixels to obtain greater optimization efforts, while small spots are often ignored. In addition, higher pixel-level accuracy evaluation indicators cannot effectively indicate the detection accuracy (i.e., positioning accuracy) of the change area. For example, when a large number of small spots are missed or misdetected, the current pixel-level accuracy evaluation indicators will still reflect higher accuracy values ​​due to the influence of large spots with more pixels, which will gradually shift the optimization direction during model training.

[0004] Disadvantages of existing technology

[0005] Currently, there is still a lack of change detection models that have both high contour accuracy and positioning accuracy. The research intends to conduct research on remote sensing image change detection algorithms that couple instance-level constraints and large visual models. Based on the mechanism of problem generation, this project intends to construct a change detection loss function from the scale of patch instances, improve the patch positioning ability of the change detection algorithm, and further improve the integrity and accuracy of the patches based on the large visual model, and collaboratively optimize the positioning accuracy and contour accuracy of the change patches. Ultimately, this project will form a set of change detection algorithms that have both high contour accuracy and positioning accuracy, meet the actual needs of current large-scale change detection tasks, and improve the application and landing capabilities of deep learning change detection algorithms. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a technical solution of an instance-level change detection method with coordinated optimization of contour accuracy and positioning accuracy. In order to solve the above technical problems, the concept of "instance" is introduced to construct an instance-level change detection algorithm. The comparison between the instance-level change detection results and the SLCD, OLCD and PLCD results is as follows: Figure 3 As shown in Figure 2. The concept of “instance” comes from the field of computer vision and represents the set of all pixels belonging to the same object. When this concept is introduced into change detection, different change spots can be regarded as different instances, and each instance is an equal individual with a clear position and fine contour (such as Figure 3 Column 6, pixels of different colors belong to different instances), missed detections and false detections of large and small spots can be regarded as missed detections and false detections of an instance. Instance-level change detection is conducive to focusing on the detection of small spots, and for this reason it is more in line with the requirements of high positioning accuracy and high contour accuracy in current remote sensing change detection applications. The instance-level change detection algorithm, based on PLCD, aims to address the problem of lack of attention to small spots in loss function calculation and accuracy evaluation in PLCD, construct an instance-level loss function and instance-level accuracy evaluation index, increase the penalty for missed detection and false detection of small spots, and improve the positioning accuracy of the change detection result spots. At the same time, in order to avoid over-optimizing positioning accuracy and ignoring contour accuracy, the project plans to migrate the deep learning segmentation large model SAM (Kirillov et al., 2023) to the change detection task, and build a bridge for collaborative optimization of positioning accuracy and contour accuracy by combining the high positioning accuracy change spot prompts and SAM's low-cost fine contour acquisition capability, further improving the accuracy of instance-level change detection results.

[0007] The first aspect of the present invention discloses an instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy, the method comprising:

[0008] Step S1, collecting high-resolution satellite data and preprocessing the satellite data; labeling the preprocessed satellite data for change detection; dividing the satellite data and the corresponding labels into a training set and a validation set;

[0009] Step S2, applying the training set, using a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint, to train a change detection model;

[0010] Step S3, adapting the instance-level evaluation index AP10 to the change detection result evaluation; applying the validation set, and using the adapted evaluation index AP10 to evaluate the trained change detection model;

[0011] Step S4: input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots.

[0012] According to the method of the first aspect of the present invention, in step S1, performing change detection tagging on the pre-processed satellite data includes:

[0013] Combined with the before and after phase images, ArcGIS is used as a sample marking tool to identify and outline the change spots; the change spots are used as samples with change detection labels; the marked change spots are for changes in cultivated land, woodland, gardens, grasslands, water bodies, roads, residential areas, greenhouses and construction areas outside the built-up area; the samples require the marking of change spots with an area greater than 150 square meters.

[0014] According to the method of the first aspect of the present invention, in step S2, the joint loss formed by the pixel-level loss and the instance-level loss with positioning accuracy constraint is:

[0015] loss = l changt +l inst

[0016]

[0017] Among them, loss represents the joint loss; l change represents the pixel-level loss function based on dice calculation; l inst represents the instance loss function; L inst,i Indicates the prediction result; P inst,i Represents the label of the sample; dice() represents dice calculation; N represents the total number of instances.

[0018] According to the method of the first aspect of the present invention, in step S3, adapting the instance-level evaluation indicator AP10 to the change detection result evaluation includes:

[0019] AP10 distributes the accuracy evaluation of the entire image to each change patch instance, calculates the IoU between the predicted patch and the true value patch based on the instance, and regards the patch with IoU greater than 0.1 as a correctly predicted patch, i.e. TP; the patch with IoU less than 0.1 is regarded as a falsely detected patch, i.e. FP; the true value patch that is not detected is regarded as a missed detection patch, i.e. FN; if there are multiple predicted patches with IoU greater than 0.1 for the same true value, the one with the highest confidence is TP, and the others are FP.

[0020] According to the method of the first aspect of the present invention, in step S4, the initial change spots and the dual-phase image obtained by the trained change detection model are input into the SAM change detection algorithm to obtain the change spots, which includes:

[0021] Based on the trained change detection model, the initial change pattern is obtained, and the prompt points of the change area are generated according to the initial change pattern; the prompt points are passed through the prompt encoder of SAM to obtain the prompt features with the location information of the change area; the dual-phase image is input into the image encoder of SAM and encoded separately to obtain the dual-phase features; the dual-phase features are fused through the feature fusion module to obtain the change features; the change features and the prompt features are input into the decoder of SAM together to output the change pattern.

[0022] According to the method of the first aspect of the present invention, in step S4, the feature fusion module fuses the bi-phase features based on the cross-layer cross attention module to obtain the change feature;

[0023] The cross-layer cross attention module inputs the previous phase feature and the next phase feature into two cross-layer self-attention modules respectively, takes the previous phase feature as the query vector, and the next phase feature as the key vector and the value vector, and sends them into the first cross-layer self-attention module, combines the output result vector of the first cross-layer self-attention module and the previous phase feature vector into the first residual connection layer normalization, and sends the output result of the first residual connection layer normalization into the first feedforward neural network, and then combines the output result of the first feedforward neural network and the output result of the first residual connection layer normalization into the second residual connection normalization. , the output result of the second residual link normalization is used as the query vector, the post-phase feature is used as the key vector and the value vector combination and input into the second cross-layer self-attention module, the output result of the second residual link normalization and the output result of the second cross-layer self-attention module are combined and input into the third residual connection normalization, the output result of the third residual connection normalization is sent to the second feedforward neural network, the output result of the second feedforward neural network and the output result of the third residual link layer normalization are combined and input into the fourth residual connection layer normalization, and the output result of the fourth residual connection layer normalization is the obtained change feature.

[0024] According to the method of the first aspect of the present invention, in step S4, generating prompt points of the change area according to the initial change pattern includes:

[0025] For the initial change spot, different instance spot regions are obtained by image connected domain calculation;

[0026] For each of the example pattern spots, use a grid of a predefined size to divide the example pattern spots;

[0027] The central pixel of each grid is selected as a cue point, and the foreground / background category to which the central pixel belongs represents the category of the cue point.

[0028] A second aspect of the present invention discloses an instance-level change detection system for collaboratively optimizing contour accuracy and positioning accuracy, the system comprising:

[0029] The first processing module is configured to collect high-resolution satellite data and preprocess the satellite data; perform change detection labeling on the preprocessed satellite data; and divide the satellite data and corresponding labels into a training set and a validation set;

[0030] The second processing module is configured to apply the training set to train a change detection model using a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint.

[0031] The third processing module is configured to adapt the instance-level evaluation index AP10 to the change detection result evaluation; apply the validation set and use the adapted evaluation index AP10 to evaluate the trained change detection model;

[0032] The fourth processing module is configured to input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots.

[0033] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in the instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy in any one of the first aspects of the present disclosure are implemented.

[0034] The fourth aspect of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy in any one of the first aspects of the present disclosure are implemented.

[0035] In summary, the solution proposed in the present invention can meet the dual requirements of high positioning accuracy and high contour accuracy, improve the integrity of large spots, reduce false detection and missed detection of small spots, enhance the boundary fit between the detected area and the real change area, and improve the contour accuracy and positioning accuracy of the change detection results in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1A flowchart of an instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to an embodiment of the present invention;

[0038] Figure 2 It is a scene for evaluating partial positioning accuracy and contour accuracy according to the background technology of the present invention;

[0039] Figure 3 It is a schematic diagram of the detection results of different particle size changes according to the background technology of the present invention;

[0040] Figure 4 A technical roadmap according to an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of an image processing flow according to an embodiment of the present invention;

[0042] Figure 6 A schematic diagram of calculating an instance-level loss function according to an embodiment of the present invention;

[0043] Figure 7 Schematic diagram of AP10 evaluation according to an embodiment of the present invention;

[0044] Figure 8 A SAM change detection framework integrating remote sensing cues according to an embodiment of the present invention;

[0045] Fig. 9 A change feature acquisition module based on cross-layer cross attention according to an embodiment of the present invention;

[0046] Fig.10 A schematic diagram of a SAM-oriented change detection prompt generation strategy according to an embodiment of the present invention;

[0047] Fig.11 A structural diagram of an instance-level change detection system for collaboratively optimizing contour accuracy and positioning accuracy according to an embodiment of the present invention;

[0048] Fig.12 The figure is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] A first aspect of the present invention discloses an instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy. Figure 1 Flow chart of an instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to an embodiment of the present invention. Figure 1 and Figure 4 As shown, the method includes:

[0051] Step S1, collecting high-resolution satellite data and preprocessing the satellite data; labeling the preprocessed satellite data for change detection; dividing the satellite data and the corresponding labels into a training set and a validation set;

[0052] Step S2, applying the training set, using a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint, to train a change detection model;

[0053] Step S3, adapting the instance-level evaluation index AP10 to the change detection result evaluation; applying the validation set, and using the adapted evaluation index AP10 to evaluate the trained change detection model;

[0054] Step S4: input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots.

[0055] In step S1, high-resolution satellite data is collected and preprocessed; change detection labels are applied to the preprocessed satellite data; and the satellite data and corresponding labels are divided into a training set and a validation set.

[0056] In some embodiments, in step S1, performing change detection tagging on the preprocessed satellite data includes:

[0057] Combined with the before and after phase images, ArcGIS is used as a sample marking tool to identify and outline the change spots; the change spots are used as samples with change detection labels; the marked change spots are for changes in cultivated land, woodland, gardens, grasslands, water bodies, roads, residential areas, greenhouses and construction areas outside the built-up area; the samples require the marking of change spots with an area greater than 150 square meters.

[0058] Specifically, high-resolution satellite data is collected and preprocessed, including: conducting land cover change research in the Guangdong-Hong Kong-Macao Greater Bay Area, and selecting domestic high-resolution satellite data from 2023 to 2024, including GF1, GF2, GF6, ZY3 and other satellites. The image is processed to a 2-meter resolution after strict data preprocessing operations, and only the RGB channel is retained. The image processing process is as follows: Figure 5 shown.

[0059] In step S2, the training set is applied to train a change detection model by adopting a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint.

[0060] In some embodiments, in step S2, the joint loss formed by the pixel-level loss and the instance-level loss with positioning accuracy constraint is:

[0061] loss = l change +l inst

[0062]

[0063] Among them, loss represents the joint loss; l change represents the pixel-level loss function based on dice calculation; l inst represents the instance loss function; L inst,i Indicates the prediction result; P inst,i Represents the label of the sample; dice() represents dice calculation; N represents the total number of instances.

[0064] Specifically, the instance-level loss function and instance-level accuracy evaluation index are used to constrain the positioning accuracy of the model. Both algorithms are decoupled from the change detection network and can be embedded in any change detection model, such as ChangeMamba (Chen et al., 2024) and SNUNet-CD (Fang et al., 2021). To this end, it is proposed to use the pixel-level change detection network FCCDN (Chen et al., 2022b) as the benchmark structure and embed instance-level constraints to optimize the model positioning accuracy.

[0065] At present, the constraint strategy of mainstream network learning still remains at the pixel-level constraint, lacking optimization for the positioning accuracy of change spots, resulting in serious misdetection and missed detection of small spots, which also limits the actual application ability of the algorithm. To this end, the project will design an instance-level change spot constraint algorithm, while retaining a certain contour accuracy, adding positioning accuracy constraints for each spot, and improving the change detection model's ability to locate change spots.

[0066] The overall schematic diagram of the instance-level loss function calculation used in this embodiment is as follows: Figure 6 As shown in the figure, in order to help the network optimize better, a joint loss consisting of a pixel-level loss that is easier to optimize and an instance-level loss with positioning accuracy constraints is used. Instance-level constraints extract the result and label spot instances through connected domain extraction, calculate the dice of the corresponding label instances and result instances in turn and accumulate them to obtain the instance-level loss function of the entire image. The final loss function can be expressed as follows:

[0067] loss = lchange +l inst

[0068]

[0069] Among them, loss represents the joint loss; l change represents the pixel-level loss function based on dice calculation; l inst represents the instance loss function; L inst,i Indicates the prediction result; P inst,i Represents the label of the sample; dice() represents dice calculation; N represents the total number of instances.

[0070] In order to improve the efficiency of loss function calculation and avoid the connected domain calculation operation in the loss calculation process, it is proposed to calculate the connected domain of the label in advance and not calculate the connected domain of the real-time output result to improve efficiency. In the loss calculation process, the circumscribed rectangle of each change instance in the label is directly used to delimit the loss calculation range, and the label and prediction results are calculated within this range to detect only the dice loss. In theory, this processing method is equivalent to the one-to-one loss calculation between direct instances, and this processing can also avoid the problem of matching between instances.

[0071] In step S3, the instance-level evaluation index AP10 is adapted to the change detection result evaluation; the validation set is applied, and the trained change detection model is evaluated using the adapted evaluation index AP10.

[0072] In some embodiments, in step S3, adapting the instance-level evaluation indicator AP10 to the change detection result evaluation includes:

[0073] AP10 distributes the accuracy evaluation of the entire image to each change patch instance, calculates the IoU between the predicted patch and the true value patch based on the instance, and regards the patch with IoU greater than 0.1 as a correctly predicted patch, i.e. TP; the patch with IoU less than 0.1 is regarded as a falsely detected patch, i.e. FP; the true value patch that is not detected is regarded as a missed detection patch, i.e. FN; if there are multiple predicted patches with IoU greater than 0.1 for the same true value, the one with the highest confidence is TP, and the others are FP.

[0074] Specifically, existing change detection algorithms basically use pixel-level indicators (IoU, kappa, etc.) to evaluate model accuracy. However, this pixel-by-pixel evaluation method does not have the ability to evaluate whether the model can accurately locate the changed area. In order to better evaluate the model's ability to evaluate the accuracy of spot positioning, this embodiment adapts the instance-level evaluation indicator AP10 to the change detection result evaluation. Instances are generated using change detection results, and evaluation is completed based on AP10.

[0075] Detailed diagram as follows Figure 7 As shown in the figure, a simple rectangular box represents a change instance. There are predicted spots P1, P2, P3 and true value spots G1 and G2. Among them, P3 does not meet the IoU with the true value greater than 0.1 and is considered FP. The IoU of P1 and P2 is greater than 0.1, but P2 has a higher confidence, so P1 is FP and P2 is TP. Since G2 is not detected, it is considered FN. Finally, the precision and recall are calculated based on the TP, FP, and FN of the spot instance:

[0076]

[0077] Different precisions are sampled according to the recall value, and the average of all sampled precisions is taken as AP10. AP10 uses patches as the basic unit to evaluate the change detection accuracy, while taking into account the evaluation of the positioning accuracy and contour accuracy of the predicted patches. The evaluation results meet the needs of a large number of change detection engineering applications.

[0078] In step S4, the initial change spots and the dual-phase image obtained by the trained change detection model are input into the SAM change detection algorithm to obtain the change spots.

[0079] In some embodiments, in step S4, inputting the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots includes:

[0080] Based on the trained change detection model, the initial change pattern is obtained, and the prompt points of the change area are generated according to the initial change pattern; the prompt points are passed through the prompt encoder of SAM to obtain the prompt features with the location information of the change area; the dual-phase image is input into the image encoder of SAM and encoded separately to obtain the dual-phase features; the dual-phase features are fused through the feature fusion module to obtain the change features; the change features and the prompt features are input into the decoder of SAM together to output the change pattern.

[0081] The feature fusion module fuses the bi-phase features based on the cross-layer cross attention module to obtain the change features;

[0082] like Fig. 9As shown, the cross-layer cross attention module inputs the previous phase feature and the subsequent phase feature into two cross-layer self-attention modules respectively, takes the previous phase feature as the query vector, and the subsequent phase feature as the key vector and the value vector, and sends them to the first cross-layer self-attention module, combines the output result vector of the first cross-layer self-attention module and the previous phase feature vector into the first residual connection layer normalization, and sends the output result of the first residual connection layer normalization into the first feedforward neural network, and then combines the output result of the first feedforward neural network and the output result of the first residual connection layer normalization into the second residual connection layer normalization. In the normalization, the output result of the second residual link normalization is used as the query vector, and the post-phase feature is input into the second cross-layer self-attention module as a combination of the key vector and the value vector. The output result of the second residual link normalization and the output result of the second cross-layer self-attention module are combined and input into the third residual connection normalization. The output result of the third residual connection normalization is sent to the second feedforward neural network. The output result of the second feedforward neural network and the output result of the third residual link layer normalization are combined and input into the fourth residual connection layer normalization. The output result of the fourth residual connection layer normalization is the obtained change feature.

[0083] Generating the prompt points of the change area according to the initial change pattern includes:

[0084] For the initial change spot, different instance spot regions are obtained by image connected domain calculation;

[0085] For each of the example pattern spots, use a grid of a predefined size to divide the example pattern spots;

[0086] The central pixel of each grid is selected as a cue point, and the foreground / background category to which the central pixel belongs represents the category of the cue point.

[0087] Specifically, the SAM change detection algorithm framework integrated with remote sensing prompts: Thanks to the massive image data training, SAM has demonstrated a strong segmentation capability for natural scene images, and can complete the contour extraction of objects in remote sensing images based on simple point prompts. Therefore, this embodiment intends to use SAM in conjunction with the output results of the change detection model to obtain accurate change spots and make up for the partial contour accuracy lost in the process of optimizing the spot positioning accuracy.

[0088] The adopted SAM change detection framework integrating remote sensing cues is as follows Figure 8As shown. Based on the trained change detection model, the initial change spots are obtained, and the prompt points of the change area are generated according to the initial change spots; the prompt points are passed through the prompt encoder of SAM to obtain prompt features with the location information of the change area; the dual-phase images are input into the image encoder of SAM for encoding respectively to obtain dual-phase features; the dual-phase features are fused through the feature fusion module to obtain the change features; the change features and the prompt features are input into the decoder of SAM together to output the change spots.

[0089] SAM is designed for single-phase tasks and cannot handle dual-phase change detection tasks. Therefore, a change feature extraction module for dual-phase SAM image features is designed to convert SAM decoding into a single-phase task. This part is the only trainable module in the entire framework. It is both a dual-phase feature fusion module and an adapter for remote sensing data (Houlsby et al., 2019, Pfeiffer et al., 2020). The feature fusion module fuses dual-phase features based on the cross-layer cross-attention module to obtain change features.

[0090] SAM supports a variety of prompt inputs, including points, rectangles, text, and masks. Rectangles and text do not match the output of the change detection model. At the same time, considering that the initial change spot contour accuracy of the model output is not high, the mask information of the spot is difficult to directly use as the prompt information of SAM. This embodiment intends to use point prompts to assist SAM in completing spot acquisition. The acquisition method of SAM prompt points is as follows: Fig.10 The specific process includes:

[0091] For the initial change spot, different instance spot regions are obtained by image connected domain calculation;

[0092] For each of the example pattern spots, use a grid of a predefined size to divide the example pattern spots;

[0093] The central pixel of each grid is selected as a cue point, and the foreground / background category to which the central pixel belongs represents the category of the cue point.

[0094] In summary, the solution proposed in the present invention can meet the dual requirements of high positioning accuracy and high contour accuracy, improve the integrity of large spots, reduce false detection and missed detection of small spots, enhance the boundary fit between the detected area and the real change area, and improve the contour accuracy and positioning accuracy of the change detection results in complex scenes.

[0095] A second aspect of the present invention discloses an instance-level change detection system with collaborative optimization of contour accuracy and positioning accuracy. Fig.11 FIG. 4 is a structural diagram of an instance-level change detection system according to an embodiment of the present invention that coordinates optimization of contour accuracy and positioning accuracy; Fig.11 As shown, the system 100 includes:

[0096] The first processing module 101 is configured to collect high-resolution satellite data and pre-process the satellite data; label the pre-processed satellite data for change detection; and divide the satellite data and the corresponding labels into a training set and a validation set;

[0097] The second processing module 102 is configured to apply the training set to train a change detection model using a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint.

[0098] The third processing module 103 is configured to adapt the instance-level evaluation index AP10 to the change detection result evaluation; apply the validation set and use the adapted evaluation index AP10 to evaluate the trained change detection model;

[0099] The fourth processing module 104 is configured to input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots.

[0100] According to the system of the second aspect of the present invention, the first processing module 101 is specifically configured as follows: performing change detection labeling on the pre-processed satellite data includes:

[0101] Combined with the before and after phase images, ArcGIS is used as a sample marking tool to identify and outline the change spots; the change spots are used as samples with change detection labels; the marked change spots are for changes in cultivated land, woodland, gardens, grasslands, water bodies, roads, residential areas, greenhouses and construction areas outside the built-up area; the samples require the marking of change spots with an area greater than 150 square meters.

[0102] Specifically, high-resolution satellite data is collected and preprocessed, including: conducting land cover change research in the Guangdong-Hong Kong-Macao Greater Bay Area, and selecting domestic high-resolution satellite data from 2023 to 2024, including GF1, GF2, GF6, ZY3 and other satellites. The image is processed to a 2-meter resolution after strict data preprocessing operations, and only the RGB channel is retained. The image processing process is as follows: Figure 5 shown.

[0103] According to the system of the second aspect of the present invention, the second processing module 102 is specifically configured such that the joint loss formed by the pixel-level loss and the instance-level loss with positioning accuracy constraint significance is:

[0104] loss = l change +l inst

[0105]

[0106] Among them, loss represents the joint loss; l change represents the pixel-level loss function based on dice calculation; l inst represents the instance loss function; L inst,i Indicates the prediction result; P inst,i Represents the label of the sample; dice() represents dice calculation; N represents the total number of instances.

[0107] Specifically, the instance-level loss function and instance-level accuracy evaluation index are used to constrain the positioning accuracy of the model. Both algorithms are decoupled from the change detection network and can be embedded in any change detection model, such as ChangeMamba (Chen et al., 2024) and SNUNet-CD (Fang et al., 2021). To this end, it is proposed to use the pixel-level change detection network FCCDN (Chen et al., 2022b) as the benchmark structure and embed instance-level constraints to optimize the model positioning accuracy.

[0108] At present, the constraint strategy of mainstream network learning still remains at the pixel-level constraint, lacking optimization for the positioning accuracy of change spots, resulting in serious misdetection and missed detection of small spots, which also limits the actual application ability of the algorithm. To this end, the project will design an instance-level change spot constraint algorithm, while retaining a certain contour accuracy, adding positioning accuracy constraints for each spot, and improving the change detection model's ability to locate change spots.

[0109] The overall schematic diagram of the instance-level loss function calculation used in this embodiment is as follows: Figure 6 As shown in the figure, in order to help the network optimize better, a joint loss consisting of a pixel-level loss that is easier to optimize and an instance-level loss with positioning accuracy constraints is used. Instance-level constraints extract the result and label spot instances through connected domain extraction, calculate the dice of the corresponding label instances and result instances in turn and accumulate them to obtain the instance-level loss function of the entire image. The final loss function can be expressed as follows:

[0110] loss = l change +l inst

[0111]

[0112] Among them, loss represents the joint loss; l change represents the pixel-level loss function based on dice calculation; l inst represents the instance loss function; L inst,i Indicates the prediction result; P inst,iRepresents the label of the sample; dice() represents dice calculation; N represents the total number of instances.

[0113] In order to improve the efficiency of loss function calculation and avoid the connected domain calculation operation in the loss calculation process, it is proposed to calculate the connected domain of the label in advance and not calculate the connected domain of the real-time output result to improve efficiency. In the loss calculation process, the circumscribed rectangle of each change instance in the label is directly used to delimit the loss calculation range, and the label and prediction results are calculated within this range to detect only the dice loss. In theory, this processing method is equivalent to the one-to-one loss calculation between direct instances, and this processing can also avoid the problem of matching between instances.

[0114] According to the system of the second aspect of the present invention, the third processing module 103 is specifically configured as follows: the adapting the instance-level evaluation indicator AP10 to the change detection result evaluation includes:

[0115] AP10 distributes the accuracy evaluation of the entire image to each change patch instance, calculates the IoU between the predicted patch and the true value patch based on the instance, and regards the patch with IoU greater than 0.1 as a correctly predicted patch, i.e. TP; the patch with IoU less than 0.1 is regarded as a falsely detected patch, i.e. FP; the true value patch that is not detected is regarded as a missed detection patch, i.e. FN; if there are multiple predicted patches with IoU greater than 0.1 for the same true value, the one with the highest confidence is TP, and the others are FP.

[0116] Specifically, existing change detection algorithms basically use pixel-level indicators (IoU, kappa, etc.) to evaluate model accuracy. However, this pixel-by-pixel evaluation method does not have the ability to evaluate whether the model can accurately locate the changed area. In order to better evaluate the model's ability to evaluate the accuracy of spot positioning, this embodiment adapts the instance-level evaluation indicator AP10 to the change detection result evaluation. Instances are generated using change detection results, and evaluation is completed based on AP10.

[0117] Detailed diagram as follows Figure 7 As shown in the figure, a simple rectangular box represents a change instance. There are predicted spots P1, P2, P3 and true value spots G1 and G2. Among them, P3 does not meet the IoU with the true value greater than 0.1 and is considered FP. The IoU of P1 and P2 is greater than 0.1, but P2 has a higher confidence, so P1 is FP and P2 is TP. Since G2 is not detected, it is considered FN. Finally, the precision and recall are calculated based on the TP, FP, and FN of the spot instance:

[0118]

[0119] Different precisions are sampled according to the recall value, and the average of all sampled precisions is taken as AP10. AP10 uses patches as the basic unit to evaluate the change detection accuracy, while taking into account the evaluation of the positioning accuracy and contour accuracy of the predicted patches. The evaluation results meet the needs of a large number of change detection engineering applications.

[0120] According to the system of the second aspect of the present invention, the fourth processing module 104 is specifically configured to input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots, including:

[0121] Based on the trained change detection model, the initial change pattern is obtained, and the prompt points of the change area are generated according to the initial change pattern; the prompt points are passed through the prompt encoder of SAM to obtain the prompt features with the location information of the change area; the dual-phase image is input into the image encoder of SAM and encoded separately to obtain the dual-phase features; the dual-phase features are fused through the feature fusion module to obtain the change features; the change features and the prompt features are input into the decoder of SAM together to output the change pattern.

[0122] The feature fusion module fuses the bi-phase features based on the cross-layer cross attention module to obtain the change features;

[0123] like Fig. 9 As shown, the cross-layer cross attention module inputs the previous phase feature and the subsequent phase feature into two cross-layer self-attention modules respectively, takes the previous phase feature as the query vector, and the subsequent phase feature as the key vector and the value vector, and sends them to the first cross-layer self-attention module, combines the output result vector of the first cross-layer self-attention module and the previous phase feature vector into the first residual connection layer normalization, and sends the output result of the first residual connection layer normalization into the first feedforward neural network, and then combines the output result of the first feedforward neural network and the output result of the first residual connection layer normalization into the second residual connection layer normalization. In the normalization, the output result of the second residual link normalization is used as the query vector, and the post-phase feature is input into the second cross-layer self-attention module as a combination of the key vector and the value vector. The output result of the second residual link normalization and the output result of the second cross-layer self-attention module are combined and input into the third residual connection normalization. The output result of the third residual connection normalization is sent to the second feedforward neural network. The output result of the second feedforward neural network and the output result of the third residual link layer normalization are combined and input into the fourth residual connection layer normalization. The output result of the fourth residual connection layer normalization is the obtained change feature.

[0124] Generating the prompt points of the change area according to the initial change pattern includes:

[0125] For the initial change spot, different instance spot regions are obtained by image connected domain calculation;

[0126] For each of the example pattern spots, use a grid of a predefined size to divide the example pattern spots;

[0127] The central pixel of each grid is selected as a cue point, and the foreground / background category to which the central pixel belongs represents the category of the cue point.

[0128] Specifically, the SAM change detection algorithm framework integrated with remote sensing prompts: Thanks to the massive image data training, SAM has demonstrated a strong segmentation capability for natural scene images, and can complete the contour extraction of objects in remote sensing images based on simple point prompts. Therefore, this embodiment intends to use SAM in conjunction with the output results of the change detection model to obtain accurate change spots and make up for the partial contour accuracy lost in the process of optimizing the spot positioning accuracy.

[0129] The adopted SAM change detection framework integrating remote sensing cues is as follows Figure 8 As shown. Based on the trained change detection model, the initial change spots are obtained, and the prompt points of the change area are generated according to the initial change spots; the prompt points are passed through the prompt encoder of SAM to obtain prompt features with the location information of the change area; the dual-phase images are input into the image encoder of SAM for encoding respectively to obtain dual-phase features; the dual-phase features are fused through the feature fusion module to obtain the change features; the change features and the prompt features are input into the decoder of SAM together to output the change spots.

[0130] SAM is designed for single-phase tasks and cannot handle dual-phase change detection tasks. Therefore, a change feature extraction module for dual-phase SAM image features is designed to convert SAM decoding into a single-phase task. This part is the only trainable module in the entire framework. It is both a dual-phase feature fusion module and an adapter for remote sensing data (Houlsby et al., 2019, Pfeiffer et al., 2020). The feature fusion module fuses dual-phase features based on the cross-layer cross-attention module to obtain change features.

[0131] SAM supports a variety of prompt inputs, including points, rectangles, text, and masks. Rectangles and text do not match the output of the change detection model. At the same time, considering that the initial change spot contour accuracy of the model output is not high, the mask information of the spot is difficult to directly use as the prompt information of SAM. This embodiment intends to use point prompts to assist SAM in completing spot acquisition. The acquisition method of SAM prompt points is as follows: Fig.10 The specific process includes:

[0132] For the initial change spot, different instance spot regions are obtained by image connected domain calculation;

[0133] For each of the example pattern spots, use a grid of a predefined size to divide the example pattern spots;

[0134] The central pixel of each grid is selected as a cue point, and the foreground / background category to which the central pixel belongs represents the category of the cue point.

[0135] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy disclosed in any one of the first aspects of the present invention are implemented.

[0136] Fig.12 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Fig.12 As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0137] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] The fourth aspect of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy disclosed in any one of the first aspects of the present invention are implemented.

[0139] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. An instance-level change detection method with collaborative optimization of contour accuracy and positioning accuracy, characterized in that: The method comprises: Step S1, collecting high-resolution satellite data and preprocessing the satellite data; labeling the preprocessed satellite data for change detection; dividing the satellite data and the corresponding labels into a training set and a validation set; Step S2, applying the training set, using a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint, to train a change detection model; Step S3, adapting the instance-level evaluation index AP10 to the change detection result evaluation; applying the validation set, and using the adapted evaluation index AP10 to evaluate the trained change detection model; Step S4: input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots.

2. The instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to claim 1, characterized in that: In step S1, performing change detection tagging on the preprocessed satellite data includes: Combined with the before and after phase images, ArcGIS is used as a sample marking tool to identify and outline the change spots; the change spots are used as samples with change detection labels; the marked change spots are for changes in cultivated land, woodland, gardens, grasslands, water bodies, roads, residential areas, greenhouses and construction areas outside the built-up area; the samples require the marking of change spots with an area greater than 150 square meters.

3. The instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to claim 1, characterized in that: In step S2, the joint loss composed of the pixel-level loss and the instance-level loss with positioning accuracy constraint is: loss=l change +l inst Among them, loss represents the joint loss; l change represents the pixel-level loss function based on dice calculation; l inst represents the instance loss function; L inst,i Indicates the prediction result; P inst,i Represents the label of the sample; dice() represents dice calculation; N represents the total number of instances.

4. The instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to claim 1, characterized in that: In step S3, adapting the instance-level evaluation indicator AP10 to the change detection result evaluation includes: AP10 distributes the accuracy evaluation of the entire image to each change patch instance, calculates the IoU between the predicted patch and the true value patch based on the instance, and regards the patch with IoU greater than 0.1 as a correctly predicted patch, i.e. TP; the patch with IoU less than 0.1 is regarded as a falsely detected patch, i.e. FP; the true value patch that is not detected is regarded as a missed detection patch, i.e. FN; if there are multiple predicted patches with IoU greater than 0.1 for the same true value, the one with the highest confidence is TP, and the others are FP.

5. The instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to claim 1, characterized in that: In step S4, the initial change spots and the dual-phase image obtained by the trained change detection model are input into the SAM change detection algorithm to obtain the change spots, which includes: Based on the trained change detection model, the initial change pattern is obtained, and the prompt points of the change area are generated according to the initial change pattern; the prompt points are passed through the prompt encoder of SAM to obtain the prompt features with the location information of the change area; the dual-phase image is input into the image encoder of SAM and encoded separately to obtain the dual-phase features; the dual-phase features are fused through the feature fusion module to obtain the change features; the change features and the prompt features are input into the decoder of SAM together to output the change pattern.

6. The instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to claim 5, characterized in that: In the step S4, the feature fusion module fuses the bi-phase features based on the cross-layer cross attention module to obtain the change feature; The cross-layer cross attention module inputs the previous phase feature and the next phase feature into two cross-layer self-attention modules respectively, takes the previous phase feature as the query vector, and the next phase feature as the key vector and the value vector, and sends them into the first cross-layer self-attention module, combines the output result vector of the first cross-layer self-attention module and the previous phase feature vector into the first residual connection layer normalization, and sends the output result of the first residual connection layer normalization into the first feedforward neural network, and then combines the output result of the first feedforward neural network and the output result of the first residual connection layer normalization into the second residual connection normalization. , the output result of the second residual link normalization is used as the query vector, the post-phase feature is used as the key vector and the value vector combination and input into the second cross-layer self-attention module, the output result of the second residual link normalization and the output result of the second cross-layer self-attention module are combined and input into the third residual connection normalization, the output result of the third residual connection normalization is sent to the second feedforward neural network, the output result of the second feedforward neural network and the output result of the third residual link layer normalization are combined and input into the fourth residual connection layer normalization, and the output result of the fourth residual connection layer normalization is the obtained change feature.

7. The instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy according to claim 5, characterized in that: In the step S4, generating prompt points of the change area according to the initial change pattern includes: For the initial change spot, different instance spot regions are obtained by image connected domain calculation; For each of the example pattern spots, use a grid of a predefined size to divide the example pattern spots; The central pixel of each grid is selected as a cue point, and the foreground / background category to which the central pixel belongs represents the category of the cue point.

8. An instance-level change detection system for collaborative optimization of contour accuracy and positioning accuracy, characterized in that: The system comprises: The first processing module is configured to collect high-resolution satellite data and preprocess the satellite data; perform change detection labeling on the preprocessed satellite data; and divide the satellite data and corresponding labels into a training set and a validation set; The second processing module is configured to apply the training set to train a change detection model using a joint loss consisting of a pixel-level loss and an instance-level loss with a positioning accuracy constraint. The third processing module is configured to adapt the instance-level evaluation index AP10 to the change detection result evaluation; apply the validation set and use the adapted evaluation index AP10 to evaluate the trained change detection model; The fourth processing module is configured to input the initial change spots and the dual-phase image obtained by the trained change detection model into the SAM change detection algorithm to obtain the change spots.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the instance-level change detection method for collaboratively optimizing contour accuracy and positioning accuracy described in any one of claims 1 to 7 are implemented.

Images