Large model-based open category remote sensing change detection method and related system

Abstract

The present application belongs to the field of remote sensing image intelligent interpretation, and discloses an open category remote sensing change detection method based on a large model and a related system. The present application effectively breaks through the existing remote sensing change detection method, especially the strong dependence of the semantic change detection method on the closed change category system and large-scale artificial annotation samples, by introducing a large model driven graphic-text joint modeling mechanism. After obtaining double-time remote sensing images, the change detection problem is improved from the traditional pixel-level difference judgment to object-level change understanding with semantic orientation through slicing and text description generation, thereby reducing the demand for accurate pixel annotation from the source. On this basis, a suspected change object set is obtained through object extraction, so that the detection process focuses on the potential change area, reduces irrelevant background interference, and is conducive to improving the detection efficiency and stability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent interpretation of remote sensing images, specifically involving an open-category remote sensing change detection method and related systems based on a large model. Background Technology

[0002] Remote sensing change detection refers to the comparative analysis of remote sensing images of the same geographic area acquired at different times, using image processing or mathematical modeling methods to identify changes in ground features or natural and anthropogenic phenomena over time. These changes include not only variations in the location and spatial extent of ground features, but also changes in their properties and states. Change detection technology provides crucial technical support for understanding the evolutionary patterns of ground features and analyzing the dynamic development processes under the influence of natural or anthropogenic factors at both geospatial and temporal scales.

[0003] Based on whether the change detection results include land cover category information, change detection methods can be divided into binary change detection (BCD) and semantic change detection (SCD). BCD only determines whether a pixel or region has changed, while SCD not only identifies the changed region but also further describes the semantic direction of the change, i.e., "from which land cover category to another," providing more refined and richer change information.

[0004] Early research on remote sensing change detection mainly focused on traditional methods. These methods typically acquire change features through image differencing, ratio calculation, principal component analysis, or manually designed features, and then combine them with threshold segmentation or machine learning methods to distinguish between changes and invariances. However, in large-scale, high-resolution remote sensing image scenarios, traditional methods generally suffer from limited feature representation capabilities, sensitivity to noise and registration errors, and insufficient generalization ability.

[0005] With the continuous growth of remote sensing data and the development of deep learning technology, change detection methods based on deep learning have gradually become a research hotspot. These methods typically employ deep neural networks to map bi-temporal or multi-temporal remote sensing images end-to-end into a high-dimensional feature space, and then identify changed regions based on deep features. Compared to traditional methods, deep learning-based change detection has significant advantages in reducing manual intervention, improving detection accuracy, and enhancing robustness and generalization ability.

[0006] In deep learning change detection research, BCD (Browser-Cut Mapping) is the earliest and most mature approach. Early BCD methods were mostly scene-level detection, classifying changes in bi-temporal image patches to determine whether they were changed. However, these methods only yielded coarse-grained results and struggled to depict the fine boundaries of changed regions. With the advent of fully convolutional neural networks, pixel-level BCD methods gradually became mainstream, directly outputting changed patches for refined change detection. Subsequently, researchers have continuously improved feature extraction and temporal modeling capabilities by refining network structures, introducing attention mechanisms, and incorporating Transformer architectures. Furthermore, object-level BCD methods, drawing inspiration from object detection, predict the bounding rectangle of changed regions, showing advantages in detecting small-change objects and mitigating registration errors. However, their application remains limited due to the difficulty in directly obtaining fine-grained patches.

[0007] While the BCD method holds an advantage in terms of research volume and application maturity, its detection results do not include land cover category attributes, making it difficult to meet the current practical needs of large-scale, refined remote sensing change analysis. Therefore, semantic change detection (SCD) has gradually gained attention. SCD research mainly focuses on obtaining the semantic categories of land cover in the changed area in previous and subsequent time phases through joint or direct modeling, thereby characterizing the direction of change. Existing research has proposed various SCD implementation strategies, among which directly outputting semantic change results and jointly learning land cover classification and change detection have become mainstream.

[0008] However, existing deep learning SCD methods typically rely on closed-class settings, meaning the change categories of the model are predefined during the training phase. When the application scenario or target category changes, it is often necessary to reconstruct labeled samples and retrain the model, resulting in high application costs, insufficient flexibility, and difficulty in adapting to complex and ever-changing real-world change detection needs.

[0009] In recent years, the development of large-scale model technology has provided new possibilities for change detection with low data costs and high generalization capabilities. With the rapid development of large-scale pre-trained models in the fields of vision and language, large-scale visual models trained on massive remote sensing data have gradually emerged in the field of remote sensing, demonstrating stronger feature representation capabilities in downstream tasks. In change detection research, some works have attempted to use large-scale visual models to extract remote sensing image features to improve detection accuracy, but existing methods are mostly focused on a single visual modality and have not yet effectively incorporated the high-level semantic knowledge contained in large-scale language models.

[0010] Overall, how to overcome the differences between natural scenes and remote sensing image domains, effectively integrate the semantic understanding capabilities of multimodal large models with the temporal dimension analysis in remote sensing change detection tasks, and achieve semantic change detection with open-category capabilities, remains a key problem that urgently needs to be solved in current research. Summary of the Invention

[0011] The purpose of this invention is to overcome the problem that existing remote sensing change detection methods, especially semantic change detection, rely on closed categories and a large amount of labeled data, making it difficult to integrate the semantic knowledge of large models to achieve refined change detection with open category capabilities, low cost and high generalization. This invention provides an open category remote sensing change detection method and related system based on a large model.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an open-category remote sensing change detection method based on a large model, comprising the following steps: Acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions for the dual-temporal sample pairs; Object extraction was performed on the dual-temporal samples to obtain a set of suspected changed objects; Extract text features of change categories from the text description, embed object features and unify dimensions for the suspected change object set, align and fuse the change category text features with the object feature embedding to obtain the final image-text relevance matrix; Based on the final image-text correlation matrix, a set of true image-text matches is constructed, and an open-category remote sensing change detection model is built.

[0013] A further improvement of this invention lies in the following method for acquiring dual-temporal remote sensing images, slicing the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extracting text descriptions for the dual-temporal sample pairs: Acquire dual-temporal remote sensing images of the same geographic area; Spatial registration is performed on the dual-temporal remote sensing images. After registration, the images are sliced ​​according to the required window size to obtain dual-temporal sample pairs. The slice index, original image location and geographic coordinate mapping relationship are recorded. A multimodal large model is used to process dual-temporal sample pairs to generate text descriptions.

[0014] A further improvement of this invention lies in the following method for extracting objects from dual-temporal samples to obtain a set of suspected changed objects: Two-phase samples are obtained, and a large visual model is used to process the two-phase samples to obtain object candidates and multi-scale features. Based on object candidates and multi-scale features, generate a set of object candidate masks; The candidate mask set is matched across time to obtain candidate objects; Extract the candidate objects that have changed as a set of suspected changed objects.

[0015] A further improvement of this invention lies in extracting the text features of the change categories in the text description, embedding object features and unifying dimensions for the suspected change object set, and aligning and fusing the change category text features with the object feature embeddings to obtain the final image-text relevance matrix. The specific method is as follows: Construct a set of change category text prompts, and use the set of change category text prompts to extract change category text features from the text descriptions; Spatial dimension compression and deformation are performed on the image features of each changed object in the suspected changed object set, and position encoding is added to obtain the object feature embedding; A cross-attention mechanism is used to align and fuse the text features of the change category with the object feature embeddings. Obtain the image-text correlation matrix between the fused change category text features and object feature embeddings, and normalize the column dimensions of the image-text correlation matrix to obtain the final image-text correlation matrix.

[0016] A further improvement of this invention lies in the following method for constructing an open-category remote sensing change detection model by building a true matching set of images and texts based on the final image-text correlation matrix: Based on a pre-constructed open-category remote sensing change detection model using dual-temporal sample pairs; Obtain the final image-text relevance matrix and construct a set of true image-text matches; Calculate the loss for matching images to text and the loss for matching text to images to obtain the final total loss for contrastive learning; The pre-built open-category remote sensing change detection model is trained using the final contrastive learning total loss to obtain the open-category remote sensing change detection model.

[0017] Secondly, the present invention provides an open-category remote sensing change detection system based on a large model, comprising: The data acquisition module is used to acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions of the dual-temporal sample pairs. The variable object extraction module is used to extract objects from dual-temporal samples to obtain a set of suspected variable objects; The object classification module is used to extract text features of change categories from text descriptions, perform object feature embedding and dimension unification on the suspected object set, and align and fuse the text features of change categories with the object feature embeddings to obtain the final image-text relevance matrix. The model building module is used to construct a set of true matching images and texts based on the final image-text correlation matrix, and to build an open-category remote sensing change detection model.

[0018] A further improvement of this invention is that the data acquisition module's function is implemented through the following method: Acquire dual-temporal remote sensing images of the same geographic area; Spatial registration is performed on the dual-temporal remote sensing images. After registration, the images are sliced ​​according to the required window size to obtain dual-temporal sample pairs. The slice index, original image location and geographic coordinate mapping relationship are recorded. A multimodal large model is used to process dual-temporal sample pairs to generate text descriptions.

[0019] A further improvement of this invention is that the function of the changing object extraction module is implemented through the following method: Two-phase samples are obtained, and a large visual model is used to process the two-phase samples to obtain object candidates and multi-scale features. Based on object candidates and multi-scale features, generate a set of object candidate masks; The candidate mask set is matched across time to obtain candidate objects; Extract the candidate objects that have changed as a set of suspected changed objects.

[0020] A further improvement of this invention is that the function of the variable object classification module is implemented through the following method: Construct a set of change category text prompts, and use the set of change category text prompts to extract change category text features from the text descriptions; Spatial dimension compression and deformation are performed on the image features of each changed object in the suspected changed object set, and position encoding is added to obtain the object feature embedding; A cross-attention mechanism is used to align and fuse the text features of the change category with the object feature embeddings. Obtain the image-text correlation matrix between the fused change category text features and object feature embeddings, and normalize the column dimensions of the image-text correlation matrix to obtain the final image-text correlation matrix.

[0021] A further improvement of this invention is that the functionality of the model building module is implemented through the following method: Based on a pre-constructed open-category remote sensing change detection model using dual-temporal sample pairs; Obtain the final image-text relevance matrix and construct a set of true image-text matches; Calculate the loss for matching images to text and the loss for matching text to images to obtain the final total loss for contrastive learning; The pre-built open-category remote sensing change detection model is trained using the final contrastive learning total loss to obtain the open-category remote sensing change detection model.

[0022] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of an open-category remote sensing change detection method based on a large model.

[0023] Fourthly, the present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of an open-category remote sensing change detection method based on a large model.

[0024] Compared with the prior art, the present invention has the following beneficial effects: This invention effectively overcomes the strong dependence of existing remote sensing change detection methods, especially semantic change detection methods, on closed change category systems and large-scale manually labeled samples by introducing a large-model-driven image-text joint modeling mechanism. After acquiring dual-temporal remote sensing images, it first generates change detection by tiling and text description, elevating the problem from traditional pixel-level difference judgment to object-level change understanding with semantic direction, thus reducing the need for precise pixel labeling from the source. This invention obtains a set of suspected changed objects through object extraction, focusing the detection process on potential change areas, reducing irrelevant background interference, and improving detection efficiency and stability. This invention utilizes the dimensional unification and alignment fusion of change category text features and object feature embeddings to explicitly introduce the general semantic knowledge contained in the large model into the remote sensing change detection process. This enables the model to complete change category discrimination based on semantic similarity under conditions of no or few samples, thereby breaking away from the limitations of traditional methods on fixed category sets and possessing a natural open category expansion capability. By constructing a final image-text correlation matrix and forming a true image-text matching set based on it to complete model training, this not only reduces the reliance on pixel-by-pixel fine-grained annotation data but also enhances the model's generalization ability across different sensors, regions, and land cover types. In conclusion, this invention, while ensuring the precision of change detection, achieves adaptive identification of unknown change categories, significantly reduces the cost of data construction and manual annotation, and improves the model's adaptability and transferability to complex and diverse remote sensing scenarios. Therefore, it provides an effective solution for constructing a remote sensing change detection technology with open-category capabilities, low cost, and high generalization. Attached Figure Description

[0025] Figure 1 This is a flowchart of the present invention; Figure 2 This is a system diagram of the present invention; Figure 3 This is a schematic diagram of an open-category semantic change detection framework based on image-text feature alignment; Figure 4 This is a schematic diagram of a text-image feature alignment system based on cross-attention. Figure 5 A schematic diagram illustrating the calculation of the correlation between category features and object features; Figure 6 A schematic diagram illustrating the construction process of an open-category semantic change detection application; Figure 7 This is a flowchart of Example 6; Figure 8 This is a schematic diagram of the system in Example 7. Detailed Implementation

[0026] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0027] Example 1: See Figure 1 The open-category remote sensing change detection method based on a large model includes the following steps: S1. Acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions for the dual-temporal sample pairs.

[0028] S2, extract objects from the dual-temporal samples to obtain a set of suspected changed objects.

[0029] S3. Extract the text features of the change category in the text description, perform object feature embedding and dimension unification on the suspected change object set, and align and fuse the change category text features with the object feature embedding to obtain the final image-text relevance matrix.

[0030] S4. Construct a set of true matching images and texts based on the final image-text correlation matrix, and build an open-category remote sensing change detection model.

[0031] Example 2: See Figure 2 Open-category remote sensing change detection systems based on large models include: The data acquisition module is used to acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions of the dual-temporal sample pairs.

[0032] The variable object extraction module is used to extract objects from dual-temporal samples to obtain a set of suspected variable objects.

[0033] The object classification module is used to extract text features of change categories from text descriptions, embed object features and unify dimensions for suspected change object sets, and align and fuse the change category text features with the object feature embeddings to obtain the final image-text relevance matrix.

[0034] The model building module is used to construct a set of true matching images and texts based on the final image-text correlation matrix, and to build an open-category remote sensing change detection model.

[0035] Example 3: In this embodiment, dual-temporal remote sensing images are acquired, sliced, and dual-temporal sample pairs are obtained. The specific method for extracting text descriptions of the dual-temporal sample pairs is as follows: Step 1: Acquire dual-temporal remote sensing images of the same geographic area. With dual-temporal remote sensing images ,in For the previous phase, This is a later time phase. The images may come from the Gaofen series or Ziyuan series satellites, such as: Gaofen-1 (GF-1), Gaofen-2 (GF-2), Gaofen-6 (GF-6), Ziyuan-3 (ZY-3), etc.

[0036] Step 2: Process the dual-temporal remote sensing images With dual-temporal remote sensing images Perform spatial registration to ensure pixel-level correspondence between the two temporal phases; then apply the registration to a fixed window size (e.g., ...). , , The slices are dual-temporal sample pairs. It also records the slice index, the original image location, and the mapping relationship between geographic coordinates.

[0037] Step 3: To improve open-category understanding, generate text descriptions for samples or regions. Text descriptions can be generated by multimodal large models (such as GPT-5, Gemini, Qwen-VL, DeepSeek, etc.), and the descriptions include land cover categories, scene structure, and possible change types. Text description generation methods include, but are not limited to: generating scene-level text from the entire image; generating object-level text from candidate changed objects / segmentation masks; and generating text from previous temporal images. and subsequent phase By comparing the changes, generating explanatory texts of the differences, and finally obtaining standardized dual-temporal sample pairs, a standardized two-phase sample pair is obtained. and text description .

[0038] Example 3: This embodiment utilizes a large visual model to extract object-level candidates from dual-temporal images and obtains a set of suspected changed objects through change discrimination. The key lies in first segmenting / proposing objects, and then determining whether they have changed, thereby reconstructing change detection from end-to-end pixel discrimination into an interpretable object-level process. The specific method for extracting objects from dual-temporal samples to obtain the set of suspected changed objects is as follows: Step 1: Load large-scale visual models, including but not limited to: 1. SAM (Segment Anything Model) or FastSAM (Fast Segment Anything Model) for object mask generation; 2. DINO (Self-Distillation Unsupervised Visual Representation Model) / remote sensing visual foundation models (such as GFM, Geospatial Foundation Model) for feature extraction. Combine dual-temporal samples... Input a large visual model to obtain object candidates and multi-scale features.

[0039] Step 2: Object Candidate Generation (Mask / Region Proposal) Generate a set of object candidate masks for each temporal phase:

[0040] in, It is a binary mask (pixel values ​​are 0 / 1). The number of candidates is specified. Candidate generation can be driven by point hints / box hints / mesh hints to use SAM output masks, multi-scale sliding windows and candidate merging strategies, or morphological and connected component filtering to remove excessively small noise regions.

[0041] Step 3: Set up the candidate mask for the object. With object candidate mask set Perform cross-time matching to obtain candidate pairs Matching is based on: 1. the IoU (Intersection over Union) or center point distance of the mask bounding box; 2. visual feature similarity (cosine similarity); 3. shape consistency constraints. A variation score is calculated for each candidate pair. ,when The object is identified as changing: ;in For visual feature extraction networks, Indicates mask clipping / weighting. The threshold value is used.

[0042] Step 4: Output the set of suspected changed objects. The candidate objects determined to have changed will be output as a set of suspected changed objects.

[0043] in, For the bounding box or polygon outline of the object. The number of objects that change.

[0044] Example 4: See Figure 3 This embodiment is for the obtained set of suspected changed objects. This invention conducts open-category semantic discrimination, outputting the semantic change category and direction information for each changed object. To overcome the differences in feature size and semantic dimension between the visual features of remote sensing images and the semantic features of text, this invention constructs an image-text feature alignment based on cross-attention. The CLIP (Contrastive Language–Image Pretraining) framework is used to calculate the correlation between the changed object and the change category text, forming an image-text contrast learning constraint to achieve semantic change detection. In open-category applications, only the change category text prompts need to be adjusted according to task requirements to identify new category changes not covered by the training samples, without the need to supplement labeled samples or retrain the model. Specifically, the method for extracting the change category text features from the text description, embedding and unifying the dimensions of the suspected changed object set, and aligning and fusing the change category text features with the object feature embeddings to obtain the final image-text correlation matrix is ​​as follows: Step 1: Construct a set of text prompts for change categories Each of them This includes text prompts that indicate changes in direction, such as from water to bridges, or from low vegetation to buildings. These prompts can be provided manually or automatically expanded by a large language model using synonyms, hyponyms, and attribute descriptions (e.g., road → highway / street / expressway, bridge → elevated / interchange, etc.), thus forming an open-category change prompt library. The input text encoder (which can be implemented by a large language model or a vision-language model text encoder) extracts the text features of the change category, resulting in:

[0045] in, For the number of change categories (directions), The feature dimensions are encoded for each text. Simultaneously, the global text descriptions of the preceding and following time-phase images are encoded separately to obtain the global text features for the preceding and following time-phase images: The aforementioned text features are used for subsequent image-text feature alignment and semantic injection.

[0046] Step two, to achieve alignment between the features of the changed object and the text features, firstly, for each changed object... The image features are spatially compressed and deformed, and positional encoding is added to obtain the object feature embedding:

[0047] in, For the number of potentially changing objects, For spatial pooling, This is a feature transformation (compressing the spatial dimension into a vector). Spatial location is encoded. Then, a multilayer perceptron maps the object embeddings to the text feature dimension, ensuring that each object embedding satisfies: To facilitate subsequent batch calculations, all changed object features are stacked row-wise to form an object feature matrix: , its first Row corresponding object A unified dimensional feature representation.

[0048] Step 3, regarding the characteristics of the changing object Text features After unifying the dimensions, cross attention (CoA) is used to achieve image-text feature alignment and semantic fusion. Its basic calculation form is as follows:

[0049] in, These are query, key, and value vectors, respectively. For transpose, The output features are aligned. Different features are used to construct features at different stages. This allows for the injection of textual semantics into object representations: on the one hand, alignment of earlier-time text features with object features can be achieved; on the other hand, alignment of later-time text features with object features can be achieved; alternatively, alignment of combined earlier- and later-time text features with object features can also be used. See also Figure 4 After cross-attention fusion, the aligned semantic feature representation of the changed object is obtained: This feature simultaneously incorporates visual information about the changing object and deep semantic information injected by a large language model, providing support for subsequent open-category classification. Step four: After completing the image-text alignment, this invention calculates the correlation between the text features of the change category and the features of the change object based on the CLIP architecture, constructs an image-text correlation matrix through matrix multiplication, and normalizes it in the column dimensions:

[0050] in, , This indicates normalization of the matrix column dimensions. Because... Each column corresponds to a probability distribution of matching the changed object with all changed category texts. Therefore, for the first column... For each changed object, its classification result is the change category corresponding to the most relevant text prompt in that column, thus obtaining the semantic change direction information of that object. See also Figure 5 Since the change category text prompt itself contains a directional description from something to something, this step can directly output the open category semantic change detection results.

[0051] Example 5: In this embodiment, the specific method for constructing an open-category remote sensing change detection model by building a true matching set of images and texts based on the final image-text correlation matrix is ​​as follows: Step 1: Construct training and validation samples on publicly available change detection / semantic change detection datasets, such as SECOND, Hi-UCD, and HRSCD. The training phase may include ground truth masks of changes. (Binary change or semantic change annotation), no truth value input is required during the inference phase.

[0052] Step two, during the model training phase, uses the CLIP contrastive learning loss function to constrain model learning. A set of true text-image matches is constructed based on the ground truth labels. ,in Indicates the first The object of change and the first The changed text prompt is a true match, corresponding to The true value is 1, and the rest are 0. First, calculate the loss for image-to-text matching:

[0053] Then, the loss for matching text to images is calculated:

[0054] The final total learning loss is:

[0055] The loss function described above promotes a higher correlation between the true matching changed objects and the changed text prompts, enabling the model to learn stable image-text alignment and open category change discrimination capabilities.

[0056] Example 6: See Figure 6 and Figure 7 In the application of open-category semantic change detection, this step combines the already trained semantic change detection model framework, adjusts the change category text prompts to achieve open-category semantic change detection, and uses the open-category semantic change results for evaluation, cross-dataset validation, and output of surface application products.

[0057] Step one: In the application process, change text prompts are automatically constructed for open-category land features not present in the training sample set. Then, based on the text encoding capabilities of the large language model, text features for changes in the new category are extracted. These new category change text features are concatenated with known category change text features and input into the CLIP contrastive learning framework to obtain the similarity between each category change and the potential changed object, thereby achieving open-category semantic change detection. In this process, there is no need to supplement and label unknown new category samples or retrain the model; through deep semantic feature mining of the large language model, inference about open-category land feature changes is quickly achieved. The specific process is shown in the figure below: Step two involves evaluating the results of change detection and semantic change detection, with metrics including but not limited to: F1 score; IoU (Intersection over Union); OA (Overall Accuracy); and can be further extended to include category-level mIoU, change direction accuracy, etc.

[0058] Step 3: To verify the generalization ability of the present invention under different regions, different sensors, and different annotation systems, cross-validation experiments were conducted on publicly available semantic change detection datasets such as SECOND, Hi-UCD, and HRSCD. Specifically, this included "cross-dataset transfer validation," which involves training the model on dataset A and then testing it directly on dataset B to evaluate its robustness across scenarios and domains; and "cross-classification level validation," which involves training the model on a certain class level (such as a coarse-grained class system) in dataset A and testing it on other class levels (such as a fine-grained class system or an extended class system) in the same dataset to verify the model's adaptability to changes in class system and open class extensions.

[0059] Step four involves writing the mask / vector boundary and change direction labels of the changed objects into the geographic information product to form surface application results, such as the thematic layer for "Shenzhen 2024–2025 Key Land Feature Semantic Change Detection". Output formats include: raster: change mask GeoTIFF; vector: changed object polygon Shapefile / GeoJSON; attribute table: before and after time labels, confidence level, area, location, etc.

[0060] The experimental results of applying this invention to the public dataset SECOND are as follows: Table 1 Evaluation metrics of this invention on the SECOND dataset

[0061] As shown in the table, this invention exhibits the best overall performance on the SECOND dataset. Compared to the comparative methods, this invention achieves the highest or near-highest IoUc and F1 scores for all six land cover categories, with no categories showing significant performance degradation, indicating that the method possesses more stable and balanced classification and segmentation capabilities. Particularly in categories with complex semantic boundaries, such as trees, low vegetation, and non-vegetated surfaces, which are easily confused with surrounding land cover, this invention shows further improvements in both IoUc and F1, demonstrating its stronger ability to characterize fine-grained target structures and semantic differences. In categories with smaller sample sizes or uneven spatial distribution, such as water bodies and playgrounds, this invention also maintains relatively high recognition accuracy, demonstrating better robustness and generalization ability. In summary, the experimental results show that this invention does not rely on simple model scaling, but rather achieves overall performance improvement under multi-category and multi-scene conditions through methodological improvements, verifying its effectiveness and practical value in open-vocabulary remote sensing semantic segmentation tasks.

[0062] In conclusion, this invention introduces a visual-language large-scale model with image-text alignment and an open vocabulary mechanism after object extraction. This enables the model to perform semantic discrimination on unknown category changes without being constrained by a fixed classification system, thus achieving true open-category semantic change detection capability. This invention first explicitly proposes potential changes in dual-temporal images as object candidates using a large-scale visual model. Then, it performs semantic recognition on the same object in both preceding and subsequent temporal phases and combines directional information. This allows change analysis to be compared on an object-by-object and temporal-by-temporal basis, resulting in clearer logic, stronger interpretability, and better suppression of misjudgments caused by temporal differences. This invention directly outputs the semantic category of the changed object and the direction of change from one element to another, and can simultaneously output object mask / vector boundaries and confidence information. This allows the results to be more directly applied to the surface, reducing manual interpretation costs and improving application efficiency.

[0063] Example 7: Please see Figure 8 As shown, the present invention also provides an electronic device 100 for an open-category remote sensing change detection method based on a large model; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0064] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the open-category remote sensing change detection method based on a large model as described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0065] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0066] The memory 101 in the electronic device 100 stores multiple instructions to implement an open-class remote sensing change detection method based on a large model, and the processor 102 can execute the multiple instructions to achieve the following: Acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions for the dual-temporal sample pairs; Object extraction was performed on the dual-temporal samples to obtain a set of suspected changed objects; Extract text features of change categories from the text description, embed object features and unify dimensions for the suspected change object set, align and fuse the change category text features with the object feature embedding to obtain the final image-text relevance matrix; Based on the final image-text correlation matrix, a set of true image-text matches is constructed, and an open-category remote sensing change detection model is built.

[0067] Example 8: If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0068] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An open-category remote sensing change detection method based on a large model, characterized in that, Includes the following steps: Acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions for the dual-temporal sample pairs; Object extraction was performed on the dual-temporal samples to obtain a set of suspected changed objects; Extract text features of change categories from the text description, embed object features and unify dimensions for the suspected change object set, align and fuse the change category text features with the object feature embedding to obtain the final image-text relevance matrix; Based on the final image-text correlation matrix, a set of true image-text matches is constructed, and an open-category remote sensing change detection model is built.

2. The open-category remote sensing change detection method based on a large model according to claim 1, characterized in that, The specific method for acquiring dual-temporal remote sensing images, slicing the images to obtain dual-temporal sample pairs, and extracting text descriptions from these pairs is as follows: Acquire dual-temporal remote sensing images of the same geographic area; Spatial registration is performed on the dual-temporal remote sensing images. After registration, the images are sliced ​​according to the required window size to obtain dual-temporal sample pairs. The slice index, original image location and geographic coordinate mapping relationship are recorded. A multimodal large model is used to process dual-temporal sample pairs to generate text descriptions.

3. The open-category remote sensing change detection method based on a large model according to claim 1, characterized in that, The specific method for extracting objects from dual-temporal samples to obtain a set of suspected changed objects is as follows: Two-phase samples are obtained, and a large visual model is used to process the two-phase samples to obtain object candidates and multi-scale features. Based on object candidates and multi-scale features, generate a set of object candidate masks; The candidate mask set is matched across time to obtain candidate objects; Extract the candidate objects that have changed as a set of suspected changed objects.

4. The open-category remote sensing change detection method based on a large model according to claim 1, characterized in that, The specific method for extracting text features of change categories from text descriptions, embedding object features and unifying dimensions for the set of suspected changed objects, and aligning and fusing the text features of change categories with the object feature embeddings to obtain the final image-text relevance matrix is ​​as follows: Construct a set of change category text prompts, and use the set of change category text prompts to extract change category text features from the text descriptions; Spatial dimension compression and deformation are performed on the image features of each changed object in the suspected changed object set, and position encoding is added to obtain the object feature embedding; A cross-attention mechanism is used to align and fuse the text features of the change category with the object feature embeddings. Obtain the image-text correlation matrix between the fused change category text features and object feature embeddings, and normalize the column dimensions of the image-text correlation matrix to obtain the final image-text correlation matrix.

5. The open-category remote sensing change detection method based on a large model according to claim 1, characterized in that, The specific method for constructing an open-category remote sensing change detection model by building a true matching set of images and texts based on the final image-text correlation matrix is ​​as follows: Based on a pre-constructed open-category remote sensing change detection model using dual-temporal sample pairs; Obtain the final image-text relevance matrix and construct a set of true image-text matches; Calculate the loss for matching images to text and the loss for matching text to images to obtain the final total loss for contrastive learning; The pre-built open-category remote sensing change detection model is trained using the final contrastive learning total loss to obtain the open-category remote sensing change detection model.

6. An open-category remote sensing change detection system based on a large model, characterized in that: include: The data acquisition module is used to acquire dual-temporal remote sensing images, slice the dual-temporal remote sensing images to obtain dual-temporal sample pairs, and extract text descriptions of the dual-temporal sample pairs. The variable object extraction module is used to extract objects from dual-temporal samples to obtain a set of suspected variable objects; The object classification module is used to extract text features of change categories from text descriptions, perform object feature embedding and dimension unification on the suspected object set, and align and fuse the text features of change categories with the object feature embeddings to obtain the final image-text relevance matrix. The model building module is used to construct a set of true matching images and texts based on the final image-text correlation matrix, and to build an open-category remote sensing change detection model.

7. The open-category remote sensing change detection system based on a large model according to claim 6, characterized in that, The data acquisition module's functionality is implemented using the following methods: Acquire dual-temporal remote sensing images of the same geographic area; Spatial registration is performed on the dual-temporal remote sensing images. After registration, the images are sliced ​​according to the required window size to obtain dual-temporal sample pairs. The slice index, original image location and geographic coordinate mapping relationship are recorded. A multimodal large model is used to process dual-temporal sample pairs to generate text descriptions.

8. The open-category remote sensing change detection system based on a large model according to claim 6, characterized in that, The functionality of the changed object extraction module is implemented through the following methods: Two-phase samples are obtained, and a large visual model is used to process the two-phase samples to obtain object candidates and multi-scale features. Based on object candidates and multi-scale features, generate a set of object candidate masks; The candidate mask set is matched across time to obtain candidate objects; Extract the candidate objects that have changed as a set of suspected changed objects.

9. The open-category remote sensing change detection system based on a large model according to claim 6, characterized in that, The functionality of the variable object classification module is implemented through the following methods: Construct a set of change category text prompts, and use the set of change category text prompts to extract change category text features from the text descriptions; Spatial dimension compression and deformation are performed on the image features of each changed object in the suspected changed object set, and position encoding is added to obtain the object feature embedding; A cross-attention mechanism is used to align and fuse the text features of the change category with the object feature embeddings. Obtain the image-text correlation matrix between the fused change category text features and object feature embeddings, and normalize the column dimensions of the image-text correlation matrix to obtain the final image-text correlation matrix.

10. The open-category remote sensing change detection system based on a large model according to claim 6, characterized in that, The functionality of the model building module is implemented through the following methods: Based on a pre-constructed open-category remote sensing change detection model using dual-temporal sample pairs; Obtain the final image-text relevance matrix and construct a set of true image-text matches; Calculate the loss for matching images to text and the loss for matching text to images to obtain the final total loss for contrastive learning; The pre-built open-category remote sensing change detection model is trained using the final contrastive learning total loss to obtain the open-category remote sensing change detection model.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the open-category remote sensing change detection method based on a large model as described in any one of claims 1 to 5.

12. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the open-category remote sensing change detection method based on a large model as described in any one of claims 1 to 5.

Images