Anti-illusion multi-modal large model change detection method for autonomous land management agent

Abstract

The embodiment of the invention relates to the technical field of change detection, and particularly discloses an anti-illusion multi-modal large model change detection method for an autonomous land management agent. According to the embodiment of the invention, remote sensing images at two different time points are received, difference capture is carried out on the two remote sensing images, and a normalized time difference chart is generated; importing into a preset large language model for processing, and generating a rough change graph; constructing iterative input based on the normalized time difference graph, the text marker and the previous calibration mask, and performing iterative updating and convergence processing to obtain a change detection graph; and receiving a plurality of change detection tasks, generating a plurality of reasoning paths, and performing prediction aggregation to generate final prediction. According to the method, the generalization ability of the model in a complex and dynamic environment can be improved, the detection result can be gradually optimized, the illusion problem is remarkably reduced, the accuracy and reliability of change detection are improved, the robustness and consistency of the detection result are ensured, and the method is suitable for a complex and changeable remote sensing scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of change detection, and in particular to an anti-hallucination multimodal large model change detection method for an autonomous land management agent. Background Art

[0002] Multimodal data processing refers to the fusion and analysis of information from different modalities (such as images and text) to achieve accurate detection of target changes. Change detection (CD) is one of the core tasks of remote sensing technology, which aims to identify changes in the surface in different time periods by analyzing multi-temporal remote sensing images.

[0003] Currently, deep learning (DL) based change detection (CD) methods are mainly divided into two categories: convolutional neural network (CNN) based methods and Transformer architecture based methods.

[0004] CNN-based CD methods excel in extracting local spatial features and are suitable for learning complex features from high-resolution satellite and aerial images. Such methods are usually combined with preprocessing techniques such as image alignment and segmentation to enhance the model's ability to detect changes. However, due to the inherent local receptive field limitations of CNN, it performs poorly in capturing global context and long-frequency information. In addition, when the amount of data is limited, CNN methods are prone to overfitting problems and are difficult to effectively adapt to dynamic or subtle changes, especially in large-scale datasets or multi-temporal image processing scenarios.

[0005] Methods based on the Transformer architecture were originally designed for natural language processing (NLP) tasks. In recent years, they have gradually attracted attention in the CD field because their attention mechanism can model long-distance dependencies and global context information. Transformer-based CD methods improve the accuracy of change detection by capturing spatiotemporal relationships, and are particularly suitable for complex scenes and dynamic environments. However, such methods have limitations in extracting local features, which is crucial for detecting fine-grained changes in high-resolution images. In addition, the computational complexity of the Transformer is high and requires large hardware resources.

[0006] Therefore, although existing technologies have achieved significant performance improvements, they still face challenges in dealing with complex or dynamically changing environments. For example, when encountering scenes that deviate from the distribution of training data or when the detection task involves highly variable environmental conditions, these methods often show poor robustness and generalization. Summary of the invention

[0007] The purpose of the embodiments of the present invention is to provide an anti-hallucination multimodal large model change detection method for an autonomous land management agent, aiming to solve the problems raised in the background technology.

[0008] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: A hallucination-resistant multimodal large model change detection method for autonomous land management agents, the method specifically comprising the following steps: Receiving remote sensing images at two different time points, performing difference capture on the two remote sensing images through a SigLip image encoder, and generating a normalized time difference map; Acquire text tags, combine the normalized time difference graph with the text tags, import them into a preset large language model for processing, and generate a rough change graph; Based on the normalized time difference map, the text mark and the previous calibration mask, construct an iterative input, perform iterative update and convergence processing, and obtain a change detection map; Receive multiple change detection tasks, generate multiple reasoning paths, and perform prediction aggregation to generate the final prediction.

[0009] As a further limitation of the technical solution of the embodiment of the present invention, the receiving of two remote sensing images at different time points, performing difference capture on the two remote sensing images through a SigLip image encoder, and generating a normalized time difference map specifically includes the following steps: Receive remote sensing images at two different time points; The two remote sensing images are input into a shared encoder network through a SigLip image encoder for feature representation, thereby obtaining two feature maps; Projecting the two feature maps into a shared latent space through a projection head to obtain two latent space dimensions; A normalized temporal difference map is computed based on two of the latent space dimensions.

[0010] As a further limitation of the technical solution of the embodiment of the present invention, the expressions of the two feature graphs are: ; ; in, and are the feature maps corresponding to the two remote sensing images, and Represents two remote sensing images, represents the encoder network, is the parameter set of the encoder network; The expressions for the two latent space dimensions are: ; ; in, and are the latent space dimensions corresponding to the two remote sensing images, Represents the projection head, is the parameter set of the projection head; The calculation formula of the normalized time difference map is: ; in, Small constant introduced for numerical stability.

[0011] As a further limitation of the technical solution of the embodiment of the present invention, the obtaining of the text mark, combining the normalized time difference map with the text mark, importing the normalized time difference map into a preset large language model for processing, and generating a rough change map specifically comprises the following steps: Get the text tag; Combining the normalized time difference map with the text markup to generate a multimodal input; The multimodal input is imported into a preset large language model for processing to generate a rough change map.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the expression of the multimodal input is: ; in, For multimodal input, Mark for text; The expression of the rough change map is: ; in, Represents a rough change diagram.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, the step of constructing an iterative input based on the normalized time difference map, the text mark and the previous calibration mask, performing iterative updating and convergence processing, and obtaining a change detection map specifically includes the following steps: constructing an iterative input based on the normalized time difference map, the textual label and a previous calibration mask; Iteratively updating the change map according to the iterative input, and calculating a current calibration mask; The iterative process continues until the change graph converges, and finally the change detection graph is output.

[0014] As a further limitation of the technical solution of the embodiment of the present invention, the expression for iterative update of the change graph is: ; in, is the change graph for iterative updates, Input for iteration; The calculation formula of the current calibration mask is: ; The iteration process continues until the expression of the change graph converges is: ; in, is the preset convergence threshold.

[0015] As a further limitation of the technical solution of the embodiment of the present invention, the receiving of multiple change detection tasks, generating multiple reasoning paths, and performing prediction aggregation to generate a final prediction specifically includes the following steps: Receive multiple change detection tasks; According to the plurality of change detection tasks, a plurality of reasoning paths are generated, and a plurality of corresponding logical assumptions or conditions are associated; Generate intermediate CD graphs corresponding to multiple reasoning paths; Based on the multiple intermediate CD graphs, a final CD graph is aggregated and calculated to generate a final prediction.

[0016] As a further limitation of the technical solution of the embodiment of the present invention, the expression of generating multiple reasoning paths according to the multiple change detection tasks and associating multiple corresponding logical assumptions or conditions is: ; in, For the The reasoning path, For the The logical assumptions or conditions corresponding to the reasoning paths; The expressions of the plurality of said intermediate CD graphs are: ; in, For the The intermediate CD graph corresponding to the reasoning path; The expression of the final CD graph is: ; in, For the final CD map, Represents an aggregate function.

[0017] As a further limitation of the technical solution of the embodiment of the present invention, another expression of the final CD map is: ; in, For the The path confidence weight corresponding to the inference path.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The embodiment of the present invention receives remote sensing images at two different time points, captures the difference between the two remote sensing images, and generates a normalized time difference map; imports them into a preset large language model for processing to generate a rough change map; constructs iterative input based on the normalized time difference map, text tags, and the previous calibration mask, performs iterative update and convergence processing, and obtains a change detection map; receives multiple change detection tasks, generates multiple reasoning paths, and performs prediction aggregation to generate a final prediction. It can improve the generalization ability of the model in complex and dynamic environments, and can gradually optimize the detection results, significantly reduce the hallucination problem, and improve the accuracy and reliability of change detection, ensuring the robustness and consistency of the detection results, making it suitable for complex and changeable remote sensing scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0020] Figure 1 A flow chart of a method for detecting changes in an autonomous land management agent using multimodal models with an anti-hallucination feature is shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] It is understandable that in the prior art, CNN-based CD methods perform poorly in capturing global context and long-frequency information due to the inherent local receptive field limitations of CNN. In addition, when the amount of data is limited, CNN methods are prone to overfitting problems and are difficult to effectively adapt to dynamic or subtle changes, especially in large-scale data sets or multi-temporal image processing scenarios; methods based on the Transformer architecture have limitations in extracting local features, which is crucial for detecting fine-grained changes in high-resolution images. In addition, the Transformer has high computational complexity and requires large hardware resources. Therefore, although the prior art has significantly improved performance, it still faces challenges in dealing with complex or dynamically changing environments. For example, when encountering scenarios that deviate from the distribution of training data, or when the detection task involves highly variable environmental conditions, these methods often show poor robustness and generalization.

[0023] To solve the above problems, the embodiment of the present invention receives remote sensing images at two different time points, and uses the SigLip image encoder to capture the difference between the two remote sensing images to generate a normalized time difference map; obtains text tags, combines the normalized time difference map with the text tags, and imports them into a preset large language model for processing to generate a rough change map; constructs iterative input based on the normalized time difference map, text tags and the previous calibration mask, performs iterative updates and convergence processing, and obtains a change detection map; receives multiple change detection tasks, generates multiple reasoning paths, and performs prediction aggregation to generate a final prediction. It can improve the generalization ability of the model in complex and dynamic environments, and can gradually optimize the detection results, significantly reduce the hallucination problem, and improve the accuracy and reliability of change detection, ensuring the robustness and consistency of the detection results, making it suitable for complex and changeable remote sensing scenes.

[0024] Figure 1 A flow chart of a method for detecting changes in an autonomous land management agent using multimodal models with an anti-hallucination feature is shown.

[0025] Specifically, the anti-hallucination multimodal large model change detection method of the autonomous land management agent comprises the following steps: Step S101, receiving two remote sensing images at different time points, and performing difference capture on the two remote sensing images through a SigLip image encoder to generate a normalized time difference map.

[0026] In the embodiment of the present invention, by receiving remote sensing images of two different time points of the autonomous land management agent, the two remote sensing images are input into the shared encoder network for feature representation using the SigLip image encoder to obtain two feature maps, and then the two feature maps are projected into the shared latent space through the projection head to obtain two latent space dimensions, and then the normalized time difference map is calculated according to the two latent space dimensions. Specifically, the expressions of the two feature maps are: ; ; in, and are the feature maps corresponding to the two remote sensing images, and Represents two remote sensing images, represents the encoder network, is the parameter set of the encoder network; The expression for the two latent space dimensions is: ; ; in, and are the latent space dimensions corresponding to the two remote sensing images, Represents the projection head, is the parameter set of the projection head; The calculation formula of the normalized time difference map is: ; in, Small constant introduced for numerical stability.

[0027] As can be understood, the SigLip encoder processes images from two different time points and generates feature embeddings that represent the temporal differences of the images. These feature representations are input into the LLM along with the corresponding text data, which generates preliminary change predictions by capturing patterns in visual and contextual modalities. As a multimodal model, LLM leverages the complementarity of image and text features to enhance the understanding of changes in the environment, thereby being able to capture complex relationships that are difficult for traditional CNN or Transformer models to handle, especially in novel or unexpected change scenarios.

[0028] It can be understood that the normalized temporal difference map captures the magnitude and direction of temporal changes in the latent feature space, which is able to highlight key changes between images, such as urbanization or deforestation, while suppressing background noise.

[0029] Step S102, obtaining text tags, combining the normalized time difference graph with the text tags, importing them into a preset large language model for processing, and generating a rough change graph.

[0030] In the embodiment of the present invention, by obtaining the text mark, the normalized time difference map is combined with the text mark to generate a multimodal input, and then the multimodal input is imported into a preset large language model for processing to generate a rough change map. Specifically, the expression of the multimodal input is: ; in, For multimodal input, Mark for text; The expression of the rough change map is: ; in, Represents a rough change diagram.

[0031] Understandably, in a multimodal framework, the SigLip encoder processes each image individually to generate feature representations that emphasize temporal differences rather than static attributes, and the resulting normalized temporal difference maps are combined with text tags as the main input to subsequent stages of the model (including the CVA calibration loop), thereby guiding the model to focus on detecting meaningful changes rather than irrelevant background features. Modular and robust difference extraction enables the system to maintain accuracy and adaptability in a variety of environments.

[0032] It is understood that the coarse change map provides a preliminary estimate of the probability of change for each pixel. However, this coarse prediction usually contains spatial inaccuracies that need to be further corrected through an iterative optimization process.

[0033] Step S103, constructing an iterative input based on the normalized time difference map, the text mark and the previous calibration mask, performing iterative updating and convergence processing, and obtaining a change detection map.

[0034] In the embodiment of the present invention, an iterative input is constructed based on the normalized time difference map, the text mark and the previous calibration mask, and then the change map is iteratively updated according to the iterative input, and the current calibration mask is calculated. The iterative process continues until the change map converges, and finally the change detection map is output. Specifically, the expression of the iterative update of the change map is: ; in, is the change graph for iterative updates, Input for iteration; The current calibration mask is calculated as: ; The iteration process continues until the expression of the change graph converges: ; in, is the preset convergence threshold.

[0035] It can be understood that the calibration mask is used to correct the prediction change map, ensuring that the optimization process focuses on meaningful temporal changes while reducing the impact of noise; the change detection map combines the semantic understanding of LLM with the spatial accuracy of CVA to reduce hallucination phenomena and improve detection accuracy.

[0036] Step S104, receiving multiple change detection tasks, generating multiple reasoning paths, and performing prediction aggregation to generate a final prediction.

[0037] In the embodiment of the present invention, multiple change detection tasks are received, multiple reasoning paths are generated according to the multiple change detection tasks, and multiple corresponding logical assumptions or conditions are associated, and then intermediate CD graphs corresponding to the multiple reasoning paths are generated, and then based on the multiple intermediate CD graphs, the final CD graph is aggregated and calculated to generate the final prediction. Specifically, the expression of generating multiple reasoning paths according to the multiple change detection tasks and associating multiple corresponding logical assumptions or conditions is: ; in, For the The reasoning path, For the The logical assumptions or conditions corresponding to the reasoning paths; The expressions of multiple intermediate CD graphs are: ; in, For the The intermediate CD graph corresponding to the reasoning path; The final CD graph expression is: ; in, For the final CD map, Represents an aggregate function; Furthermore, another expression of the final CD graph is: ; in, For the The path confidence weight corresponding to the inference path.

[0038] It can be understood that step S104 is a CoT inference process that effectively deals with uncertainties in remote sensing data, such as noise or ambiguity in observed changes. By generating multiple hypotheses and evaluating them through aggregation, the model ensures that the most consistent and credible explanations are prioritized. For example, if one path attributes the detected changes to urbanization, while another path attributes them to seasonal vegetation changes, the aggregation step can combine these hypotheses, thereby reducing the risk of overfitting to irrelevant features. This capability enables the model to consider complex environmental factors that affect observed changes.

[0039] Understandably, in addition to handling uncertainty, CoT inference significantly improves the robustness of the model. By exploring multiple hypotheses simultaneously, the model reduces the likelihood of misinterpreting background noise or spurious patterns as real changes. Each inference path is designed to explore a different hypothesis or explanation, refining the predictions in the intermediate steps, allowing the model to better generalize to new or unseen environments. This is particularly important in remote sensing, where data quality can vary due to atmospheric interference, sensor limitations, or uneven lighting conditions.

[0040] Understandably, another key advantage of CoT reasoning is its ability to reduce hallucinations, which are false positives or irrelevant changes due to overgeneralization or noise. By generating multiple reasoning paths, CoT reasoning reduces the impact of any single path that may induce hallucinations. The aggregation process ensures that such spurious predictions are covered by more consistent and credible path results, ultimately improving the overall accuracy of the model. This approach is particularly valuable in complex tasks such as urbanization detection, where the model must distinguish between real changes (such as new buildings) and irrelevant phenomena (such as seasonal vegetation changes).

[0041] As can be understood, CoT reasoning provides a structured framework for change detection through iterative hypothesis generation and optimization. Each reasoning path is refined through intermediate steps, incorporating additional features, semantic context, and prior knowledge to improve predictions. The final CD output is obtained by aggregating the results of these paths, giving more weight to the most consistent and credible hypotheses. This iterative and structured reasoning makes CoT reasoning a powerful tool for reliable and accurate change detection in dynamic and noisy remote sensing environments.

[0042] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0043] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0044] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0045] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Hallucination-resistant multimodal large-scale change detection method for autonomous land management agents, characterized by: The method specifically comprises the following steps: Receiving remote sensing images at two different time points, performing difference capture on the two remote sensing images through a SigLip image encoder, and generating a normalized time difference map; Acquire text tags, combine the normalized time difference graph with the text tags, import them into a preset large language model for processing, and generate a rough change graph; Based on the normalized time difference map, the text mark and the previous calibration mask, construct an iterative input, perform iterative update and convergence processing, and obtain a change detection map; Receive multiple change detection tasks, generate multiple reasoning paths, and perform prediction aggregation to generate the final prediction.

2. The anti-hallucination multimodal large model change detection method for autonomous land management agents according to claim 1, characterized in that The receiving of two remote sensing images at different time points, performing difference capture on the two remote sensing images through a SigLip image encoder, and generating a normalized time difference map specifically comprises the following steps: Receive remote sensing images at two different time points; The two remote sensing images are input into a shared encoder network through a SigLip image encoder for feature representation, thereby obtaining two feature maps; Projecting the two feature maps into a shared latent space through a projection head to obtain two latent space dimensions; A normalized temporal difference map is computed based on two of the latent space dimensions.

3. The anti-hallucination multimodal large model change detection method for autonomous land management agents according to claim 2, characterized in that: The expressions of the two feature maps are: ； ； in, and are the feature maps corresponding to the two remote sensing images, and Represents two remote sensing images, represents the encoder network, is the parameter set of the encoder network; The expressions for the two latent space dimensions are: ； ； in, and are the latent space dimensions corresponding to the two remote sensing images, Represents the projection head, is the parameter set of the projection head; The calculation formula of the normalized time difference map is: ； in, Small constant introduced for numerical stability.

4. The hallucination-resistant multimodal large model change detection method for autonomous land management agents according to claim 3, characterized in that: The step of obtaining the text mark, combining the normalized time difference map with the text mark, and importing the normalized time difference map into a preset large language model for processing, and generating a rough change map specifically includes the following steps: Get the text tag; Combining the normalized time difference map with the text markup to generate a multimodal input; The multimodal input is imported into a preset large language model for processing to generate a rough change map.

5. The anti-hallucination multimodal large model change detection method for autonomous land management agents according to claim 4, characterized in that: The expression of the multimodal input is: ； in, For multimodal input, Mark for text; The expression of the rough change map is: ； in, Represents a rough change diagram.

6. The hallucination-resistant multimodal large model change detection method for autonomous land management agents according to claim 5, characterized in that: The step of constructing an iterative input based on the normalized time difference map, the text mark and the previous calibration mask, performing iterative updating and convergence processing, and obtaining a change detection map specifically includes the following steps: constructing an iterative input based on the normalized time difference map, the textual label and a previous calibration mask; Iteratively updating the change map according to the iterative input, and calculating a current calibration mask; The iterative process continues until the change graph converges, and finally the change detection graph is output.

7. The hallucination-resistant multimodal large model change detection method for autonomous land management agents according to claim 6, characterized in that: The expression for iterative update of the change graph is: ； in, is the change graph for iterative updates, Input for iteration; The calculation formula of the current calibration mask is: ； The iteration process continues until the expression of the change graph converges is: ； in, is the preset convergence threshold.

8. The anti-hallucination multimodal large model change detection method for autonomous land management agents according to claim 5, characterized in that: The receiving of multiple change detection tasks, generating multiple reasoning paths, and performing prediction aggregation to generate a final prediction specifically includes the following steps: Receive multiple change detection tasks; According to the plurality of change detection tasks, a plurality of reasoning paths are generated, and a plurality of corresponding logical assumptions or conditions are associated; Generate intermediate CD graphs corresponding to multiple reasoning paths; Based on the multiple intermediate CD graphs, a final CD graph is aggregated and calculated to generate a final prediction.

9. The hallucination-resistant multimodal large model change detection method for autonomous land management agents according to claim 8, characterized in that: The expression of generating multiple reasoning paths according to the multiple change detection tasks and associating multiple corresponding logical assumptions or conditions is: ； in, For the The reasoning path, For the The logical assumptions or conditions corresponding to the reasoning paths; The expressions of the plurality of intermediate CD graphs are: ； in, For the The intermediate CD graph corresponding to the reasoning path; The expression of the final CD graph is: ； in, For the final CD map, Represents an aggregate function.

10. The hallucination-resistant multimodal large model change detection method for autonomous land management agents according to claim 9, characterized in that: Another expression of the final CD graph is: ； in, For the The path confidence weight corresponding to the inference path.

Images