A remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation

Abstract

This invention relates to the field of remote sensing image processing and intelligent retrieval technology, and discloses a natural language intelligent retrieval method for remote sensing images based on image-text alignment and grid segmentation. The method includes: preprocessing multi-source remote sensing data to form unified coordinates and time markers; grid segmenting the data and extracting metadata to construct spatial and temporal indexes; extracting high-dimensional image features through a multimodal model to construct a semantic vector index; parsing natural language queries to obtain structured data and marking temporal changes; when there are no changes, performing a triple-filtering process of spatiotemporal semantics to find the intersection; when there are changes, performing sub-query operations to verify the validity of the changes; and finally, visualizing the results. This invention achieves multimodal fusion retrieval of natural language and remote sensing images, improves efficiency and accuracy through triple indexing, and automates multi-time comparisons, solving the problems of inflexible and cumbersome operation in traditional retrieval techniques.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and intelligent retrieval technology, specifically to a natural language intelligent retrieval method for remote sensing images based on image-text alignment and grid segmentation. Background Technology

[0002] With the rapid development of remote sensing technology, high-resolution, multi-source remote sensing data is experiencing explosive growth. As the core carrier of geospatial information, the efficient management and intelligent retrieval of remote sensing imagery have become key technical requirements in the field of remote sensing image processing. This technology widely supports intelligent applications in multiple fields such as resource surveys, environmental monitoring, and urban planning. It needs to simultaneously meet multiple requirements, including accurate spatial positioning, continuous time-series analysis, and accurate semantic recognition of ground features, driving the upgrading of related fields from traditional manual analysis to automation and intelligence.

[0003] Currently, mainstream remote sensing image retrieval and related technical solutions mainly fall into three categories. The first category is remote sensing image management systems based on metadata retrieval. This approach constructs spatial indexes and text retrieval mechanisms, using metadata such as the image capture time, sensor type, and band information as core retrieval fields. Users can query and retrieve image data by inputting specific keywords or setting metadata filtering conditions. It relies heavily on the direct matching of metadata and retrieval conditions, possessing basic spatial filtering capabilities. The second category is partially deep learning-based retrieval solutions. These solutions utilize deep learning models to extract features from remote sensing images, focusing on scene classification or target detection for single-modality images (such as optical images). By pre-setting fixed land cover category labels, they achieve rapid retrieval of specific target images, offering advantages in single land cover identification and retrieval efficiency. The third category is multi-temporal image comparison systems based on mesh partitioning. This approach uses mesh partitioning methods such as Geosot, QuadTree, or DGGS to segment remote sensing images, forming standardized mesh units. Through the organization and management of these mesh units, it supports multi-temporal image comparison analysis of local areas, possessing basic multi-scale, multi-temporal data processing capabilities.

[0004] However, in practice, due to insufficient flexibility in semantic retrieval, existing technologies rely on fixed metadata field matching or are limited to preset land cover category labels, lacking the ability to parse complex semantics of natural language. This makes it difficult to meet the combined retrieval needs of multiple conditions and dimensions, such as changes in land cover types within a specific spatiotemporal range. Furthermore, the multimodal fusion capability is lacking. Existing solutions are mostly for single-modal images or only utilize metadata text information, making it difficult to deeply align and fuse the visual features of remote sensing images with the semantic features of natural language, and difficult to uncover potential semantic relationships in images. At the same time, the multi-temporal analysis technology process is cumbersome. Multi-temporal image comparison systems based on segmentation require users to manually complete independent steps such as multi-time point image retrieval, data alignment, and difference calculation. The lack of an automated temporal change analysis mechanism leads to high technical complexity and a tendency to generate data bias. Summary of the Invention

[0005] This invention provides a natural language intelligent retrieval method for remote sensing images based on image-text alignment and grid segmentation, which solves at least one of the technical problems mentioned in the background art, realizes intelligent retrieval and difference comparison of remote sensing images in multiple time phases, multiple scenes, and multiple modalities, and can support the effect of retrieval conditions described flexibly in natural language.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation, comprising the following steps: S1. Acquire multi-source remote sensing data, preprocess the multi-source remote sensing data, and form standardized data with a unified coordinate reference system and time stamp; S2. The standardized data is divided into grids to generate grid cells with unique codes. The meta-information of the grid cells is extracted, and the grid cells and meta-information are stored and a spatial index and a temporal index are constructed. S3. Use a multimodal model to extract features from the grid cells to obtain high-dimensional image feature vectors, store the high-dimensional feature vectors and construct a semantic vector index; S4. Receive a natural language query, parse the natural language query, extract key information and convert it into structured data, and mark whether it contains time-varying status. S5. Based on the time-change status markers of the structured data, execute the corresponding retrieval process to obtain the target grid set; S6. Visualize the target mesh set.

[0007] Preferably, in step S1, the multi-source remote sensing data includes optical images, multispectral images, and SAR images; the preprocessing sequentially performs distortion correction, radiometric correction, registration, and fusion; the unified coordinate reference system is a preset coordinate system; and the time stamp is a standardized timestamp in UTC format.

[0008] Preferably, in step S2, the grid segmentation adopts a discrete global grid system segmentation method, a Geosot segmentation method, a quadtree segmentation method, or a block strategy based on administrative divisions; the unique code is a three-level code of hierarchy, region code, and unit code; the metadata includes shooting time, sensor type, grid hierarchy, spatial boundary, and effective pixel count; the grid unit and metadata are stored in a relational database with a spatial extension module, and the grid unit is pushed to the task queue to wait for feature extraction.

[0009] Preferably, in step S3, the multimodal model is a RemoteCLIP model based on CLIP fine-tuning or other contrastive language-image pre-trained models; the feature extraction involves taking grid cells from the task queue in parallel, inputting them into the image encoder of the multimodal model to obtain high-dimensional image feature vectors, and storing the high-dimensional image feature vectors in a vector database after normalization; the semantic vector index is constructed based on an approximate nearest neighbor algorithm, and the vector database is FAISS, Milvus, or QDrant.

[0010] Preferably, in step S4, the parsing adopts a combination of named entity recognition and parsing rules; the key information includes time range, spatial range, land cover type and change status; the structured data is in JSON format and includes time range start and end information, spatial geometric coordinate information, target type information and change status marker information; the time change status marker is a Boolean value.

[0011] Preferably, in step S5, when the time change status is marked as false, the retrieval process includes: The target feature type and spatial extent description in the structured data are input into the multimodal model to generate text feature vectors; The set of grid cells that intersects with the query spatial range is filtered by the spatial index, and the set of grid cells that are within the query time range is filtered by the time index. By performing a nearest neighbor query on the text feature vector using the semantic vector index, a set of semantically similar grid cells is obtained; The target mesh set is obtained by taking the intersection of the three sets.

[0012] Preferably, in step S5, when the time change status is true, the retrieval process includes: The natural language query is broken down into multiple subqueries with no time-varying states using a large language model, with each subquery corresponding to a single point in time. Execute each sub-query according to the retrieval process to obtain the grid cell set at the corresponding time point; Perform intersection operations, depth segmentation network operations, or time-series-based detection network operations on multiple sets of said grid cells to obtain a set of grid cells representing the changed regions; Remove scattered grid cells with an area smaller than a preset threshold, verify the effectiveness of the changes in the remaining grid cells, and obtain the target grid set.

[0013] Preferably, the validity of the change is determined by calculating the similarity of the high-dimensional image feature vectors of the corresponding grid cells before and after the change. If the similarity is lower than a preset threshold, the change is confirmed to be valid.

[0014] Preferably, in step S6, the visualization includes: highlighting the changing areas using color coding, marking the decreasing areas with red, the increasing areas with green, and the unchanged areas with gray; displaying multi-temporal images overlaid, supporting transparency adjustment and layer switching; dynamically calculating the area and distribution data of the target area, and displaying it in chart form.

[0015] Preferably, the relational database with spatial expansion module is PostgreSQL+PostGIS.

[0016] This invention provides a natural language intelligent retrieval method for remote sensing images based on image-text alignment and grid segmentation. It has the following beneficial effects:

[0017] 1. This invention achieves image-text semantic depth alignment through multimodal models such as RemoteCLIP, mapping the visual features of remote sensing images and natural language descriptions to the same semantic space. This allows users to flexibly express complex search needs through natural language without needing to master professional search syntax, thus making up for the shortcomings of traditional solutions in supporting complex semantic expressions.

[0018] 2. This invention constructs a triple indexing framework consisting of spatial grid subdivision, timestamps, and feature vectors, which enables comprehensive filtering of spatial, temporal, and semantic dimensions during the retrieval process, greatly improving the efficiency and accuracy of remote sensing image retrieval. It can also accurately calculate the classification and change status of land features in multiple time phases.

[0019] 3. This invention automatically decomposes complex natural language queries with time-varying characteristics into multiple subqueries using a large language model. By combining grid set operations with semantic feature deep learning comparison, it automates the detection of multi-temporal differences without requiring users to manually perform multiple steps, reducing the usage threshold for non-professional users, while ensuring the accuracy and consistency of the results.

[0020] 4. This invention preprocesses multi-source remote sensing data by performing distortion correction, radiometric correction, registration, and fusion to form a unified coordinate reference system and time stamp, which is compatible with various data types such as optical images, multispectral images, and SAR images, thereby achieving efficient integration and standardized management of multi-source data.

[0021] 5. This invention employs visualization schemes such as color coding to highlight changing areas, multi-temporal image overlay display, and dynamic statistical analysis to present search results intuitively. Users can quickly identify the changing state of the target area and compare the land cover situation at different times, significantly improving the practicality and readability of the search results. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a flowchart illustrating the overall process of remote sensing image data storage according to the present invention. Figure 3 This is a schematic diagram illustrating the core working principle of the CLIP model of this invention. Figure 4 This is a schematic diagram illustrating the alignment of text and images in this invention; Figure 5 This is a flowchart of the intelligent search process of the present invention. Detailed Implementation

[0023] To better understand the above technical solutions, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see the appendix Figure 1 - Appendix Figure 5 This invention provides a method for intelligent natural language retrieval of remote sensing images based on image-text alignment and grid segmentation, comprising the following steps: S1. Acquire multi-source remote sensing data, preprocess the multi-source remote sensing data, and form standardized data with a unified coordinate reference system and time stamp; S2. Perform grid segmentation on the standardized data to generate grid cells with unique codes, extract the meta-information of the grid cells, store the grid cells and meta-information, and construct spatial and temporal indexes. S3. Use a multimodal model to extract features from grid cells to obtain high-dimensional image feature vectors, store the high-dimensional feature vectors and construct a semantic vector index; S4. Receive natural language queries, parse natural language queries, extract key information and convert it into structured data, and mark whether it contains time-varying status. S5. Based on the time-varying status markers of the structured data, execute the corresponding retrieval process to obtain the target grid set; S6. Visualize the target mesh set.

[0025] Specifically, after acquiring remote sensing data covering multiple types, distortion correction, radiometric correction, registration, and fusion processing are performed sequentially to form a unified preset coordinate reference system and UTC-formatted time stamps, thus obtaining standardized data. The standardized data is then divided into grids using an appropriate partitioning strategy, generating grid cells with unique three-level codes: hierarchy, region code, and cell code. Meta-information, including the capture time, sensor type, grid hierarchy, spatial boundaries, and effective pixel count, is extracted. The grid cells and meta-information are stored in a relational database with a spatial extension module, and spatial and temporal indexes are constructed based on the spatial boundaries and time stamps of the grid cells, respectively. A multimodal model based on contrastive language-image pre-training is used, with grid cells retrieved in parallel from the task queue and input into the model's image encoder. This extracts high-dimensional image feature vectors that characterize the semantic and visual features of the remote sensing images, which are then processed using formulas. The feature vectors of high-dimensional images are normalized, where, This is the normalized high-dimensional image feature vector. The original high-dimensional image feature vector, The L2 norm of the original high-dimensional image feature vector is used to normalize the high-dimensional image feature vector, which is then stored in a vector database. A semantic vector index is constructed based on the approximate nearest neighbor algorithm. After receiving a natural language query from the user, the query is parsed using a combination of named entity recognition and parsing rules to extract key information such as the time range, spatial range, land cover type, and change status. This key information is then converted into structured data in JSON format, and a boolean value is used to indicate whether the query includes a time change status. When marked as not containing time-varying states, the target land cover type and spatial extent description from the structured data are input into a multimodal model to generate text feature vectors. A set of grid cells intersecting with the query spatial extent is filtered using a spatial index, and a set of grid cells within the query time range is filtered using a temporal index. A nearest neighbor lookup is performed on the text feature vectors using a semantic vector index to obtain a set of semantically similar grid cells. The intersection of these three sets yields the target grid set. When marked as containing time-varying states, a Large Language Model (LLM) is used to split the natural language query into multiple subqueries corresponding to single time points without time-varying states. Each subquery is executed according to the above retrieval process for non-time-varying states to obtain a set of grid cells at the corresponding time point. Corresponding operations are performed on these grid cell sets to obtain a set of grid cells representing changed areas. Scattered grid cells are removed, and the validity of the change is verified by calculating the similarity of high-dimensional image feature vectors of the corresponding grid cells before and after the change. Finally, the target grid set is obtained. The system uses color coding to highlight areas with different types of changes in the target grid set, enabling the overlay display of multi-temporal images and supporting transparency adjustment and layer switching. It dynamically collects relevant data of the target area and presents them in the form of charts, allowing users to intuitively and clearly identify the land features and changes of the target area. This significantly reduces the operational threshold while improving the practicality and readability of the search results, meeting users' needs for flexible and efficient retrieval and change analysis of remote sensing images.

[0026] In step S1, the multi-source remote sensing data includes optical images, multispectral images, and SAR images; the preprocessing sequentially performs distortion correction, radiometric correction, registration, and fusion, with the unified coordinate reference system being the preset coordinate system, and the time stamp being a standardized timestamp in UTC format.

[0027] Specifically, the multi-source remote sensing data includes optical imagery, multispectral imagery, and SAR imagery. Preprocessing is carried out in the order of distortion correction, radiometric correction, registration, and fusion. Distortion correction uses a rational function model combined with ground control points or sensor-provided parameters to correct geometric deviations in the remote sensing imagery caused by sensor attitude and terrain undulations. Radiometric correction is performed using a formula... The original image grayscale values ​​are converted into radiance values. The radiance value after radiation correction Gain coefficient The original grayscale value of the remote sensing image The offset coefficient is used to eliminate radiation distortion caused by atmospheric scattering sensor noise. Registration is based on a preset coordinate system and uses a feature matching algorithm to achieve spatial alignment of remote sensing images from different sources or at different times. Fusion uses an appropriate fusion algorithm to integrate the spatial details and spectral information of multi-source images. The uniformity of the preset coordinate system is maintained throughout the process, and a standardized UTC format timestamp is added to the processed standardized data, so that all data have a unified spatial reference and time identifier. This effectively solves the problem of inconsistent formats of multi-source and multi-temporal remote sensing data, and provides a consistent and reliable data foundation for spatiotemporal matching in the subsequent grid segmentation index construction and retrieval process, ensuring that multi-source remote sensing data can be efficiently integrated and accurately retrieved.

[0028] In step S2, grid segmentation employs methods such as discrete global grid system segmentation, Geosot segmentation, quadtree segmentation, or a block strategy based on administrative divisions; unique encoding is a three-level encoding of hierarchy, region code, and unit code; metadata includes shooting time, sensor type, grid hierarchy, spatial boundary, and effective pixel count; grid units and metadata are stored in a relational database with spatial extension modules, and grid units are pushed to the task queue to await feature extraction.

[0029] Specifically, the grid segmentation employs either the discrete global grid system segmentation method, the Geosot segmentation method, the quadtree segmentation method, or a block strategy based on administrative divisions. Among these, the discrete global grid system segmentation organizes multi-resolution spatial data by discretizing the Earth's surface into regular grid units. Other segmentation strategies aim for efficient indexing and storage by segmenting large-scale remote sensing images into blocks, generating grid units with three levels of unique codes: hierarchy, region code, and unit code. The codes correspond one-to-one with the grid units to ensure the uniqueness of spatial identifiers. Meta-information, including shooting time, sensor type, grid hierarchy, spatial boundaries, and effective pixel count, is extracted. This meta-information serves as additional filtering conditions for subsequent retrieval, supplementing the data attribute dimensions. The grid cells and their corresponding metadata are stored in a relational database with a spatial extension module. The database's spatial data processing capabilities support the construction and efficient querying of spatial and temporal indexes. At the same time, the segmented grid cells are pushed to a task queue according to a preset priority to await feature extraction. This achieves structured storage and orderly flow of grid data, providing a reliable data organization foundation for accurate spatiotemporal range filtering and efficient semantic feature extraction in the subsequent retrieval process. This meets the needs of large-scale management and flexible retrieval of multi-source and multi-temporal remote sensing images.

[0030] In step S3, the multimodal model is a RemoteCLIP model based on CLIP fine-tuning or other contrastive language-image pre-trained models; feature extraction involves taking grid cells in parallel from the task queue, inputting them into the image encoder of the multimodal model to obtain high-dimensional image feature vectors, and storing the high-dimensional image feature vectors in a vector database after normalization. The semantic vector index is constructed based on the approximate nearest neighbor algorithm, and the vector database is FAISS, Milvus, or QDrant.

[0031] Specifically, the multimodal model employs the RemoteCLIP model based on CLIP fine-tuning or other contrastive language-image pre-trained models. These models, through pre-training with large-scale remote sensing imagery and corresponding text descriptions, map the visual features of remote sensing images and the semantic features of natural language to the same shared semantic space to achieve deep alignment. During feature extraction, pre-segmented grid cells are retrieved in parallel from the task queue. The remote sensing image patch corresponding to each grid cell is input into the image encoder of the multimodal model to extract high-dimensional image feature vectors that accurately represent the features and semantic information of the imagery. These vectors are then processed using a formula... The feature vectors of the high-dimensional image are subjected to L2 normalization, where This represents the original high-dimensional image feature vector of a grid cell remote sensing image patch. The L2 norm of the feature vector of the original high-dimensional image. This is a normalized high-dimensional image feature vector. The normalized high-dimensional image feature vector is stored in a FAISS, Mius, or QDrant vector database, and a semantic vector index is constructed based on the approximate nearest neighbor algorithm. This makes the feature vectors corresponding to semantically similar grid cells closer in the vector space, providing efficient support for subsequent natural language queries and semantic matching of remote sensing images. This ensures that semantically related grid cells can be quickly and accurately located during the retrieval process, fully leveraging the advantages of multimodal fusion to meet the requirements of multi-source remote sensing image intelligent retrieval for semantic matching accuracy and efficiency.

[0032] In step S4, the parsing uses a combination of named entity recognition and parsing rules; key information includes time range, spatial range, land cover type and change status; the structured data is in JSON format, containing start and end information of the time range, spatial geometric coordinates, target type information and change status marker information; the time change status marker is a Boolean value.

[0033] Specifically, upon receiving a natural language query, named entity recognition technology combined with preset parsing rules is used to perform deep analysis of the query content. This accurately identifies key information such as time range, spatial range, feature type, and change status. This key information is then converted into structured data in JSON format according to a unified standard. This structured data explicitly includes start and end time range information, spatial geometric coordinate information, target type information, and change status marker information. The change status marker is presented in Boolean form to indicate whether the query involves time-related requirements. This parsing method can effectively mine the deep semantics and constraints in natural language, transforming ambiguous natural language descriptions into structured instructions that the system can directly recognize and process. This avoids dependence on professional search syntax and provides accurate and standardized data support for subsequent execution of corresponding search processes based on change status, ensuring that natural language queries can be accurately mapped to the search logic.

[0034] In step S5, when the time change status is marked as false, the retrieval process includes: Input the target feature type and spatial extent description from the structured data into the multimodal model to generate text feature vectors; Filter the set of grid cells that intersect with the query spatial range using the spatial index, and filter the set of grid cells that are within the query time range using the time index; By performing nearest neighbor queries on text feature vectors using semantic vector indexing, a set of semantically similar grid cells is obtained; The intersection of the three sets yields the target grid set.

[0035] Specifically, the target feature type and spatial extent description from structured data are input into a multimodal model. The model's text-image semantic alignment capability is used to generate text feature vectors that characterize the query requirements. A set of grid cells intersecting with the query spatial extent is selected using a spatial index, and a set of grid cells within the query time range is selected using a temporal index. Based on the semantic vector, a nearest neighbor query is performed on the text feature vectors. During the query process, formulas are used... Calculate text feature vectors High-dimensional image feature vectors corresponding to grid cells The similarity, of which This is the text feature vector after natural language query transformation. High-dimensional image feature vectors of grid cell remote sensing images and The L2 norms of text feature vectors and high-dimensional image feature vectors are used to filter out semantically similar grid cell sets. The intersection of the three sets obtained from spatial, temporal, and semantic filtering is then taken to obtain the target grid set. This triple-index collaborative filtering method achieves precise constraints on spatial, temporal, and semantic dimensions, fully leverages the semantic matching advantages of multimodal models and the efficient filtering capabilities of grid indexes, and ensures that the retrieval results not only meet the spatiotemporal range requirements of the query but also have accurate semantic relevance, effectively improving the efficiency and accuracy of queries in states with no time changes.

[0036] In step S5, when the time change status is marked as true, the retrieval process includes: The natural language query is broken down into multiple subqueries with no time-varying states by using a large language model, with each subquery corresponding to a single point in time. Execute each sub-query according to the retrieval process to obtain the grid cell set corresponding to the time point; Perform intersection operations, depth segmentation network operations, or time-series-based detection network operations on multiple sets of grid cells to obtain a set of grid cells representing the changed region. Remove scattered grid cells with an area smaller than a preset threshold, verify the effectiveness of the changes in the remaining grid cells, and obtain the target grid set.

[0037] Specifically, when the time-change state is marked as true, the natural language query is parsed and decomposed using a large language model to obtain multiple subqueries corresponding to a single time point and not containing time-change states. Each subquery is executed sequentially according to the retrieval process for time-change states, i.e., spatiotemporal range filtering is performed using spatial and temporal indexes, and semantic matching is achieved using semantic vector indexes. This yields a set of grid cells corresponding to each time point. These grid cell sets are then subjected to intersection operations, deep segmentation network operations, or time-series-based detection network operations, depending on actual needs. This filters out potentially changing grid cell sets, removes small, scattered grid cells, and then uses a formula... ) Calculate the similarity of high-dimensional image feature vectors of corresponding grid cells before and after the change, where This represents the high-dimensional image feature vector of the grid cell at the previous time point. The high-dimensional image feature vector of the grid cell corresponding to the next time point and The L2 norms of the two vectors are used to verify the validity of the changes in the remaining grid cells, and finally the target grid set is obtained. This process realizes the automated decomposition and execution of complex time change queries without the need for manual comparison of multi-temporal images. It fully combines the advantages of triple indexing and multiple change detection algorithms to ensure the accuracy and efficiency of change area retrieval and meet users' flexible needs for multi-temporal remote sensing image change analysis.

[0038] The validity of the change is determined by calculating the similarity of the high-dimensional image feature vectors of the corresponding grid cells before and after the change. If the similarity is lower than a preset threshold, the change is considered valid.

[0039] Specifically, by associating the semantic and visual features of remote sensing images learned through multimodal models, the abstract vector similarity calculation is directly correlated with the land cover change status of grid units. When the calculated similarity is lower than a preset threshold, it is confirmed that there is a valid change in the grid unit. This judgment method makes full use of the advantages of multimodal feature alignment, accurately filters out false changes caused by noise or small fluctuations, and ensures the authenticity and reliability of the changed area.

[0040] In step S6, the visualization includes: using color coding to highlight changing areas, using red to mark areas that decrease, green to mark areas that increase, and gray to mark areas that do not change; enabling the overlay display of multi-temporal images, supporting transparency adjustment and layer switching; dynamically calculating relevant data on the area and distribution of the target region, and displaying them in chart form.

[0041] Specifically, color coding is used to highlight changed areas in the target grid set: red indicates areas with decreased features, green indicates areas with increased features, and gray indicates areas with no change. Interactive displays allow users to quickly identify changes in areas of interest. Remote sensing images from different time points are overlaid in a multi-temporal manner, supporting transparency adjustment and layer switching, allowing users to intuitively compare the status of features at different times within the same view. Dynamic statistics on the area, distribution, and other relevant data of the target area are presented in chart form, eliminating the need for manual analysis and significantly improving the intuitiveness and practicality of search results. This lowers the operational threshold for non-professional users, enabling them to clearly and comprehensively grasp the features and changes in the target area, fully meeting the needs for visual presentation and in-depth analysis of search results.

[0042] The relational database with spatial extension modules is PostgreSQL+PostGIS.

[0043] Specifically, PostgreSQL combined with PostGIS spatial extension is used as a relational database with a spatial extension module. PostGIS's spatial data storage and processing capabilities are adapted to the spatial attribute storage requirements of grid cells. The remote sensing image data corresponding to the grid cells and its metadata, including the time of capture, sensor type, grid level, spatial boundary, and effective pixel count, are stored in a structured format. Using PostGIS's spatial indexing function, an efficient spatial index is built based on the spatial boundary information of the grid cells. At the same time, a time index is combined to achieve rapid filtering of spatiotemporal dimensions. The advantages of the spatial extension module are fully utilized to ensure reliable storage and efficient retrieval of grid cells and metadata. This provides stable data support for spatial range filtering and time range filtering in the subsequent retrieval process, ensuring the accuracy and efficiency of spatiotemporal matching during the retrieval process.

[0044] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for natural language intelligent retrieval of remote sensing images based on text-image alignment and grid segmentation, characterized in that, Includes the following steps: S1. Acquire multi-source remote sensing data, preprocess the multi-source remote sensing data, and form standardized data with a unified coordinate reference system and time stamp; S2. The standardized data is divided into grids to generate grid cells with unique codes. The meta-information of the grid cells is extracted, and the grid cells and meta-information are stored and a spatial index and a temporal index are constructed. S3. Use a multimodal model to extract features from the grid cells to obtain high-dimensional image feature vectors, store the high-dimensional feature vectors and construct a semantic vector index; S4. Receive a natural language query, parse the natural language query, extract key information and convert it into structured data, and mark whether it contains time-varying status. S5. Based on the time-change status markers of the structured data, execute the corresponding retrieval process to obtain the target grid set; S6. Visualize the target mesh set. 2.The natural language intelligent retrieval method for remote sensing images based on text-image alignment and grid segmentation according to claim 1, characterized in that, In step S1, the multi-source remote sensing data includes optical images, multispectral images, and SAR images; the preprocessing sequentially performs distortion correction, radiometric correction, registration, and fusion; the unified coordinate reference system is a preset coordinate system; and the time stamp is a standardized timestamp in UTC format.

3. The remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation according to claim 1, characterized in that, In step S2, the grid segmentation adopts the discrete global grid system segmentation method, the Geosot segmentation method, the quadtree segmentation method, or the block strategy based on administrative divisions; the unique code is a three-level code of hierarchy, region code, and unit code; the metadata includes shooting time, sensor type, grid level, spatial boundary, and effective pixel count; the grid unit and metadata are stored in a relational database with spatial expansion module, and the grid unit is pushed to the task queue to wait for feature extraction.

4. The remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation according to claim 1, characterized in that, In step S3, the multimodal model is a RemoteCLIP model based on CLIP fine-tuning or other contrastive language-image pre-trained models; the feature extraction involves taking grid cells in parallel from the task queue, inputting them into the image encoder of the multimodal model to obtain high-dimensional image feature vectors, and storing the high-dimensional image feature vectors in a vector database after normalization; the semantic vector index is constructed based on the approximate nearest neighbor algorithm, and the vector database is FAISS, Milvus, or QDrant.

5. The remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation according to claim 1, characterized in that, In step S4, the parsing adopts a combination of named entity recognition and parsing rules; the key information includes time range, spatial range, land cover type and change status; the structured data is in JSON format, containing time range start and end information, spatial geometric coordinate information, target type information and change status marker information; the time change status marker is a Boolean value.

6. The method for intelligent natural language retrieval of remote sensing images based on image-text alignment and grid segmentation according to claim 1, characterized in that, In step S5, when the time change status is marked as false, the retrieval process includes: The target feature type and spatial extent description in the structured data are input into the multimodal model to generate text feature vectors; The set of grid cells that intersects with the query spatial range is filtered by the spatial index, and the set of grid cells that are within the query time range is filtered by the time index. By performing a nearest neighbor query on the text feature vector using the semantic vector index, a set of semantically similar grid cells is obtained; The target mesh set is obtained by taking the intersection of the three sets.

7. The remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation according to claim 1, characterized in that, In step S5, when the time change status is true, the retrieval process includes: The natural language query is broken down into multiple subqueries with no time-varying states using a large language model, with each subquery corresponding to a single point in time. Execute each sub-query according to the retrieval process to obtain the grid cell set at the corresponding time point; Perform intersection operations, depth segmentation network operations, or time-series-based detection network operations on multiple sets of said grid cells to obtain a set of grid cells representing the changed regions; Remove scattered grid cells with an area smaller than a preset threshold, verify the effectiveness of the changes in the remaining grid cells, and obtain the target grid set.

8. The remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation according to claim 7, characterized in that, The validity of the change is determined by calculating the similarity of the high-dimensional image feature vectors of the corresponding grid cells before and after the change. If the similarity is lower than a preset threshold, the change is confirmed to be valid.

9. The remote sensing image natural language intelligent retrieval method based on image-text alignment and grid segmentation according to claim 1, characterized in that, In step S6, the visualization includes: highlighting changing areas using color coding, marking decreasing areas with red, increasing areas with green, and areas with no change with gray; displaying multi-temporal images overlaid, supporting transparency adjustment and layer switching; dynamically calculating relevant data on the area and distribution of the target region, and displaying them in chart form.

10. The method for intelligent natural language retrieval of remote sensing images based on image-text alignment and grid segmentation according to claim 3, characterized in that, The relational database with spatial expansion module is PostgreSQL+PostGIS.

Images