A remote sensing image retrieval method and device, electronic equipment and medium

Abstract

This application discloses a method, apparatus, electronic device, and medium for retrieving remote sensing images. The method includes: acquiring a retrieval request for remote sensing images; parsing the retrieval request using a target big data model to obtain initial retrieval conditions; determining the target geofence for the current image retrieval based on the initial retrieval conditions and a pre-constructed point-of-interest (POI) database; and performing a search in the remote sensing image database based on the initial retrieval conditions and the target geofence to obtain the target remote sensing image. Thus, by using a target big data model, retrieval requests with complex geographical descriptions and ambiguous textual descriptions can be parsed, thereby accurately understanding the user's remote sensing image retrieval intent. Simultaneously, by combining this with a pre-constructed POI database, a precise target geofence can be obtained, locking the remote sensing retrieval within a more accurate geographical location range, thereby improving the retrieval accuracy of the target remote sensing image.

Description

Technical Field

[0001] This application relates to the field of remote sensing data processing technology, and in particular to a method, apparatus, electronic device and medium for retrieving remote sensing images. Background Technology

[0002] Remote sensing image retrieval is a core component of intelligent geographic information processing, with broad and significant applications in fields such as agricultural monitoring, disaster assessment, and urban planning. With the rapid development of satellite sensor technology, remote sensing data is becoming increasingly diversified, multimodal (e.g., optical, SAR, hyperspectral, and time-series imagery), and massive in volume, placing higher demands on remote sensing image retrieval.

[0003] Currently, remote sensing image retrieval is mainly based on keywords, location coordinates, and low-level visual features (such as color and texture). This approach struggles to interpret complex user search requests, resulting in low retrieval accuracy. Furthermore, when a user's search request text is ambiguous, the search intent is often unclear, further hindering the image retrieval accuracy from meeting user expectations.

[0004] Therefore, how to improve the accuracy of remote sensing image retrieval and meet users' expectations for high-precision retrieval is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, one aspect of this application provides a method for retrieving remote sensing images, the method comprising:

[0006] Retrieve a remote sensing image retrieval request;

[0007] The search request is parsed using a target large model to obtain initial search conditions;

[0008] Based on the initial search criteria and the pre-built database of points of interest, the target geofence for the current image search is determined;

[0009] Based on the initial search criteria and the target geofence, a search is performed in the remote sensing image database to obtain the target remote sensing image.

[0010] Optionally, the initial search criteria include geographic attribute criteria, and determining the target geofence for the current image search based on the initial search criteria and a pre-built point-of-interest database includes:

[0011] When the geographic attribute conditions include explicit conditions, a search is performed in the point of interest database based on the geographic attribute conditions to obtain the geographic location information of the current image retrieval;

[0012] When the geographic attribute conditions include implicit conditions, the first inference condition is obtained by reasoning the geographic attribute conditions through the target large model; and based on the first inference condition, a search is performed in the point of interest database to obtain the geographic location information.

[0013] The target geofence is generated based on the geographic location information.

[0014] Optionally, the geographic location information includes the target POI, the metadata information corresponding to the target POI, and the target POI type. Generating the target geofence based on the geographic location information includes:

[0015] Determine the geographic vector boundary based on the target POI and the metadata information;

[0016] Based on the target POI type, the geographic vector boundary is adjusted to obtain the initial geofence;

[0017] Determine whether the initial search criteria include non-geographic attribute criteria;

[0018] If so, the initial geofence is trimmed according to the non-geographical attribute conditions to obtain the target geofence;

[0019] If not, the initial geofence shall be used as the target geofence.

[0020] Optionally, adjusting the geographic vector boundary according to the target POI type to obtain an initial geofence includes:

[0021] When the target POI is a point type, the geographic vector boundary is expanded by a specified radius with the center of the target POI as the center to obtain the initial geofence;

[0022] When the target POI type is linear, the geographic vector boundary is extended by a specified distance along two directions perpendicular to the linear POI to obtain the initial geofence;

[0023] When the target POI is of the isal type, the geographic vector boundary is used as the initial geofence.

[0024] Optionally, the step of trimming the initial geofence based on the non-geographic attribute conditions includes:

[0025] When the non-geographic attribute conditions include explicit conditions, the first constraint condition corresponding to the non-geographic attribute conditions is obtained through an external data interface; and the initial geofence is trimmed based on the first constraint condition.

[0026] When the non-geographical attribute conditions include implicit conditions, the second inference condition is obtained by reasoning about the non-geographical attribute conditions through the target large model; the second constraint condition corresponding to the second inference condition is obtained through the external data interface; and the initial geofence is trimmed based on the second constraint condition.

[0027] Optionally, the step of constructing the point of interest database includes:

[0028] Obtain POI data;

[0029] Extract core geographic fields from the POI data; and use the specified association information of the core geographic fields in the POI data as metadata information;

[0030] Assign corresponding weights to the different types of metadata information;

[0031] Based on the weights and metadata information, the core geographic fields are vectorized to obtain POI vectors;

[0032] A vector index is constructed for the POI vectors to obtain the point of interest database.

[0033] Optionally, the remote sensing image retrieval method includes:

[0034] Obtain the user's input feedback signal regarding the current image retrieval;

[0035] If the feedback signal is a signal to adjust the target geofence, the point of interest database is adjusted according to the feedback signal;

[0036] If the feedback signal indicates dissatisfaction with the current image retrieval, the following steps are performed:

[0037] Collect historical data of the current conversation;

[0038] By using a specified model, reasoning is performed on the historical data to determine the current retrieval blind spot; an external data interface corresponding to the current retrieval blind spot is added;

[0039] Based on the historical data, a training dataset is constructed; and the target large model is fine-tuned using the training dataset.

[0040] Another aspect of this application provides a remote sensing image retrieval device, the device comprising:

[0041] The retrieval request acquisition module is used to acquire retrieval requests for remote sensing images;

[0042] The retrieval request parsing module is used to parse the retrieval request through the target large model to obtain initial retrieval conditions;

[0043] The target geofence determination module is used to determine the target geofence for the current image retrieval based on the initial retrieval conditions and a pre-built point of interest database.

[0044] The target remote sensing image retrieval module is used to retrieve the target remote sensing image from the remote sensing image database based on the initial retrieval conditions and the target geofence.

[0045] Another aspect of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the remote sensing image retrieval method.

[0046] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the remote sensing image retrieval method.

[0047] The remote sensing image retrieval method, apparatus, electronic device, and medium provided in this application have the following beneficial effects: by using a target large model to parse retrieval requests for complex geographical descriptions and fuzzy textual descriptions, the user's remote sensing image retrieval intent can be accurately understood. At the same time, by combining a pre-built point of interest database, a precise target geofence can be obtained, locking the remote sensing retrieval within a more precise geographical location range, thereby improving the retrieval accuracy of the target remote sensing image. Attached Figure Description

[0048] Figure 1 A schematic flowchart illustrating a remote sensing image retrieval method provided in an embodiment of this application;

[0049] Figure 2 A schematic diagram illustrating the principle of a remote sensing image retrieval method provided in an embodiment of this application;

[0050] Figure 3 A schematic diagram illustrating the principle of determining a target geofence, provided as an embodiment of this application;

[0051] Figure 4 A schematic diagram illustrating the principle of adjusting geographic vector boundaries provided in an embodiment of this application;

[0052] Figure 5 A schematic diagram illustrating the principle of constructing a point of interest database as provided in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram illustrating the principle of determining the current search blind zone, provided in an embodiment of this application.

[0054] Figure 7 A schematic diagram of the structure of a remote sensing image retrieval device provided in an embodiment of this application;

[0055] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0056] The reference numerals in the attached diagram are as follows: 70 is the retrieval request acquisition module, 71 is the retrieval request parsing module, 72 is the target geofence determination module, 73 is the target remote sensing image retrieval module, 80 is the memory, 81 is the processor, 82 is the display screen, 83 is the input / output interface, 84 is the communication interface, 85 is the power supply, 86 is the communication bus, 801 is the computer program, 802 is the operating system, and 803 is the data. Detailed Implementation

[0057] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0058] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0059] Figure 1 The following is a flowchart illustrating a remote sensing image retrieval method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0060] S10: Request a remote sensing image retrieval request;

[0061] S11: Analyze the search request using the target large model to obtain initial search conditions;

[0062] Figure 2 The schematic diagram illustrates the principle of a remote sensing image retrieval method provided in this application embodiment. Figure 2As shown in the specific embodiment, the user inputs a retrieval request for remote sensing images through the terminal. The retrieval request can be in the form of text, images, and voice, or of course, it can be a multimodal retrieval request. This application does not limit this.

[0063] Furthermore, to ensure accurate understanding of the retrieval request, in one optional embodiment, the retrieval request can be parsed using a target big model to generate structured initial retrieval conditions, i.e., retrieval constraints. The target big model may include, but is not limited to, the GPT series, Tongyi Qianwen, and DeepSeek; this application does not limit its scope.

[0064] In one alternative embodiment, the target large model can be a pre-trained model in the remote sensing domain. Specifically, an initial large model is pre-built, and a large training dataset in the remote sensing domain is collected to train the initial large model, thereby obtaining the target large model. This target large model can thus more accurately interpret specialized knowledge in the remote sensing domain, improving the accuracy of remote sensing image retrieval.

[0065] In a specific embodiment, when parsing the retrieval request through the target large model, a prompt word project is constructed. Based on the prompt word project, the retrieval request is input into the target large model for inference, thereby obtaining initial retrieval conditions. These initial retrieval conditions are used to narrow down and lock onto the target remote sensing image. Examples will be provided below for ease of understanding.

[0066] For example, the search request is to download a cloudless Gaofen-2 image of the Yangtze River Delta in June 2023. The target large model is ChatGLM-6B. The prompt word engineering is:

[0067] "You are a remote sensing image retrieval expert. Please parse the following user's search request into structured initial search criteria:"

[0068] User-input search request: {input text}

[0069] Output JSON format: {time, geographic range, cloud cover, sensor type...}

[0070] Output structured initial search criteria:

[0071] {

[0072] "time": {"start": "2023-06-01", "end": "2023-06-30"},

[0073] "geo": {"type": "region", "name": "Yangtze River Delta"},

[0074] "cloud_cover": {"max": 5},

[0075] "sensor": "Gaofen 2"

[0076] }

[0077] In one optional embodiment, the initial search conditions are checked using a target large model. If there are conflicts in the initial search conditions, the initial conditions can be corrected using the target large model. Alternatively, the conflict results can be returned to the terminal so that the user can adjust and re-enter the search request, thereby improving the accuracy of subsequent remote sensing image retrieval.

[0078] For example, if the initial search condition is "search for images of plateau landforms in Shanghai", this initial search condition obviously has a logical conflict. In this case, it can be corrected using a target model. For example, "search for images of plateau landforms in Shanghai" can be changed to "search for images of mountains in Shanghai". Of course, the conflict can also be returned to the terminal for the user to actively modify.

[0079] S12: Determine the target geofence for the current image retrieval based on the initial search criteria and the pre-built database of points of interest;

[0080] S13: Based on the initial search criteria and the target geofence, search the remote sensing image database to obtain the target remote sensing image.

[0081] Furthermore, such as Figure 2 As shown, to further improve the retrieval accuracy of remote sensing images, an optional embodiment introduces a point-of-interest (POI) database. This POI database stores geographic location-related information. The storage format of the POI database is not limited in this application and may include, but is not limited to, vectorized storage or tabular storage. In a specific embodiment, the initial search criteria and the POI database are combined to determine the target geofence for the current image retrieval.

[0082] Specifically, based on the initial search criteria, a search is performed in the Point of Interest (POI) database, which narrows down the scope of the current image search, i.e., determines the geographical area of ​​the current image search. Therefore, based on the initial search criteria and the target geofence, a search in the remote sensing image database can accurately obtain the target remote sensing image needed by the user.

[0083] In one optional embodiment, after acquiring the target remote sensing image, historical retrieval data is recorded. Specifically, this includes the retrieval request entered during the retrieval dialogue, the initial retrieval conditions parsed from the target large model, the hash value of the retrieval results, and the target remote sensing image. It is understood that acquiring historical retrieval data allows for the evaluation of retrieval quality and the adjustment and optimization of data stored in the point of interest database, thereby improving retrieval accuracy through continuous retrieval processes.

[0084] Therefore, the remote sensing image retrieval method provided in this application realizes the parsing of retrieval requests for complex geographical descriptions and fuzzy textual descriptions through a target large model, thereby accurately understanding the user's remote sensing image retrieval intent. At the same time, combined with a pre-built point of interest database, it obtains accurate target geofences, locking the remote sensing retrieval within a more precise geographical location range, thereby improving the retrieval accuracy of target remote sensing images.

[0085] In one optional embodiment, the initial search criteria include geographic attribute criteria. Based on the initial search criteria and a pre-built point-of-interest database, the target geofence for the current image search is determined, including:

[0086] When the geographic attribute conditions include explicit conditions, a search is performed in the point of interest database based on the geographic attribute conditions to obtain the geographic location information of the current image search.

[0087] When the geographic attribute conditions include implicit conditions, the first inference condition is obtained by reasoning about the geographic attribute conditions through the target large model; and based on the first inference condition, the geographic location information is obtained by searching the point of interest database.

[0088] Generate a target geofence based on geographic location information.

[0089] Figure 3 This is a schematic diagram illustrating the principle of determining a target geofence, provided in an embodiment of this application. In a specific embodiment, as shown... Figure 3 As shown, the initial search criteria include geographic attribute criteria, which refer to criteria containing geographic information. However, geographic attribute criteria include at least one of explicit criteria and implicit criteria.

[0090] Explicit conditions refer to information that can be directly extracted and transformed into search criteria. Explicit conditions include, but are not limited to, specific time ranges, administrative divisions, remote sensing satellite types, remote sensing image types, and remote sensing image imaging modes. Implicit conditions refer to information that cannot be directly extracted but is still a key search criterion, including but not limited to vague time ranges (e.g., seasons combined with geographical location), place name alternatives (e.g., Pearl River Delta), and descriptions containing geographical types (e.g., mountainous areas and tidal flat areas).

[0091] In one optional embodiment, when the geographic attribute condition includes explicit conditions, the initial search conditions characterizing the current parsing and extraction can be directly used for searching the point-of-interest (POI) database to obtain the geographic location information of the current image retrieval. For example, if the geographic attribute condition is "retrieve remote sensing images of Beijing", a search is performed in the POI database based on this geographic attribute condition to obtain geographic location information about Beijing, which may include, but is not limited to, the latitude and longitude of Beijing.

[0092] In another optional embodiment, when the geographic attribute conditions include implicit conditions, the current initial search conditions cannot be directly retrieved from the point of interest database; that is, the current geographic attribute conditions are fuzzy conditions. Therefore, as... Figure 3 As shown, implicit geographic attribute conditions can be inferred from the target large model to obtain the first inference condition. Furthermore, based on the first inference condition, a search is performed in the point of interest database to obtain geographic location information. For ease of understanding, an example will be provided below.

[0093] For example, when the geographic attribute condition is "retrieve remote sensing images of Shanghai," Shanghai is clearly a geographic attribute condition. However, this condition is implicit, meaning it cannot be directly used as a retrieval condition in the Point of Interest (POI) database. Therefore, we can use a target-based large model to infer that the first inference condition is Shanghai, i.e., Shanghai refers to Shanghai. Thus, "retrieve remote sensing images of Shanghai" can be transformed into "retrieve remote sensing images of Shanghai," at which point we can retrieve Shanghai's geographic location information from the POI database.

[0094] In one optional embodiment, when reasoning through the target large model, preset typical steps of search conditions (such as time, administrative division, place name, geographical features, satellite type, image type, etc.) can be constructed, and LLM can be used to read and analyze the user's historical search records to perform dynamic search condition reasoning.

[0095] In one optional embodiment, the geographic location information includes the target POI (Point of Interest), the metadata information corresponding to the target POI, and the target POI type. Based on the geographic location information, a target geofence is generated, including:

[0096] Determine the geographic vector boundaries based on the target POI and metadata information;

[0097] Based on the target POI type, the geographic vector boundary is adjusted to obtain the initial geofence;

[0098] Determine whether the initial search criteria include non-geographical attribute criteria;

[0099] If so, the initial geofence is trimmed based on non-geographical attribute conditions to obtain the target geofence;

[0100] If not, use the initial geofence as the target geofence.

[0101] In specific embodiments, it is understood that the geographic vector boundary can be determined based on the target POI and its corresponding metadata information; that is, the retrieval boundary area of ​​the current remote sensing image can be determined. Here, metadata information refers to the relevant geographic information of the target POI, i.e., more detailed geographic information description fields. For example, when the target POI is Beijing and the metadata information is the latitude and longitude of Beijing, the geographic vector boundary regarding Beijing can be locked based on the target POI and the metadata information.

[0102] It should be noted that in specific embodiments, there may be multiple target POIs. In this case, multiple target POIs can be returned to the terminal for the user to confirm the POI used to generate the target geofence.

[0103] like Figure 3 As shown, in a specific embodiment, in addition to geographic attribute conditions, the initial search conditions may also include non-geographic attribute conditions. It is understood that although non-geographic attribute conditions do not contain information about geographical location, they can still affect the accuracy of the target geofence generation. For ease of understanding, examples will be provided below.

[0104] For example, when the non-geographical attribute condition is time, specifically, the initial search condition is "search for remote sensing images of Beijing in June 2023". Obviously, the geographic attribute condition is Beijing, and the non-geographical attribute condition is June 2023. Clearly, the time limit has a significant impact on the search accuracy.

[0105] Therefore, in one optional embodiment, to further improve the retrieval accuracy of remote sensing images, before generating the target geofence, it is first determined whether the initial retrieval conditions include non-geographic attribute conditions. If so, the geographic vector boundary is adjusted according to the target POI type to obtain the initial geofence. Specifically, the adjustment of the geographic vector boundary can be boundary expansion, reduction, etc., which is not limited in this application. In addition, in specific embodiments, adjustments can be made using a large model or manually, which is also not limited in this application.

[0106] Furthermore, the initial geofence is trimmed based on non-geographical attribute conditions to obtain the target geofence. Specifically, the initial geographical location is refined based on non-geographical attribute conditions, i.e., narrowed down to a more precise geographical area to obtain the target geofence. An example will be provided below for clarity.

[0107] For example, if the geographic attribute condition is "Yangtze River Delta" and the non-geographic attribute condition is "dates with cloud cover ≤5% in June 2023", then first, a search is performed in the Point of Interest (POI) database based on the Yangtze River Delta, finding the geographic vector boundary to be between 118° and 123° east longitude and 29° and 33° north latitude. Further, based on the target POI type, the geographic vector boundary is adjusted to obtain an initial geofence. Then, using the date with cloud cover ≤5% in June 2023, the initial geofence is trimmed to obtain the accurate target geofence.

[0108] Figure 4 This application provides a schematic diagram illustrating the principle of adjusting geographic vector boundaries. Based on the above embodiments, as an optional embodiment, the geographic vector boundaries are adjusted according to the target POI type to obtain an initial geofence, including:

[0109] When the target POI is a point type, the geographic vector boundary is expanded with a specified radius using the center of the target POI as the center to obtain the initial geofence;

[0110] When the target POI is a linear type, the geographic vector boundary is extended by a specified distance along two directions perpendicular to the linear POI to obtain the initial geofence;

[0111] When the target POI is of the isal type, the geographic vector boundary is used as the initial geofence.

[0112] Understandably, the accuracy of the target geofence is crucial to the precision of remote sensing image retrieval. Specifically, if the target geofence is smaller than the actual geographic area being retrieved, that is, if the target geofence does not completely encompass the actual geographic area being retrieved, the retrieval may fail.

[0113] Therefore, to solve the above-mentioned technical problems, based on the above embodiments, after initially obtaining the geographic vector boundary, the geographic vector boundary can be adjusted according to the target POI type. Specifically, as follows... Figure 4 As shown, when the target POI type is a point type, for example, the geometry of Beijing Station on the POI is a point, then it is appropriate to expand outward from the target POI as the center, so that the initial geofence is larger than the geographic vector boundary.

[0114] In one optional embodiment, the geographic vector boundary can be extended by a specified radius with the center of the target POI as the center. The specified radius can be selected by the user, for example, it can be set to 5 kilometers, and this application does not limit this.

[0115] like Figure 4As shown, when the target POI type is linear, it may represent a river, subway line, etc., with a line geometry within the POI. For example, Beijing Subway Line 1 has a line geometry within the POI. To improve image detection accuracy, in one optional embodiment, the geographic vector boundary can be extended by a specified distance along two directions perpendicular to the linear POI. In fact, linear extension can be understood as thickening the initial linear POI.

[0116] For areal-type target POIs, such as Olympic Forest Park, it's understandable that for POIs with a defined area, the resulting geographic vector boundary often represents the precise geographic region. Therefore, as... Figure 4 As shown, for area-type target POIs, the geographic vector boundary can be directly used as the initial geofence, i.e., the target POI is not modified. Of course, in an optional embodiment, for area-type target POIs, an extension with a specified radius can be made using the center point of the target POI as the center.

[0117] Therefore, the remote sensing image retrieval method provided in this application, after initially obtaining the geographic vector boundary, adjusts the geographic vector boundary according to the target POI type, which can ensure that the remote sensing image that the user wants to retrieve must fall within the target geofence, thereby improving the image detection accuracy.

[0118] Based on the above embodiments, as an optional embodiment, the initial geofence is trimmed according to non-geographic attribute conditions, including:

[0119] When non-geographic attribute conditions include explicit conditions, the first constraint condition corresponding to the non-geographic attribute condition is obtained through an external data interface; and the initial geofence is trimmed based on the first constraint condition.

[0120] When non-geographical attribute conditions include implicit conditions, the second inference condition is obtained by reasoning about the non-geographical attribute conditions through the target large model; the second constraint condition corresponding to the second inference condition is obtained through the external data interface; and the initial geofence is trimmed based on the second constraint condition.

[0121] like Figure 3 As shown in the specific embodiment, when there are non-geographic attribute conditions, in order to further improve the accuracy of the target geofence, it is necessary to trim the initial geofence obtained in the above embodiment according to the non-geographic attribute conditions.

[0122] It is understood that non-geographic attribute conditions may also include at least one of explicit and implicit conditions. In an optional embodiment, when the non-geographic attribute conditions include explicit conditions, external data can be directly connected through an external data interface to obtain a first constraint condition regarding the non-geographic attribute conditions, so as to trim the initial geofence according to the first constraint condition. The external data includes, but is not limited to, meteorological and traffic data, and the external data can perform spatial logic operations on the geofence.

[0123] In another alternative embodiment, such as Figure 3 As shown, when non-geographical attribute conditions include implicit conditions, the constraints cannot be directly obtained from external data. In this case, the implicit conditions can be inferred through the target large model to obtain the second inference condition, that is, the implicit conditions are first transformed into explicit conditions. Furthermore, based on the second inference condition, the corresponding second constraint condition is obtained through the external data structure. Thus, the initial geofence can be trimmed based on the second constraint condition. For ease of understanding, an example will be given below.

[0124] For example, if a user's search request is "find vegetation cover images within a 10km radius of the northern slope of the Qinling Mountains in 2022", the target large model will analyze the data and obtain the keywords "northern slope of the Qinling Mountains", "10km", "2022", and "vegetation cover". Among these, "Qinling" is an explicit geographic attribute; therefore, searching for "Qinling" in the point of interest database yields the geographic vector boundary. Simultaneously, DEM (Digital Elevation Model) data is used to calculate the northern slope area with a slope greater than 15°.

[0125] Furthermore, the north slope vector boundary is extended by 10 kilometers, i.e., the geographic vector boundary is adjusted to obtain the initial geographic fence. It is understandable that "vegetation cover" is an implicit non-geographic attribute condition. This can be inferred first through the target large model to generate a second inference condition: "vegetation cover with NDVI (Normalized Difference Vegetation Index) > 0.3". Based on this second inference condition, external data is accessed to trim the initial geographic fence, resulting in the target geographic fence.

[0126] It should be noted that when cropping geofences based on external data, conflicts may arise. In such cases, recommendations can be automatically generated by the target large model, or the user can select the final inference condition. Furthermore, after the user makes a selection, the current choice is recorded, and the priority of that inference condition in subsequent target large models is updated. An example will be provided below for clarity.

[0127] For example, if a user's search request is "obtain optical imagery of Beijing on January 1, 2023, with a resolution of 2 meters," after parsing the target large model, the structured initial search conditions include "Time = 2023-01-01, Geographical range = Beijing, Resolution = 2 meters, Sensor type = Optical." After processing external data, it is determined that there is no 2-meter resolution satellite transit data for Beijing on January 1, 2023 (e.g., only 5-meter resolution data is available).

[0128] At this point, two inference conditions can be generated: Option 1 recommends January 2, 2023 (2-meter resolution, 10% cloud cover), and Option 2 recommends January 1, 2023 (5-meter resolution, 0% cloud cover). The user can then choose the final inference condition, and the user's choice is recorded. The analysis weights of the target large model are updated, for example, by increasing the weight of resolution to improve the priority of subsequent resolutions.

[0129] Finally, by combining explicit geographic attribute conditions, the first inference condition, the second inference condition, and the target geofence, a search is performed in the remote sensing image database to obtain the target remote sensing image.

[0130] Figure 5 This is a schematic diagram illustrating the principle of constructing a point of interest (POI) database, provided as an embodiment of this application. In an optional embodiment, the steps for constructing the POI database provided in this application include:

[0131] Obtain POI data;

[0132] Extract core geographic fields from POI data; and use the specified association information of the core geographic fields in the POI data as metadata information;

[0133] Assign corresponding weights to different types of metadata information;

[0134] Based on weights and metadata information, the core geographic fields are vectorized to obtain POI vectors;

[0135] Build vector indexes for POI vectors to obtain a database of points of interest.

[0136] In a specific embodiment, such as Figure 5 As shown, when building a Points of Interest (POI) database, data collection begins. Specifically, POI data is obtained from open-source databases, including but not limited to Amap and Baidu Maps. POI data includes, but is not limited to, attribute fields such as POI name, alias category, function, provincial / municipal / city-level administrative divisions, and administrative region and latitude / longitude coordinates. For ease of understanding, examples will be provided below.

[0137] For example, the extracted POI data is:

[0138] {

[0139] "name": "Peking University",

[0140] "alias": ["Peking University", "PKU"],

[0141] "category": "Education",

[0142] "latitude": 39.9996,

[0143] "longitude": 116.3164,

[0144] "admin_code": "110108" / / Code of Haidian District;

[0145] }

[0146] As Figure 5 shown, after obtaining the initial POI data, the data is cleaned to ensure the data quality of the POI database. Further, core geographic fields are extracted from the cleaned POI data, and at the same time, the specified associated information of the core geographic fields is used as metadata information.

[0147] The core geographic fields include, but are not limited to, attribute fields such as the name, alias category, function, and multi-level administrative divisions of provinces, cities, and districts of the POI, while the metadata information includes, but is not limited to, administrative regions and longitude and latitude. In a specific embodiment, the metadata information can be understood as detailed geographic description information of the core geographic fields. It should be noted that the core geographic fields not only include the POI name. In fact, in order to improve the POI retrieval accuracy, the core geographic fields integrate multiple POI geographic information.

[0148] In an optional embodiment, in order to achieve fast retrieval of target POIs, the POI database can be a vectorized database. Therefore, in a specific embodiment, it is necessary to vectorize the core geographic fields to obtain POI vectors.

[0149] Specifically, in an optional embodiment, in order to meet the retrieval requirements of different actual target POIs, different weights are assigned to different types of metadata information. That is, according to the functional requirements of the retrieval platform, different weight levels are assigned, and different weight levels are used to enhance the attention degree for different types of metadata information.

[0150] Furthermore, the core geographic fields are vectorized by specifying a model, which can be the Sentence-BERT model, and this application is not limited to this. In an optional embodiment, a geographic terminology corpus (e.g., a Chinese geographical terminology dictionary) can be injected during vectorization to generate high-precision POI vectors.

[0151] It should be noted that, in specific embodiments, metadata information can be added or deleted according to actual business needs, and this application does not limit the type and quantity of metadata information. Furthermore, this application does not specifically limit the specific content included in the vectorized core geographic fields.

[0152] After obtaining the POI vector, further steps, such as... Figure 5 As shown, a vector index is constructed for POI vectors. Specifically, in one optional embodiment, the HNSW algorithm can be used to build the vector index, and a tree-structured hierarchical index can be constructed according to administrative division codes (province-city-district / county) to obtain the Point of Interest database. This application does not limit the method of index construction.

[0153] In one optional embodiment, the remote sensing image retrieval method provided in this application includes:

[0154] Obtain feedback signals from the user regarding the current image retrieval;

[0155] If the feedback signal indicates an adjustment to the target geofence, the point of interest database is adjusted accordingly.

[0156] If the feedback signal indicates dissatisfaction with the current image retrieval, proceed with the following steps:

[0157] Collect historical data of the current conversation;

[0158] By specifying a model, inference is performed on historical data to determine the current search blind spot; an external data interface corresponding to the current search blind spot is added;

[0159] A training dataset is constructed based on historical data; and the target large model is fine-tuned using the training dataset.

[0160] To further improve the accuracy of user intent parsing, the remote sensing image detection method provided in this application offers a dynamic index optimization strategy based on user feedback. This strategy can dynamically optimize the parsing weights (i.e., parsing focus) of point of interest data and target large model based on user feedback signals on search results.

[0161] Specifically, in one optional embodiment, if a user actively adjusts the target geofence, the adjustment can be associated with the corresponding POI vector in the Points of Interest (POI) database, thereby increasing the vector's ranking priority in subsequent searches. Alternatively, based on feedback signals, the weights of different metadata information in the POI database can be adjusted to increase the priority of metadata types that the user is currently more interested in during subsequent searches.

[0162] In another optional embodiment, when the feedback signal indicates that the user is dissatisfied with the current image retrieval, reinforcement learning fine-tuning of the target large model is triggered, thereby reducing the confidence weight of similar conditions in the large model analysis. Dissatisfaction with the retrieval can manifest as the user repeatedly asking questions, interrupting the conversation, or modifying specified conditions. These specified conditions include, but are not limited to, incorrect retrieval time requirements, incorrect POI names, excessive geofencing bias in POI generation, and incorrect image type.

[0163] Specifically, when fine-tuning the target large model, all historical data of the current dialogue is first collected, and a training dataset is constructed based on the historical data. This training dataset can then be used to fine-tune the target large model. In a specific embodiment, the training dataset can be generated based on a pre-built dataset architecture. For ease of understanding, an example will be provided below.

[0164] For example, users can revise and supplement the search criteria parsed by the engine. The collected historical data includes:

[0165] Dialogue content:

[0166] User: Search for all optical images covering Hangzhou in 2024.

[0167] System: Please list the types of satellites relevant to imaging.

[0168] User: High Score No. 2.

[0169] System: Please list the imaging types of the images.

[0170] User: No input (i.e., abandoning the conversation).

[0171] Based on the historical data above, it can be seen that the user's request was simply to retrieve all optical images. However, the system's response involved excessive questioning regarding two inaccurate search criteria, causing the user to abandon the search.

[0172] At this point, a training dataset is constructed based on historical data:

[0173] {

[0174] "instruction": "Analyze and extract user input regarding the image requirements for the desired retrieval. If the user does not specifically mention satellite type and imaging type, no further information will be required from the user; the default is to retrieve all images."

[0175] "input": "Retrieve all optical images covering Hangzhou in 2024".

[0176] "output": "Intended decomposition result, time: 2024-01-01, 2024-12-32, region: Hangzhou, object: optical image"

[0177] }

[0178] It should be noted that, in order to conserve computing resources, reinforcement learning training of the target large model can be triggered only when the training dataset reaches the specified storage space. During training, in one optional embodiment, the target large model is fine-tuned using the QLoRA (Quantized Low-Rank Adaptation) method, but this application does not limit the scope of the application.

[0179] In another alternative embodiment, in addition to fine-tuning the target large model, a new external data interface can be introduced to further improve the accuracy of subsequent image retrieval by identifying the current retrieval blind zone.

[0180] Specifically, by specifying a model, inference is performed on historical data to determine the current retrieval blind spot, thereby allowing the addition of external data interfaces corresponding to the current retrieval blind spot. The specified model can be an LSTM (Long Short-Term Memory) network, but this application does not limit its scope.

[0181] Figure 6 This is a schematic diagram illustrating the principle of determining the current retrieval blind spot provided in an embodiment of this application. When LSTM performs specific reasoning and analysis on historical data, it needs to first concatenate and text-ize the historical data. Specifically, for example... Figure 6 As shown, historical data is first preprocessed, then feature filtering is performed to identify paragraphs rejected by the user, and these rejected paragraphs are then marked. Further, a dialogue dataset is constructed, which can also be built based on a pre-built architecture; this application does not limit this approach. Then, the dialogue dataset is input into an LSTM for analysis to determine the current retrieval blind spot.

[0182] For example, if the current target model cannot parse content related to natural disasters, then the current search blind spot can be determined by processing and analyzing the current historical data. In this case, an external data interface for natural disasters can be added to provide data support for subsequent searches.

[0183] In the above embodiments, the method for retrieving remote sensing images has been described in detail. This application also provides an embodiment of a remote sensing image retrieval device.

[0184] Figure 7 This is a schematic diagram of the structure of a remote sensing image retrieval device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device includes:

[0185] The retrieval request acquisition module 70 is used to acquire retrieval requests for remote sensing images;

[0186] The retrieval request parsing module 71 is used to parse the retrieval request through the target large model to obtain the initial retrieval conditions;

[0187] The target geofence determination module 72 is used to determine the target geofence for the current image retrieval based on the initial search conditions and the pre-built point of interest database.

[0188] The target remote sensing image retrieval module 73 is used to retrieve the target remote sensing image from the remote sensing image database based on the initial retrieval conditions and the target geofence.

[0189] Furthermore, the remote sensing image retrieval device provided in this application embodiment also includes:

[0190] The geographic location information determination module is used to retrieve the geographic location information of the current image retrieval by searching the point of interest database based on the geographic attribute conditions when the geographic attribute conditions include explicit conditions; when the geographic attribute conditions include implicit conditions, it uses the target large model to infer the geographic attribute conditions to obtain the first inference condition; and retrieves the geographic location information based on the first inference condition in the point of interest database.

[0191] The fence generation module is used to generate a target geofence based on geographic location information.

[0192] The initial geofence determination module is used to determine the geographic vector boundary based on the target POI and metadata information; and adjust the geographic vector boundary according to the target POI type to obtain the initial geofence.

[0193] The search criteria determination module is used to determine whether the initial search criteria include non-geographical attribute conditions; if so, the initial geofence is cropped according to the non-geographical attribute conditions to obtain the target geofence; if not, the initial geofence is used as the target geofence.

[0194] The first extension module is used to extend the geographic vector boundary with a specified radius, using the center of the target POI as the center, when the target POI is a point type, to obtain the initial geofence;

[0195] The second extension module is used to extend the geographic vector boundary by a specified distance along two directions perpendicular to the linear POI when the target POI type is linear, so as to obtain the initial geofence;

[0196] The first processing module is used to use the geographic vector boundary as the initial geofence when the target POI type is an area type.

[0197] The first trimming module is used to obtain the first constraint condition corresponding to the non-geographic attribute condition through an external data interface when the non-geographic attribute condition includes explicit conditions; and to trim the initial geofence based on the first constraint condition.

[0198] The second trimming module is used to obtain the second inference condition by reasoning about the non-geographical attribute condition through the target large model when the non-geographical attribute condition includes implicit conditions; obtain the second constraint condition corresponding to the second inference condition through the external data interface; and trim the initial geofence based on the second constraint condition.

[0199] The POI data acquisition module is used to acquire POI data.

[0200] The data extraction module is used to extract core geographic fields from POI data and to use the specified association information of the core geographic fields in the POI data as metadata information.

[0201] The weight assignment module is used to assign corresponding weights to different types of metadata information;

[0202] The vectorization module is used to vectorize core geographic fields based on weights and metadata information to obtain POI vectors;

[0203] The index building module is used to build vector indexes for POI vectors, resulting in a database of points of interest.

[0204] The feedback signal acquisition module is used to acquire the user's input feedback signal regarding the current image retrieval;

[0205] The adjustment module is used to adjust the point of interest database according to the feedback signal when the feedback signal is a signal to adjust the target geofence;

[0206] The second processing module is used to perform the following steps when the feedback signal indicates dissatisfaction with the current image retrieval: collect historical data of the current dialogue; infer the historical data using a specified model to determine the current retrieval blind spot; add external data interfaces corresponding to the current retrieval blind spot; construct a training dataset based on the historical data; and fine-tune the target large model using the training dataset.

[0207] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 8 As shown, the electronic device includes: a memory 80 for storing computer programs;

[0208] The processor 81 is used to execute a computer program to implement the steps of the remote sensing image retrieval method as described in the above embodiments.

[0209] The electronic devices provided in this embodiment may include, but are not limited to, laptops or desktop computers.

[0210] The processor 81 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 81 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 81 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 81 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 81 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0211] The memory 80 may include one or more computer-readable storage media, which may be non-transitory. The memory 80 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 80 is used to store at least the following computer program 801, which, after being loaded and executed by the processor 81, is capable of implementing the relevant steps of the remote sensing image retrieval method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 80 may also include an operating system 802 and data 803, and the storage method may be temporary or permanent storage. The operating system 802 may include Windows, Unix, Linux, etc. The data 803 may include, but is not limited to, the relevant data involved in the remote sensing image retrieval method.

[0212] In some embodiments, the electronic device may further include a display screen 82, an input / output interface 83, a communication interface 84, a power supply 85, and a communication bus 86.

[0213] Those skilled in the art will understand that Figure 8 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0214] The electronic device provided in this application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the remote sensing image retrieval method described in the above embodiments.

[0215] It should be noted that although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Claims

1. A method for retrieving remote sensing images, characterized in that, The method includes: Retrieve a remote sensing image retrieval request; The search request is parsed using a target large model to obtain initial search conditions; Based on the initial search criteria and the pre-built database of points of interest, the target geofence for the current image search is determined; Based on the initial search criteria and the target geofence, a search is performed in the remote sensing image database to obtain the target remote sensing image. The initial search criteria include geographic attribute criteria. Determining the target geofence for the current image search based on the initial search criteria and a pre-built point-of-interest database includes: When the geographic attribute conditions include explicit conditions, a search is performed in the point of interest database based on the geographic attribute conditions to obtain the geographic location information of the current image retrieval; When the geographic attribute conditions include implicit conditions, the first inference condition is obtained by reasoning the geographic attribute conditions through the target large model; and based on the first inference condition, a search is performed in the point of interest database to obtain the geographic location information. Generate the target geofence based on the geographic location information; The geographic location information includes the target POI, the metadata information corresponding to the target POI, and the target POI type. Generating the target geofence based on the geographic location information includes: Determine the geographic vector boundary based on the target POI and the metadata information; Based on the target POI type, the geographic vector boundary is adjusted to obtain the initial geofence; Determine whether the initial search criteria include non-geographic attribute criteria; If so, the initial geofence is trimmed according to the non-geographical attribute conditions to obtain the target geofence; If not, the initial geofence shall be used as the target geofence.

2. The remote sensing image retrieval method as described in claim 1, characterized in that, The step of adjusting the geographic vector boundary according to the target POI type to obtain the initial geofence includes: When the target POI is a point type, the geographic vector boundary is expanded by a specified radius with the center of the target POI as the center to obtain the initial geofence; When the target POI type is linear, the geographic vector boundary is extended by a specified distance along two directions perpendicular to the linear POI to obtain the initial geofence; When the target POI is of the isal type, the geographic vector boundary is used as the initial geofence.

3. The remote sensing image retrieval method as described in claim 1, characterized in that, The step of trimming the initial geofence based on the non-geographic attribute conditions includes: When the non-geographic attribute conditions include explicit conditions, the first constraint condition corresponding to the non-geographic attribute conditions is obtained through an external data interface; and the initial geofence is trimmed based on the first constraint condition. When the non-geographical attribute conditions include implicit conditions, the second inference condition is obtained by reasoning about the non-geographical attribute conditions through the target large model; the second constraint condition corresponding to the second inference condition is obtained through the external data interface; and the initial geofence is trimmed based on the second constraint condition.

4. The remote sensing image retrieval method as described in claim 1, characterized in that, The steps for constructing the point of interest database include: Obtain POI data; Extract core geographic fields from the POI data; and use the specified association information of the core geographic fields in the POI data as metadata information; Assign corresponding weights to the different types of metadata information; Based on the weights and metadata information, the core geographic fields are vectorized to obtain POI vectors; A vector index is constructed for the POI vectors to obtain the point of interest database.

5. The remote sensing image retrieval method as described in claim 1, characterized in that, The method includes: Obtain the user's input feedback signal regarding the current image retrieval; If the feedback signal is a signal to adjust the target geofence, the point of interest database is adjusted according to the feedback signal; If the feedback signal indicates dissatisfaction with the current image retrieval, the following steps are performed: Collect historical data of the current conversation; By using a specified model, reasoning is performed on the historical data to determine the current retrieval blind spot; an external data interface corresponding to the current retrieval blind spot is added; Based on the historical data, a training dataset is constructed; and the target large model is fine-tuned using the training dataset.

6. A remote sensing image retrieval device, characterized in that, The device includes: The retrieval request acquisition module is used to acquire retrieval requests for remote sensing images; The retrieval request parsing module is used to parse the retrieval request through the target large model to obtain initial retrieval conditions; the initial retrieval conditions include geographic attribute conditions. The target geofence determination module is used to determine the target geofence for the current image retrieval based on the initial retrieval conditions and a pre-built point of interest database. The target remote sensing image retrieval module is used to retrieve the target remote sensing image from the remote sensing image database based on the initial retrieval conditions and the target geofence. The geographic location information determination module is used to, when the geographic attribute conditions include explicit conditions, search the point of interest database based on the geographic attribute conditions to obtain the geographic location information of the current image retrieval; when the geographic attribute conditions include implicit conditions, infer the geographic attribute conditions through the target large model to obtain a first inference condition; and search the point of interest database based on the first inference condition to obtain the geographic location information; the geographic location information includes the target POI, the metadata information corresponding to the target POI, and the target POI type; The fence generation module is used to generate the target geofence based on the geographic location information; The initial geofence determination module is used to determine the geographic vector boundary based on the target POI and the metadata information; and adjust the geographic vector boundary according to the target POI type to obtain the initial geofence. The search condition judgment module is used to determine whether the initial search conditions include non-geographic attribute conditions; if so, the initial geofence is cropped according to the non-geographic attribute conditions to obtain the target geofence; if not, the initial geofence is used as the target geofence.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the remote sensing image retrieval method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the remote sensing image retrieval method according to any one of claims 1 to 5.

Images