Retrieval method and device of remote sensing image, electronic equipment and medium

Abstract

The invention discloses a remote sensing image retrieval method and device, electronic equipment and a medium. The method comprises the steps that a remote sensing image retrieval request is acquired; analyzing the retrieval request through the target large model to obtain an initial retrieval condition; according to the initial retrieval condition and a pre-constructed interest point database, determining a target geo-fence retrieved by the current image; and based on the initial retrieval condition and the target geo-fence, performing retrieval in a remote sensing image database to obtain a target remote sensing image. Therefore, retrieval request analysis of complex geographic description and fuzzy text description is realized through the target large model, so that the remote sensing image retrieval intention of a user is accurately understood, and meanwhile, an accurate target geographic fence is obtained in combination with the pre-constructed interest point database, remote sensing retrieval is locked in a more accurate geographic position range, and the remote sensing retrieval accuracy is improved. Therefore, the retrieval precision of the target remote sensing image is improved.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing data processing, and in particular to a remote sensing image retrieval method, device, electronic equipment and medium. Background Art

[0002] Remote sensing image retrieval is a core component of intelligent geographic information processing, with broad and important applications in areas such as agricultural monitoring, disaster assessment, and urban planning. With the rapid development of satellite sensor technology, remote sensing data is becoming increasingly diverse, multimodal (e.g., optical, SAR, hyperspectral, and time-series imagery), and massive, placing higher demands on remote sensing image retrieval.

[0003] Currently, remote sensing image retrieval is primarily based on keywords, location coordinates, and low-level visual features (such as color and texture). This approach makes it difficult to parse complex user search requests, resulting in low retrieval accuracy. Furthermore, when user search requests are ambiguous, it is often difficult to understand the user's search intent, resulting in image retrieval accuracy failing to meet user expectations.

[0004] Therefore, how to improve the retrieval accuracy of remote sensing images and meet users' expectations of high-precision retrieval is an urgent problem to be solved by technical personnel in this field. Summary of the Invention

[0005] In view of this, one aspect of the present application provides a remote sensing image retrieval method, the method comprising: Obtain retrieval requests for remote sensing images; Parsing the search request through the target macro model to obtain initial search conditions; Determining a target geo-fence for current image retrieval based on the initial retrieval conditions and a pre-built database of points of interest; Based on the initial search condition and the target geographic fence, a search is performed in a remote sensing image database to obtain a target remote sensing image.

[0006] Optionally, the initial search condition includes a geographic attribute condition, and determining a target geographic fence for the current image search based on the initial search condition and a pre-built database of points of interest includes: When the geographic attribute condition includes an explicit condition, searching the point of interest database based on the geographic attribute condition to obtain geographic location information of the current image retrieval; When the geographic attribute condition includes an implicit condition, the geographic attribute condition is inferred by the target macro model to obtain a first inference condition; and based on the first inference condition, a search is performed in the POI database to obtain the geographic location information; The target geo-fence is generated according to the geographic location information.

[0007] Optionally, the geographic location information includes a target POI, metadata information corresponding to the target POI, and a target POI type, and generating the target geo-fence according to the geographic location information includes: Determining a geographic vector boundary based on the target POI and the metadata information; Adjusting the geographic vector boundary according to the target POI type to obtain an initial geographic fence; determining whether the initial search condition includes a non-geographic attribute condition; If so, the initial geo-fence is clipped according to the non-geographic attribute condition to obtain the target geo-fence; If not, the initial geofence is used as the target geofence.

[0008] Optionally, adjusting the geographic vector boundary according to the target POI type to obtain an initial geographic fence includes: When the target POI type is a point type, the geographic vector boundary is extended by a specified radius with the center of the target POI as the center of the circle to obtain the initial geographic fence; When the target POI type is a linear type, the geographic vector boundary is extended by a specified distance along two directions perpendicular to the linear POI to obtain the initial geographic fence; When the target POI type is a planar type, the geographic vector boundary is used as the initial geographic fence.

[0009] Optionally, the clipping of the initial geo-fence according to the non-geographic attribute condition includes: When the non-geographic attribute condition includes an explicit condition, obtaining a first constraint condition corresponding to the non-geographic attribute condition through an external data interface; and clipping the initial geo-fence based on the first constraint condition; When the non-geographic attribute condition includes an implicit condition, the non-geographic attribute condition is inferred through the target large model to obtain a second inference condition; a second constraint condition corresponding to the second inference condition is obtained through the external data interface; and based on the second constraint condition, the initial geographic fence is clipped.

[0010] Optionally, the step of constructing the point of interest database includes: Get POI data; Extracting core geographic fields from the POI data; and using designated associated information of the core geographic fields in the POI data as metadata information; Assigning corresponding weights to different types of metadata information; Based on the weight and the metadata information, vectorizing the core geographic field to obtain a POI vector; A vector index is constructed for the POI vector to obtain the point of interest database.

[0011] Optionally, the remote sensing image retrieval method includes: Obtaining a feedback signal input by a user regarding the current image retrieval; If the feedback signal is a signal for adjusting the target geo-fence, adjusting the POI database according to the feedback signal; If the feedback signal indicates that the current image retrieval is not satisfactory, the following steps are performed: Collect historical data of the current conversation; By specifying a model, reasoning on the historical data to determine the current retrieval blind spot; adding an external data interface corresponding to the current retrieval blind spot; Based on the historical data, a training data set is constructed; and the target large model is fine-tuned and trained using the training data set.

[0012] Another aspect of the present application provides a remote sensing image retrieval device, the device comprising: A retrieval request acquisition module is used to obtain retrieval requests for remote sensing images; A search request parsing module, configured to parse the search request using a target macro model to obtain initial search conditions; a target geo-fence determination module, configured to determine a target geo-fence for a current image search based on the initial search conditions and a pre-built database of points of interest; The target remote sensing image retrieval module is used to search the remote sensing image database based on the initial retrieval conditions and the target geographic fence to obtain the target remote sensing image.

[0013] Another aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the remote sensing image retrieval method are implemented.

[0014] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the remote sensing image retrieval method when the program is executed by a processor.

[0015] The present application provides a remote sensing image retrieval method, device, electronic device and medium, which have the following beneficial effects: parsing of retrieval requests with complex geographic descriptions and fuzzy text descriptions is achieved through a large target model, thereby accurately understanding the user's remote sensing image retrieval intention, and at the same time combining with a pre-built point of interest database to obtain a precise target geographic fence, locking the remote sensing retrieval within a more precise geographic location range, thereby improving the retrieval accuracy of the target remote sensing image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a remote sensing image retrieval method provided in an embodiment of the present application; Figure 2 A schematic diagram of the principle of a remote sensing image retrieval method provided in an embodiment of the present application; Figure 3 A schematic diagram of the principle of determining a target geographic fence provided in an embodiment of the present application; Figure 4 A schematic diagram of the principle of adjusting geographic vector boundaries provided in an embodiment of the present application; Figure 5 A schematic diagram of the principle of constructing a point of interest database provided in an embodiment of the present application; Figure 6 A schematic diagram of the principle of determining the current search blind area provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of a remote sensing image retrieval device provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0017] The accompanying drawings are marked as follows: 70 is a retrieval request acquisition module, 71 is a retrieval request parsing module, 72 is a target geographic fence determination module, 73 is a target remote sensing image retrieval module, 80 is a memory, 81 is a processor, 82 is a display screen, 83 is an input and output interface, 84 is a communication interface, 85 is a power supply, 86 is a communication bus, 801 is a computer program, 802 is an operating system, and 803 is data. DETAILED DESCRIPTION

[0018] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0020] Figure 1 A flowchart of a remote sensing image retrieval method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes: S10: Obtaining a retrieval request for remote sensing images; S11: parsing the search request through the target large model to obtain initial search conditions; Figure 2 A schematic diagram of the principle of a remote sensing image retrieval method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, in a specific embodiment, the user inputs a retrieval request for remote sensing images through a terminal. The retrieval request can be in the form of text, image, voice, etc. Of course, it can also be a multimodal retrieval request, which is not limited in this application.

[0021] Furthermore, to ensure accurate understanding of the search request, in an optional embodiment, the search request can be parsed using a target macro model to generate structured initial search conditions, i.e., search constraints. The target macro model may include, but is not limited to, the GPT series, Tongyi Qianwen, and DeepSeek, and this application does not limit this.

[0022] In an optional embodiment, the target large model can be a pre-trained model in the remote sensing field. Specifically, an initial large model is pre-built, and a large amount of remote sensing training data sets are collected. The initial large model is then trained to obtain the target large model. This target large model can thus more accurately interpret remote sensing expertise and improve the accuracy of remote sensing image retrieval.

[0023] In a specific embodiment, when parsing a search request using the target macro model, a prompt word project is constructed. Based on the prompt word project, the search request is input into the target macro model for inference, thereby obtaining initial search conditions. The initial search conditions are used to narrow down and target the target remote sensing image. To facilitate understanding, an example is provided below.

[0024] For example, the search request is to download the cloud-free Gaofen-2 image of the Yangtze River Delta in June 2023. The target large model is ChatGLM-6B. The prompt word project is: "You are a remote sensing image retrieval expert. Please parse the following user's retrieval request into structured initial retrieval conditions: User input search request: {input text} Output JSON format: {time, geographic range, cloud cover, sensor type...}" Output structured initial search conditions: { "time": {"start": "2023-06-01", "end": "2023-06-30"}, "geo": {"type": "region", "name": "Yangtze River Delta"}, "cloud_cover": {"max": 5}, "sensor": "Gaofen-2" } In an optional embodiment, the initial search conditions are checked through the target large model. When there is a conflict in the initial search conditions, the initial conditions can be corrected through the target large model, or the conflict result can be returned to the terminal, and the user can adjust the search request and re-enter it, thereby improving the retrieval accuracy of subsequent remote sensing images.

[0025] For example, if the initial search condition is "search for images of plateau landforms in the Shanghai area," there is a clear logical conflict. In this case, the target large model can be used to correct the condition. For example, "search for images of plateau landforms in the Shanghai area" can be corrected to "search for images of mountains in the Shanghai area." Of course, the conflict can also be returned to the terminal for the user to actively modify.

[0026] S12: determining the target geo-fence for the current image retrieval based on the initial retrieval conditions and the pre-built POI database; S13: Based on the initial search conditions and the target geographic fence, a search is performed in the remote sensing image database to obtain the target remote sensing image.

[0027] Further, such as Figure 2 As shown, to further improve the retrieval accuracy of remote sensing images, in an optional embodiment, a point of interest database is introduced. This point of interest database is used to store information related to geographic locations. This application does not limit the storage format of the point of interest database, which may include but is not limited to vectorized storage and table storage. In a specific embodiment, the initial search conditions are combined with the point of interest database to determine the target geofence for the current image retrieval.

[0028] Specifically, searching the POI database based on the initial search criteria can narrow the scope of the current image search criteria, that is, determine the geographic scope of the current image search. Thus, searching the remote sensing image database based on the initial search criteria and the target geographic fence can accurately obtain the target remote sensing image required by the user.

[0029] In an optional embodiment, after acquiring the target remote sensing image, historical search data is recorded. Specifically, this data includes the search request entered during the search session, the initial search criteria parsed from the target macro model, the hash value of the search result, and the target remote sensing image. It will be appreciated that acquiring historical search data allows for evaluation of search quality and adjustment and optimization of the data stored in the POI database, thereby improving search accuracy over the course of the search process.

[0030] Therefore, the remote sensing image retrieval method provided in the embodiment of the present application realizes the parsing of retrieval requests for complex geographical descriptions and fuzzy text descriptions through the target large model, thereby accurately understanding the user's remote sensing image retrieval intention, and at the same time combining with the pre-built point of interest database to obtain accurate target geographic fences, locking the remote sensing retrieval within a more accurate geographical location range, thereby improving the retrieval accuracy of the target remote sensing image.

[0031] In an optional embodiment, the initial search condition includes a geographic attribute condition. Determining a target geographic fence for the current image search based on the initial search condition and a pre-built database of points of interest includes: When the geographic attribute condition includes an explicit condition, a search is performed in the point of interest database based on the geographic attribute condition to obtain the geographic location information of the current image retrieval; When the geographic attribute condition includes an implicit condition, the geographic attribute condition is inferred by the target large model to obtain a first inference condition; and based on the first inference condition, a search is performed in the point of interest database to obtain geographic location information; Generate target geo-fence based on geographic location information.

[0032] Figure 3 This is a schematic diagram of a principle for determining a target geographic fence provided by an embodiment of the present application. In a specific embodiment, as shown in FIG. Figure 3 As shown, the initial search condition includes a geographical attribute condition, which refers to a condition containing geographical related information. However, the geographical attribute condition includes at least one of an explicit condition and an implicit condition.

[0033] Explicit conditions refer to information that can be directly extracted and converted into search criteria. Explicit conditions include, but are not limited to, specific time ranges, administrative divisions, remote sensing satellite types, remote sensing image types, and remote sensing image imaging modes. Implicit conditions refer to information that cannot be directly extracted but still serves as key search criteria. These include, but are not limited to, ambiguous time ranges (for example, seasons combined with geographic location), place names (for example, Shanghai and the Pearl River Delta), and descriptions containing geographical types (for example, mountainous areas and tidal flats).

[0034] In an optional embodiment, when the geographic attribute condition includes an explicit condition, the initial search condition representing the current parsing and extraction can be directly used to search the POI database, thereby obtaining the geographic location information for the current image search. For example, if the geographic attribute condition is "search for remote sensing images of Beijing," a search can be performed in the POI database based on this geographic attribute condition to obtain geographic location information about Beijing, specifically including but not limited to the latitude and longitude of Beijing.

[0035] In another optional embodiment, when the geographic attribute condition includes an implicit condition, it indicates that the current initial search condition cannot be directly searched in the POI database, that is, the current geographic attribute condition is a fuzzy condition. Figure 3 As shown, the target large model can be used to infer the implicit geographical attribute conditions to obtain the first inference condition. Further, based on the first inference condition, a search is performed in the POI database to obtain the geographical location information. For ease of understanding, the following example will be used.

[0036] For example, if the geographic attribute condition is "Search for remote sensing images of Shanghai," Shanghai is clearly a geographic attribute condition, but this condition is implicit and cannot be directly used as a search condition in the POI database. Therefore, we can use the target large model to perform inference, and the first inference condition we obtain is Shanghai, that is, Shanghai is Shanghai. Thus, "Search for remote sensing images of Shanghai" can be transformed into "Search for remote sensing images of Shanghai." At this point, Shanghai's geographic location information can be searched in the POI database.

[0037] In an optional embodiment, when reasoning through the target big model, typical steps of preset retrieval conditions (such as time, administrative division, place name, geographical feature, satellite type, image type, etc.) can be constructed, and LLM can be used to read and analyze user historical retrieval records to perform dynamic retrieval condition reasoning.

[0038] In an optional embodiment, the geographic location information includes a target POI (Point of Interest), metadata information corresponding to the target POI, and a target POI type. Generating a target geo-fence based on the geographic location information includes: Determine geographic vector boundaries based on target POI and metadata information; According to the target POI type, the geographic vector boundary is adjusted to obtain the initial geographic fence; Determine whether the initial search criteria include non-geographic attribute criteria; If yes, the initial geofence is clipped based on the non-geographic attribute conditions to obtain the target geofence; If not, the initial geofence is used as the target geofence.

[0039] In a specific embodiment, it is understood that based on the target POI and the metadata information corresponding to the target POI, a geographic vector boundary can be determined, that is, the retrieval boundary area of ​​the current remote sensing image can be determined. The metadata information refers to the relevant geographic information of the target POI, that is, a more detailed description field of the geographic information. For example, when the target POI is Beijing and the metadata information is the latitude and longitude of Beijing, the geographic vector boundary of Beijing can be locked based on the target POI and the metadata information.

[0040] It should be noted that, in a specific embodiment, multiple target POIs may appear. In this case, the multiple target POIs may be returned to the terminal, and the user confirms the POI for generating the target geo-fence.

[0041] like Figure 3 As shown, in a specific embodiment, in addition to geographic attribute conditions, the initial search conditions may also include non-geographic attribute conditions. It is understood that although non-geographic attribute conditions do not contain geographic location information, they can still affect the accuracy of generating the target geofence. To facilitate understanding, the following example will illustrate this.

[0042] For example, when the non-geographic attribute condition is time, specifically, the initial search condition is "retrieve remote sensing images of Beijing in June 2023". Obviously, the geographical attribute condition is Beijing, and the non-geographic attribute condition is June 2023. Obviously, the time limit has an important impact on the search accuracy.

[0043] Therefore, in an optional embodiment, to further improve the retrieval accuracy of remote sensing imagery, before generating the target geofence, it is first determined whether the initial search criteria include non-geographic attribute conditions. If so, the geovector boundary is first adjusted based on the target POI type to obtain the initial geofence. Specifically, the adjustment of the geovector boundary can include expansion or contraction of the boundary, which is not limited in this application. Furthermore, in specific embodiments, the adjustment can be performed using a large model or manually, which is also not limited in this application.

[0044] Furthermore, the initial geofence is clipped based on non-geographic attribute conditions to obtain a target geofence. Specifically, the initial geofence is further refined based on the non-geographic attribute conditions, i.e., narrowed to a more accurate geographic range, to obtain the target geofence. For ease of understanding, the following example illustrates this.

[0045] For example, if the geographic attribute condition is "Yangtze River Delta" and the non-geographic attribute condition is "Dates with cloud cover ≤ 5% in June 2023," a search is performed in the POI database based on the Yangtze River Delta, resulting in a geovector boundary of 118°-123° East longitude and 29°-33° North latitude. The geovector boundary is then adjusted based on the target POI type to obtain an initial geofence. This initial geofence is then clipped to the date with cloud cover ≤ 5% in June 2023, resulting in the accurate target geofence.

[0046] Figure 4 This is a schematic diagram of the principle of adjusting the geographic vector boundary provided in an embodiment of the present application. Based on the above embodiment, as an optional embodiment, the geographic vector boundary is adjusted according to the target POI type to obtain an initial geographic fence, including: When the target POI type is point type, the geographic vector boundary is expanded with the center of the target POI as the center of the circle by a specified radius to obtain the initial geographic fence; When the target POI type is linear, the geographic vector boundary is extended by a specified distance in two directions perpendicular to the linear POI to obtain the initial geographic fence; When the target POI type is a polygon, the geographic vector boundary is used as the initial geofence.

[0047] It is understandable that the accuracy of the target geofence is very important to the accuracy of remote sensing image retrieval. Specifically, if the target geofence is smaller than the actual retrieval geographic location area, that is, the target geofence does not completely cover the actual retrieval geographic area, it may cause retrieval failure.

[0048] Therefore, in order to solve the above technical problems, based on the above embodiment, after initially obtaining the geographic vector boundary, the geographic vector boundary can be adjusted according to the target POI type. Figure 4 As shown, when the target POI type is a point type, for example, the geometry type of Beijing Railway Station on the POI is a point, then it can be appropriately expanded outward with the target POI as the center so that the initial geographic fence is larger than the geographic vector boundary.

[0049] In an optional embodiment, the geographic vector boundary can be extended with a specified radius with the center of the target POI as the center of the circle. The specified radius can be selected by the user, for example, it can be set to 5 kilometers, which is not limited by this application.

[0050] like Figure 4 As shown, when the target POI type is linear, it may represent a river, subway line, etc., and the geometry type in the POI is line. For example, the geometry type in the POI of Beijing Subway Line 1 is line. To improve image detection accuracy, in an optional embodiment, the geographic vector boundary can be extended by a specified distance in two directions perpendicular to the linear POI. In fact, linear extension can be understood as thickening the original linear POI.

[0051] For a planar target POI type, such as the Olympic Forest Park, it is understandable that for a POI with a certain range, the obtained geographic vector boundary is often an accurate geographic area range. Therefore, Figure 4 As shown, for a planar target POI, the geographic vector boundary can be directly used as the initial geo-fence, that is, the target POI is not modified. Of course, in an optional embodiment, for a planar target POI type, the circle can be expanded with a specified radius with the target POI center point as the center.

[0052] Therefore, the remote sensing image retrieval method provided in the embodiment of the present application, after initially obtaining the geographic vector boundary, adjusts the geographic vector boundary according to the target POI type, which can ensure that the remote sensing image that the user wants to retrieve must fall within the target geographic fence, thereby improving the image detection accuracy.

[0053] Based on the above embodiment, as an optional embodiment, the initial geo-fence is trimmed according to non-geographic attribute conditions, including: When the non-geographic attribute condition includes an explicit condition, obtaining a first constraint condition corresponding to the non-geographic attribute condition through an external data interface; and clipping the initial geo-fence based on the first constraint condition; When the non-geographic attribute conditions include implicit conditions, the non-geographic attribute conditions are inferred through the target large model to obtain a second inference condition; the second constraint condition corresponding to the second inference condition is obtained through the external data interface; and the initial geographic fence is clipped based on the second constraint condition.

[0054] like Figure 3 As shown, in a specific embodiment, when non-geographic attribute conditions exist, in order to further improve the accuracy of the target geo-fence, the initial geo-fence obtained in the above embodiment needs to be clipped according to the non-geographic attribute conditions.

[0055] It is understood that non-geographic attribute conditions may also include at least one of explicit and implicit conditions. In an optional embodiment, when the non-geographic attribute conditions include explicit conditions, external data can be directly connected via an external data interface to obtain a first constraint condition related to the non-geographic attribute condition, so that the initial geofence can be tailored based on the first constraint condition. The external data includes, but is not limited to, weather and traffic conditions, and the external data can perform spatial logical operations on the geofence.

[0056] In another optional embodiment, as Figure 3 As shown, when non-geographic attribute conditions include implicit constraints, it's impossible to directly retrieve the constraints from external data. In this case, the implicit constraints can be inferred using the target macro model to obtain secondary inference conditions. This involves converting the implicit constraints into explicit ones. Furthermore, based on the secondary inference conditions, the corresponding secondary constraints can be retrieved from the external data structure. The initial geofence can then be tailored based on the secondary constraints. To facilitate understanding, an example is provided below.

[0057] For example, a user enters a search request like "Find vegetation cover images for 2022 within a 10-kilometer radius of the northern slope of the Qinling Mountains." Parsing the target large model yields the keywords "northern slope of the Qinling Mountains," "10 kilometers," "2022," and "vegetation cover." "Qinling Mountains" is an explicit geographic attribute, so a search for "Qinling Mountains" in the POI database yields a geographic vector boundary. Simultaneously, DEM (Digital Elevation Model) data is used to calculate areas on the northern slope with slopes greater than 15°.

[0058] Furthermore, the north slope vector boundary is expanded by 10 kilometers, i.e., the geographic vector boundary is adjusted to obtain the initial geofence. It is understood that "vegetation cover" is an implicit non-geographic attribute condition. The target large model can be used to first infer the second inference condition of "vegetation cover with NDVI (Normalized Difference Vegetation Index) > 0.3." Based on this second inference condition, external data is accessed and the initial geofence is clipped to obtain the target geofence.

[0059] It's important to note that when geofences are tailored based on external data, there may be conflicts. In this case, the target model can automatically generate recommendations, or the user can select the final inference criteria. Furthermore, after the user selects a decision, the current selection is recorded, and the priority of the inference criteria in subsequent target models is updated. For easier understanding, an example is provided below.

[0060] For example, a user enters a search request such as "Get optical images of Beijing with a resolution of 2 meters on January 1, 2023." After being parsed by the target large model, the structured initial search criteria are obtained, including "Time = 2023-01-01, Geographical Range = Beijing, Resolution = 2 meters, Sensor Type = Optical." After external data is collected, it is determined that there is no 2-meter resolution satellite transit data for Beijing on January 1, 2023 (for example, only 5-meter resolution data is available).

[0061] At this point, two inference conditions can be generated: Option 1 recommends January 2, 2023 (2-meter resolution, 10% cloud cover), and Option 2 recommends January 1, 2023 (5-meter resolution, 0% cloud cover). The user can then select the final inference condition. This selection is recorded and the analysis weight of the target large model is updated. For example, increasing the weight of resolution will increase its priority in subsequent resolutions.

[0062] Finally, the explicit geographic attribute condition, the first reasoning condition, the explicit geographic attribute condition, the second reasoning condition and the target geographic fence are combined to search in the remote sensing image database to obtain the target remote sensing image.

[0063] Figure 5 This is a schematic diagram of the principle of constructing a point of interest database provided in an embodiment of the present application. In an optional embodiment, the steps of constructing the point of interest database provided in the present application include: Get POI data; Extract core geographic fields from POI data; and use the designated associated information of the core geographic fields in the POI data as metadata information; Assign corresponding weights to different types of metadata information; Based on the weight and metadata information, the core geographic fields are vectorized to obtain POI vectors; Build a vector index for the POI vector and obtain the point of interest database.

[0064] In a specific embodiment, Figure 5 As shown, building a POI database begins with data collection. Specifically, POI data is obtained from open-source databases, including but not limited to Amap and Baidu Maps. POI data includes but is not limited to attribute fields such as the POI's name, nickname, function, and multi-level administrative divisions of provinces, cities, and districts, as well as administrative districts and longitude and latitude. For ease of understanding, the following examples will illustrate this.

[0065] For example, the extracted POI data is: { "name": "Peking University", "alias": ["Peking University", "Beida"], "category": "Education", "latitude": 39.9996, "longitude": 116.3164, "admin_code": "110108" / / Administrative code of Haidian District; } As Figure 5 shown, after obtaining the initial POI data, the data is cleaned to ensure the data quality of the POI database. Further, the core geographical fields are extracted from the cleaned POI data, and at the same time, the specified associated information of the core geographical fields is used as metadata information.

[0066] The core geographical fields include, but are not limited to, attribute fields such as the name, alias category, function, and multi-level administrative divisions of provinces, municipalities, and districts of the POI, while the metadata information includes, but is not limited to, administrative regions and longitude and latitude. In a specific embodiment, the metadata information can be understood as detailed geographical description information of the core geographical fields. It should be noted that the core geographical fields not only include the POI name. In fact, in order to improve the POI retrieval accuracy, the core geographical fields integrate multiple POI geographical information.

[0067] In an optional embodiment, in order to achieve fast retrieval of target POIs, the POI database can be a vectorized database. Therefore, in a specific embodiment, it is necessary to vectorize the core geographical fields to obtain POI vectors.

[0068] Specifically, in an optional embodiment, in order to meet the retrieval requirements of different actual target POIs, different weights are assigned to different types of metadata information. That is, according to the functional requirements of the retrieval platform, different weight levels are assigned to increase the attention degree for different types of metadata information through different weight levels.

[0069] Further, the core geographical fields are vectorized through a specified model, where the specified model can be the Sentence-BERT model, and this application does not make any limitations in this regard. In an optional embodiment, when vectorizing, geographical term corpora (such as the Chinese Geographical Terms Dictionary) can be injected to generate high-precision POI vectors.

[0070] It should be noted that in a specific embodiment, the metadata information can be added and deleted according to actual business needs, and this application does not make any limitations on the type and quantity of the metadata information. In addition, this application does not make any specific limitations on the specific content included in the vectorized core geographical fields.

[0071] After obtaining the POI vector, further, such as Figure 5 As shown, a vector index is constructed for the POI vector. Specifically, in an optional embodiment, the HNSW algorithm can be used to establish the vector index, and a tree-like hierarchical index can be constructed according to the administrative division code (province-city-district-county), thereby obtaining a POI database. This application does not limit the method of constructing the index.

[0072] In an optional embodiment, the remote sensing image retrieval method provided in this application includes: Obtaining feedback signals input by the user regarding the current image retrieval; If the feedback signal is a signal for adjusting the target geo-fence, the POI database is adjusted according to the feedback signal; If the feedback signal indicates that the current image retrieval is not satisfactory, perform the following steps: Collect historical data of the current conversation; By specifying a model, reasoning about historical data can be used to determine the current retrieval blind spot; external data interfaces corresponding to the current retrieval blind spot can be added; Based on historical data, a training data set is constructed; and the target large model is fine-tuned through the training data set.

[0073] In order to further improve the accuracy of parsing user intentions, the remote sensing image detection method provided in this application provides an index dynamic optimization strategy based on user feedback, which can dynamically optimize the parsing weights (i.e., parsing focus) of point of interest data and target large models based on the user's feedback signals on the retrieval results.

[0074] Specifically, in an optional embodiment, if a user proactively adjusts a target geofence, the adjustment can be associated with the corresponding POI vector in the POI database, thereby increasing the vector's ranking priority in subsequent searches. Alternatively, based on feedback, the weights of different metadata within the POI database can be adjusted, thereby increasing the priority of metadata types that the user is most interested in in subsequent searches.

[0075] In another optional embodiment, when the feedback signal indicates user dissatisfaction with the current image search, reinforcement learning fine-tuning of the target large model is triggered, thereby reducing the confidence weight of similar conditions in the large model analysis. Dissatisfaction with the search can include repeated questions, interrupting the conversation, or modifying specified conditions during the conversation. Specified conditions include, but are not limited to, incorrect search time requirements, incorrect search POI names, excessive deviation in the generated POI geofence, and incorrect image type.

[0076] Specifically, when fine-tuning the target large model, all historical data for the current conversation is first collected and a training dataset is constructed based on this historical data. This allows the target large model to be fine-tuned using the training dataset. In a specific embodiment, the training dataset can be generated based on a pre-built dataset construction architecture. For ease of understanding, an example is provided below.

[0077] For example, users can modify and supplement the search condition analysis content output by the engine. The collected historical data is: Conversation content: User: Retrieve all optical images covering Hangzhou city in 2024.

[0078] System: Please list the types of satellites that are relevant for imaging.

[0079] User: Gaofen No. 2.

[0080] System: Please list the imaging types of the images.

[0081] User: No input (i.e., abandoned the conversation).

[0082] Based on the above historical data, it can be seen that the user's request was only to retrieve all optical images. When responding, the system asked excessive questions about two inaccurate search conditions, causing the user to give up the conversation of further search.

[0083] At this point, construct a training dataset based on historical data: { "instruction": "Analyze and extract the image requirements for the user input. If the user does not specify the satellite type and imaging type, the user is not required to provide relevant information and the default is to search all images." "input": "Search for all optical images covering Hangzhou in 2024", "output": "Intent decomposition results, time: 2024-01-01, 2024-12-32, region: Hangzhou, object: optical image" } It should be noted that to conserve computing resources, reinforcement learning training of the target large model can be triggered when the training dataset reaches the specified storage space. During training, in an optional embodiment, the target large model is fine-tuned using QLoRA (Quantized Low-Rank Adaptation), which is not limited in this application.

[0084] In another optional embodiment, in addition to fine-tuning the target large model, in order to further improve the accuracy of subsequent image retrieval, a new external data interface can be introduced by determining the current retrieval blind area.

[0085] Specifically, by specifying a model and reasoning about historical data, the current search blind spot can be determined, allowing the addition of an external data interface corresponding to the current search blind spot. The specified model can be an LSTM (Long Short-Term Memory) network, which is not limited in this application.

[0086] Figure 6 The schematic diagram of the principle of determining the current search blind spot provided by the embodiment of this application is that when LSTM performs specific reasoning and analysis on historical data, it is necessary to first splice and textualize the historical data. Specifically, Figure 6 As shown, historical data is first preprocessed, followed by feature screening to identify user-rejected paragraphs and mark them. Furthermore, a conversation dataset is constructed. This conversation dataset can also be constructed based on a pre-built architecture, which is not limited in this application. The conversation dataset is then input into an LSTM for analysis to determine the current retrieval blind spot.

[0087] For example, the current target large model cannot parse content related to natural disasters. At this time, by processing and analyzing the current historical data, it can be determined that the current retrieval blind spot is natural disasters. At this time, a new external data interface for natural disasters can be added to provide data support for subsequent retrieval.

[0088] In the above embodiments, the remote sensing image retrieval method is described in detail. The present application also provides a corresponding embodiment of a remote sensing image retrieval device.

[0089] Figure 7 A schematic diagram of a remote sensing image retrieval device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the device includes: A retrieval request acquisition module 70 is used to obtain a retrieval request for remote sensing images; A search request parsing module 71 is used to parse the search request through the target macro model to obtain initial search conditions; a target geofence determination module 72 for determining a target geofence for a current image search based on initial search conditions and a pre-built database of points of interest; The target remote sensing image retrieval module 73 is used to search the remote sensing image database based on the initial retrieval conditions and the target geographic fence to obtain the target remote sensing image.

[0090] In addition, the remote sensing image retrieval device provided in the embodiment of the present application further includes: A geographic location information determination module is configured to, when the geographic attribute condition includes an explicit condition, search the POI database based on the geographic attribute condition to obtain geographic location information for the current image retrieval; when the geographic attribute condition includes an implicit condition, infer the geographic attribute condition using the target macro model to obtain a first inference condition; and search the POI database based on the first inference condition to obtain geographic location information; The fence generation module is used to generate a target geographic fence based on geographic location information.

[0091] An initial geofence determination module is used to determine a geovector boundary based on the target POI and metadata information; and adjust the geovector boundary based on the target POI type to obtain an initial geofence; The search condition judgment module is used to determine whether the initial search condition includes non-geographic attribute conditions; if so, the initial geo-fence is clipped according to the non-geographic attribute conditions to obtain the target geo-fence; if not, the initial geo-fence is used as the target geo-fence.

[0092] The first expansion module is configured to expand the geographic vector boundary by a specified radius with the center of the target POI as the center of the circle to obtain an initial geographic fence when the target POI type is a point type; The second expansion module is used to expand the geographic vector boundary by a specified distance along two directions perpendicular to the linear POI to obtain an initial geographic fence when the target POI type is a linear type; The first processing module is configured to use a geographic vector boundary as an initial geographic fence when the target POI type is a planar type.

[0093] A first clipping module is configured to, when the non-geographic attribute condition includes an explicit condition, obtain a first constraint condition corresponding to the non-geographic attribute condition through an external data interface; and clip the initial geo-fence based on the first constraint condition; The second clipping module is used to, when the non-geographic attribute conditions include implicit conditions, infer the non-geographic attribute conditions through the target large model to obtain a second inference condition; obtain the second constraint condition corresponding to the second inference condition through the external data interface; and clip the initial geographic fence based on the second constraint condition.

[0094] POI data acquisition module, used to obtain POI data; The data extraction module is used to extract the core geographic fields from the POI data and use the designated associated information of the core geographic fields in the POI data as metadata information; A weight assignment module is used to assign corresponding weights to different types of metadata information; The vectorization module is used to vectorize the core geographic fields based on weights and metadata information to obtain POI vectors; The index building module is used to build a vector index for the POI vector to obtain a point of interest database.

[0095] A feedback signal acquisition module is used to obtain a feedback signal input by a user regarding the current image retrieval; an adjustment module, configured to adjust the POI database according to the feedback signal when the feedback signal is a signal for adjusting the target geo-fence; The second processing module is used to perform the following steps when the feedback signal indicates dissatisfaction with the current image retrieval: collecting historical data of the current conversation; inferring the historical data through a specified model to determine the current retrieval blind spot; adding an external data interface corresponding to the current retrieval blind spot; constructing a training data set based on the historical data; and fine-tuning the target large model through the training data set.

[0096] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, the electronic device includes: a memory 80 for storing computer programs; The processor 81 is configured to implement the steps of the remote sensing image retrieval method mentioned in the above embodiment when executing a computer program.

[0097] The electronic device provided in this embodiment may include but is not limited to a laptop computer or a desktop computer.

[0098] Among them, the processor 81 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 81 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 81 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 81 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 81 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0099] The memory 80 may include one or more computer-readable storage media, which may be non-transitory. The memory 80 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 80 is at least used to store the following computer program 801, wherein, after the computer program is loaded and executed by the processor 81, it can implement the relevant steps of the remote sensing image retrieval method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 80 may also include an operating system 802 and data 803, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 802 may include Windows, Unix, Linux, etc. The data 803 may include but is not limited to the relevant data involved in the remote sensing image retrieval method, etc.

[0100] In some embodiments, the electronic device may further include a display screen 82 , an input / output interface 83 , a communication interface 84 , a power supply 85 , and a communication bus 86 .

[0101] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure.

[0102] The electronic device provided in the embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the remote sensing image retrieval method in the above embodiment.

[0103] It should be noted that although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A remote sensing image retrieval method, characterized in that: The method comprises: Obtain retrieval requests for remote sensing images; Parsing the search request through the target macro model to obtain initial search conditions; Determining a target geo-fence for current image retrieval based on the initial retrieval conditions and a pre-built database of points of interest; Based on the initial search condition and the target geographic fence, a search is performed in a remote sensing image database to obtain a target remote sensing image.

2. The remote sensing image retrieval method according to claim 1, wherein: The initial search condition includes a geographic attribute condition, and determining a target geographic fence for current image retrieval based on the initial search condition and a pre-built database of points of interest includes: When the geographic attribute condition includes an explicit condition, searching the point of interest database based on the geographic attribute condition to obtain geographic location information of the current image retrieval; When the geographic attribute condition includes an implicit condition, the geographic attribute condition is inferred by the target macro model to obtain a first inference condition; and based on the first inference condition, a search is performed in the POI database to obtain the geographic location information; The target geo-fence is generated according to the geographic location information.

3. The remote sensing image retrieval method according to claim 2, wherein: The geographic location information includes a target POI, metadata information corresponding to the target POI, and a target POI type. Generating the target geo-fence according to the geographic location information includes: Determining a geographic vector boundary based on the target POI and the metadata information; Adjusting the geographic vector boundary according to the target POI type to obtain an initial geographic fence; determining whether the initial search condition includes a non-geographic attribute condition; If so, the initial geo-fence is clipped according to the non-geographic attribute condition to obtain the target geo-fence; If not, the initial geofence is used as the target geofence.

4. The remote sensing image retrieval method according to claim 3, wherein: The step of adjusting the geographic vector boundary according to the target POI type to obtain an initial geographic fence includes: When the target POI type is a point type, the geographic vector boundary is extended by a specified radius with the center of the target POI as the center of the circle to obtain the initial geographic fence; When the target POI type is a linear type, the geographic vector boundary is extended by a specified distance along two directions perpendicular to the linear POI to obtain the initial geographic fence; When the target POI type is a planar type, the geographic vector boundary is used as the initial geographic fence.

5. The remote sensing image retrieval method according to claim 3, wherein: The clipping of the initial geo-fence according to the non-geographic attribute condition includes: When the non-geographic attribute condition includes an explicit condition, obtaining a first constraint condition corresponding to the non-geographic attribute condition through an external data interface; and clipping the initial geo-fence based on the first constraint condition; When the non-geographic attribute condition includes an implicit condition, the non-geographic attribute condition is inferred through the target large model to obtain a second inference condition; a second constraint condition corresponding to the second inference condition is obtained through the external data interface; and based on the second constraint condition, the initial geographic fence is clipped.

6. The remote sensing image retrieval method according to claim 1, wherein: The steps of constructing the point of interest database include: Get POI data; Extracting core geographic fields from the POI data; and using designated associated information of the core geographic fields in the POI data as metadata information; Assigning corresponding weights to different types of metadata information; Based on the weight and the metadata information, vectorizing the core geographic field to obtain a POI vector; A vector index is constructed for the POI vector to obtain the point of interest database.

7. The remote sensing image retrieval method according to claim 1, wherein: The method comprises: Obtaining a feedback signal input by a user regarding the current image retrieval; If the feedback signal is a signal for adjusting the target geo-fence, adjusting the POI database according to the feedback signal; If the feedback signal indicates that the current image retrieval is not satisfactory, the following steps are performed: Collect historical data of the current conversation; By specifying a model, reasoning on the historical data to determine the current retrieval blind spot; adding an external data interface corresponding to the current retrieval blind spot; Based on the historical data, a training data set is constructed; and the target large model is fine-tuned and trained using the training data set.

8. A remote sensing image retrieval device, characterized in that: The device comprises: A retrieval request acquisition module is used to obtain retrieval requests for remote sensing images; A search request parsing module, configured to parse the search request using a target macro model to obtain initial search conditions; a target geo-fence determination module, configured to determine a target geo-fence for a current image search based on the initial search conditions and a pre-built database of points of interest; The target remote sensing image retrieval module is used to search the remote sensing image database based on the initial retrieval conditions and the target geographic fence to obtain the target remote sensing image.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the computer program, the steps of the remote sensing image retrieval method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the remote sensing image retrieval method according to any one of claims 1 to 7 are implemented.

Images