Satellite image data interaction method and device and medium

Abstract

The invention discloses an interaction method and device for satellite image data and a medium, and the method comprises the steps: carrying out the analysis of an obtained data processing instruction and an image interpretation text of the satellite image data, so as to determine a user request; and if the user request is a satellite image processing request, calling a target visual basic model for processing the satellite image processing request, and formulating a satellite image task strategy. And controlling the target visual basic model to execute a corresponding satellite image task based on the satellite image task strategy. Therefore, the data processing instruction and the image interpretation text are combined and analyzed to comprehensively know the user request, and the small model capable of realizing the downstream task is called according to the user request to carry out the satellite image task, namely, the visual basic model capable of realizing the satellite image processing is called to carry out the satellite image processing. Therefore, on the basis of accurate analysis of the satellite image data, rapid, accurate and efficient satellite image data interaction of a user is met.

Description

Technical Field

[0001] The present application relates to the field of satellite technology, and in particular to a method, device and medium for interacting with satellite image data. Background Art

[0002] With the continuous development of satellite technology, how to efficiently and accurately analyze satellite image data is an important part of meeting the different needs of users.

[0003] At present, satellite image data is mainly parsed through a single visual basic model, so that users can extract information from satellite image data. However, due to the diversity and heterogeneity of data sources, resolutions, data types, and data fusion, a large amount of satellite image data cannot be fully and accurately parsed by a single visual basic model, and cannot quickly and accurately respond to user needs, thereby reducing user experience.

[0004] It can be seen that how to efficiently analyze satellite image data, quickly and accurately respond to user needs, and improve user experience is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] In view of this, one aspect of the present application provides a satellite image data interaction method, the method comprising: Obtain data processing instructions and image interpretation text of satellite image data; Parsing the data processing instruction and the image interpretation text to determine a user request; If the user request is a satellite image processing request, a target visual basic model for processing the satellite image processing request is retrieved from a visual basic model library, and a satellite image task strategy is formulated; Based on the satellite image task strategy, the target visual basic model is controlled to execute the corresponding satellite image task to obtain an execution result; and the execution result is transmitted to the client. Optionally, a target visual base model for processing the satellite image processing request is retrieved, and a satellite image task strategy is formulated, including: Determining a task type of the satellite imaging task according to the user request; wherein different task types correspond to one satellite imaging task; When the task type includes multiple ones, respectively calling the target visual basic model corresponding to each task type; Plan the execution order of each of the target visual basic models to obtain the satellite image mission strategy.

[0006] Optionally, planning the execution order of each of the target visual base models includes: Determining the dependency relationship between the satellite image tasks according to the task type; Divide the satellite image tasks with the dependency relationship into a task subset; and plan the execution order of the satellite image tasks in each of the task subsets; The satellite image tasks with the same execution order in each of the task subsets are set to be executed in parallel.

[0007] Optionally, determining the user request according to the data processing instruction and the image interpretation text includes: Extracting first key information from the data processing instruction and second key information from the image interpretation text; analyzing the dependency between the first key information and the second key information; Based on the dependencies, the user request is determined.

[0008] Optionally, obtain image interpretation text for satellite image data, including: Receiving the satellite image data; Preprocessing the satellite image data to obtain target image data; wherein the preprocessing includes image size adjustment and image resolution adjustment; The target image data is interpreted by graphics and text through a graphics and text description model to obtain the image interpretation text.

[0009] Optionally, the satellite image data interaction method further includes: Receiving an interactive instruction; wherein the interactive instruction includes at least one of a feedback instruction and a question instruction; Parsing the interactive instruction, the user request and the image interpretation text through a large language model to obtain a parsing result; The execution result is adjusted according to the analysis result.

[0010] Optionally, the satellite image data interaction method further includes: When a model for processing a specified satellite image task is newly added to the visual basic model library, sample data corresponding to the specified satellite image task is obtained; The large language model is fine-tuned using the sample data.

[0011] Another aspect of the present application provides a satellite image data interaction device, the device comprising: An acquisition module is used to acquire data processing instructions and image interpretation text of satellite image data; A parsing module, used for parsing the data processing instruction and the image interpretation text to determine the user request; A processing module, for retrieving a target visual basic model for processing the satellite image processing request from a visual basic model library if the user request is a satellite image processing request, and formulating a satellite image task strategy; The control module is used to control the target visual basic model to execute the corresponding satellite image task based on the satellite image task strategy, obtain the execution result; and transmit the execution result to the client.

[0012] Another aspect of the present application provides an interactive device for satellite image data, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, the steps of the interactive method for satellite image data are implemented.

[0013] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the satellite image data interaction method.

[0014] The present application provides a method, device and medium for interacting with satellite image data, which have the following beneficial effects: by combining and parsing data processing instructions and image interpretation text, the user's request can be accurately and comprehensively understood. On this basis, according to the user's request, a small model that can implement downstream tasks is retrieved to perform satellite image tasks, that is, a visual basic model that can implement satellite image processing is retrieved to implement the corresponding satellite image processing. Thus, based on the efficient and accurate analysis of satellite image data, the user's needs for fast, accurate and efficient satellite image data interaction are met, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of a satellite image data interaction method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of an interactive system for satellite image data provided in an embodiment of the present application; Figure 3 A schematic diagram of an interactive interface of an interactive system for satellite image data provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a satellite image data interaction device provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a satellite image data interaction device provided in another embodiment of the present application.

[0016] The figure numbers are as follows: 1 is an input module, 2 is a user request parsing module, 3 is a task planning module, 4 is a model calling module, 5 is an output module, 6 is a visual basic model library, 10 is a data instruction acquisition module, 11 is an image interpretation text acquisition module, 50 is a memory, 51 is a processor, 52 is a display screen, 53 is an input and output interface, 54 is a communication interface, 55 is a power supply, 56 is a communication bus, 501 is a computer program, 502 is an operating system, and 503 is data. DETAILED DESCRIPTION

[0017] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0018] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0019] Figure 1 A schematic diagram of a flow chart of a satellite image data interaction method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes: S10: Obtaining data processing instructions and image interpretation text of satellite image data; In a specific embodiment, step S10 is used to obtain data processing instructions and image interpretation text of satellite image data, wherein the data processing instruction refers to an instruction issued by a user to indicate the user's data processing intention, and the image interpretation text of satellite image data refers to the interpretation text obtained after interpreting the satellite image data in pictures and texts.

[0020] In an optional embodiment, the user can input the data processing instruction through a device such as a keyboard and a mouse, or can input the data processing instruction through voice, and the present application does not limit the method in which the user inputs the data processing instruction. In addition, the data processing instruction can be in different forms such as Chinese, English, numbers, etc., and the present application does not limit this.

[0021] Figure 2This is a schematic diagram of the structure of a satellite image data interactive system provided in an embodiment of the present application. For ease of understanding, the following will be combined with Figure 2 Specifically, the satellite image data interaction method provided in this application can be applied to Figure 2 in the system shown.

[0022] like Figure 2 As shown, the interactive system for satellite image data provided in the embodiment of the present application includes: an input module 1, a user request parsing module 2, a task planning module 3, a model calling module 4, an output module 5 and a data basic model library.

[0023] Among them, the input module 1 includes a data instruction acquisition module 10 and an image interpretation text acquisition module 11, wherein the data instruction acquisition module 10 is used to obtain data processing instructions input by the user. In an optional embodiment, the input module 1 includes an interactive interface for interacting with the user. The user can enter the data processing instructions in the instruction input box in the interactive interface in a natural language manner, and press the Enter key to provide it to the system.

[0024] The image interpretation text acquisition module 11 is used to obtain the text after the satellite image data is interpreted, that is, the image interpretation text. In an optional embodiment, the interactive interface allows the user to upload satellite image data. Specifically, the user can upload satellite image data by dragging or selecting a file, and the formats supported by the satellite image data include but are not limited to JPEG format, PNG format and TIFF format.

[0025] After the satellite image data is uploaded, the image interpretation text acquisition module 11 can interpret the satellite image data to obtain the image interpretation text. Of course, in an optional embodiment, other devices can also interpret the satellite image data to obtain the image interpretation text, and the user uploads the image interpretation text to the image interpretation text acquisition module 11 through an interactive interface, which is not limited in this application.

[0026] S11: parsing the data processing instruction and the image interpretation text to determine the user request; Furthermore, the data processing instructions and the image interpretation text are analyzed together, that is, the data processing instructions are analyzed based on the image interpretation text to determine the user request. Specifically, in an optional embodiment, the data processing instructions and the image interpretation text are simultaneously input into the large language model for analysis to build a comprehensive understanding of the user request.

[0027] like Figure 2 As shown, after the input end of the user request parsing module 2 receives the data processing instruction and the image interpretation text acquired by the input module 1, it confirms the user request.

[0028] S12: If the user request is a satellite image processing request, a target visual basic model for processing the satellite image processing request is retrieved from the visual basic model library 6, and a satellite image task strategy is formulated; S13: Based on the satellite image task strategy, control the target visual basic model to execute the corresponding satellite image task, obtain the execution result, and transmit the execution result to the client.

[0029] In an optional embodiment, it is determined according to the analysis result of step S10 whether the current user's request is a satellite image processing request, that is, whether the user's current data processing instruction requires processing of satellite image data.

[0030] If yes, it is determined that a target visual basic module that can be used to process the satellite image processing request needs to be retrieved from the visual basic model library 6. At the same time, a satellite image task strategy is formulated for the current satellite image task according to the user request.

[0031] Furthermore, through step S13, based on the formulated satellite image task strategy, the retrieved target visual basic model is controlled to execute the corresponding satellite image task to obtain the execution result, and the execution result is transmitted to the client so that the user can view the interaction result in time.

[0032] like Figure 2 As shown, when the user request parsing module 2 determines that the user request is a satellite image processing request, a signal is sent to the task planning module 3 so that the task planning module 3 formulates a satellite image task strategy. At the same time, based on the user request determined by the user request parsing module 2, the model calling module 4 retrieves the target visual basic model that can be used to process the current user request from the visual basic model library 6.

[0033] Furthermore, the model calling module 4 controls the called target visual basic model to execute the corresponding satellite imaging task according to the satellite imaging task strategy output by the task planning module 3 .

[0034] It should be noted that the satellite image processing request may include but is not limited to a graphic description request, an image classification request, a target detection request, a scene classification request, a semantic segmentation request, and an image segmentation request. Correspondingly, the models included in the visual basic model library 6 include but are not limited to a graphic description model, an image classification model, a target detection model, a scene classification model, a semantic segmentation model, and an image segmentation model. The corresponding satellite image task is obtained to obtain the execution result. The execution result is transmitted to the output module 5.

[0035] Specifically, the visual basic model library 6 includes a BLIP-RS model for generating satellite image data description, which inputs the satellite image data into a pre-trained graphic description model to obtain image interpretation text, thereby determining the description result corresponding to the image interpretation text. A ResNet Scene model for satellite image data classification, which inputs the satellite image data into a pre-trained scene classification model to determine the scene classification result corresponding to the satellite image data through the scene classification model.

[0036] The YOLO-RS model used for satellite image data target detection inputs the satellite image data into a pre-trained target detection model to determine the target detection and counting results corresponding to the satellite image data through the target detection model. The U-Net model used for satellite image data land use segmentation inputs the satellite image data into a pre-trained semantic segmentation model to determine the land use results corresponding to the satellite image data through the semantic segmentation model.

[0037] It can be understood that the target visual base model is a small model for implementing downstream tasks, that is, a small model for implementing user satellite image processing requests. In an optional embodiment, a user request may call multiple target visual base models, or may call one visual base model, specifically, determined according to the user request. When multiple target visual base models need to be called, the execution order between each target visual base model needs to be planned through the task planning module 3. For ease of understanding, an example will be given below.

[0038] For example, if the user requests to see how many ships there are in the image, the target detection model and target counting model need to be called to detect and count the ships in the satellite image. After performing target detection and target counting, the results are aggregated and summarized to obtain the execution results, which are then displayed on the interactive interface.

[0039] In addition, it should be noted that when the user request is not a satellite image processing request, for example, when the user request is an encyclopedia question, the data processing instruction can be directly parsed by the large language module to output the corresponding request result. Figure 2 As shown, the user request parsing module 2 analyzes the data processing instruction through a large language model, and outputs the request result through the output module 5.

[0040] In fact, the interactive system of satellite image data can be understood as an intelligent agent that can realize user satellite image data interaction. In an optional embodiment, the interactive method of satellite image data provided by this application constructs an intelligent agent that can understand and respond to natural language instructions based on a small-scale large language model (LLM) and a visual basis model (VFMs), thereby efficiently converting from natural language input to satellite image analysis.

[0041] In a specific embodiment, the intelligent agent combines the preset prompt template and the execution results of a single or multiple visual base models, integrates and formats the output, that is, displays or presents the execution results according to a predetermined format or rule, thereby generating a response to the user request.

[0042] The preset template is a prompt word that describes the identity of the agent, helping the agent understand its responsibilities and how to handle tasks. Specifically, when the agent is processing tasks, if it needs to analyze satellite image data, it will call the visual basic model to process the task and output the integrated results.

[0043] Therefore, by combining the pre-trained visual basic model and the large language model with small-scale parameters, the computing and resource requirements are reduced, efficient and accurate satellite data analysis is achieved, and the intelligence level and application scope of satellite multi-tasking are improved.

[0044] The interactive method for satellite image data provided in the embodiment of the present application combines the data processing instructions and the image interpretation text for analysis, and can accurately and comprehensively understand the user's request. On this basis, according to the user's request, a small model that can realize the downstream task is called to perform the satellite image task, that is, the visual basic model that can realize satellite image processing is called to realize the corresponding satellite image processing. Therefore, based on the efficient and accurate analysis of satellite image data, the user's fast, accurate and efficient satellite image data interaction is satisfied, thereby improving the user experience.

[0045] In an optional embodiment, the target visual base model for processing a satellite image processing request is retrieved, and a satellite image task strategy is formulated, including: Determine the task type of the satellite image task according to the user's request; wherein different task types correspond to one satellite image task; When there are multiple task types, the target visual basic model corresponding to each task type is retrieved respectively; Plan the execution order of each target visual basic model to obtain the satellite image mission strategy.

[0046] In a specific embodiment, if it is determined that the user request is a satellite image processing request, the task type of the satellite image task is determined according to the user request. It should be noted that different task types correspond to a satellite image task, and each satellite image task corresponds to calling a target visual basic model. Among them, the task types include but are not limited to image-text interpretation, scene classification, target detection, semantic segmentation, image segmentation, change detection, feature extraction, three-dimensional reconstruction, and disaster detection.

[0047] Furthermore, the number of task types is determined, that is, the number of target visual basic models that need to be called is determined. If the number of tasks is multiple, multiple target visual basic models need to be called. At this time, the target visual basic models corresponding to each task type are respectively called, and the execution order of each target visual basic model is planned to obtain the satellite image task strategy.

[0048] Specifically, by Figure 2 The task planning module 3 in the system plans the execution order of each target visual basic model to obtain the satellite image task strategy, and the model calling module 4 calls the corresponding model from the visual basic model library 6.

[0049] It is understandable that, in a specific embodiment, the large language model analyzes user requests of multiple task types, and the priority between the tasks is determined by the task planning module 3. In fact, the priority between different task types depends on the analysis result of the large language model on the task type.

[0050] For example, if the data processing request input by the user is the number of airplanes in the image, we first need to determine whether there are airplanes in the image, and identify and lock the airplane targets. Specifically, the task type is target detection, and the corresponding model called is the target detection module. Furthermore, after identifying the airplane targets, the number of airplanes is counted through the target counting model. Finally, the results of the two tasks are summarized and the execution results are output. For example, the output result is: the number of airplanes in the image is 10.

[0051] Based on the above examples, it can be seen that the priority of each target visual basic model in executing the task mainly depends on the recognition result of the task type. In a specific embodiment, the task planning module 3 is required to plan the execution order of each target visual basic model to obtain a satellite image task strategy so that each subsequent target visual basic model can perform the task according to the satellite image task strategy.

[0052] Therefore, the interactive method of satellite image data provided by this application allows users who do not have satellite image analysis skills to easily obtain in-depth analysis and parsing results of satellite images, thereby expanding the accessibility and application scope of satellite technology.

[0053] Based on the above embodiment, as an optional embodiment, planning the execution order of each target visual basic model includes: Determine the dependencies between satellite imagery tasks based on the task type; Divide satellite image tasks with dependencies into task subsets; and plan the execution order of satellite image tasks in each task subset; In each task subset, the satellite image tasks with the same execution order are set to be executed in parallel.

[0054] When planning the execution order of each target visual basic model, it is necessary to determine the dependencies between satellite image tasks according to the task type. Specifically, when there is a dependency between satellite image tasks, there is a sequence of execution between them, and the execution order needs to be planned according to the nature of the task type.

[0055] When performing execution sequence planning, specifically, satellite imaging tasks with dependencies are divided into a task subset. In this task subset, there are multiple independent satellite imaging tasks, and the execution sequence needs to be planned for the satellite imaging tasks in each task subset.

[0056] For example, the data processing instruction is that there are several ships in the image, and the types of tasks to be performed include target detection and target counting. Obviously, there is a dependency between the two. The ship targets must be determined first before target counting can be performed. Therefore, when performing execution sequence planning, the target detection model is controlled to perform target detection first, and then the target counting model is controlled to perform target counting.

[0057] In an optional embodiment, it is understood that there is no dependency between satellite image tasks in different task subsets, so satellite image tasks corresponding to different task subsets can execute tasks at the same time, thereby improving task processing efficiency. Specifically, satellite image tasks with the same execution order in each task subset are set to be executed in parallel. For ease of understanding, an example will be given below.

[0058] For example, task subset A includes satellite imaging tasks A1, A2, and A3, and the order of execution from first to last among the satellite imaging tasks in task subset A is satellite imaging tasks A1, A2, and A3. Task subset B includes satellite imaging tasks B1 and B2, and the order of execution from first to last among the satellite imaging tasks in task subset B is satellite imaging tasks B1 and B2.

[0059] In a specific embodiment, first, the satellite imaging task A1 and the satellite imaging task B1 are executed in parallel, then the satellite imaging task A2 and the satellite imaging task B2 are executed in parallel, and finally the satellite imaging task A3 is executed.

[0060] In an optional embodiment, different task subsets may be executed in parallel, and the execution order of satellite image tasks between different task subsets is not limited. Of course, in another optional embodiment, the execution order of different task subsets may also be planned, and this application does not limit this.

[0061] Therefore, the satellite image data interaction method provided in the embodiment of the present application divides satellite image tasks with dependencies into a task subset; and plans the execution order for the satellite image tasks in each task subset. In addition, in each task subset, the satellite image tasks with the same execution order are set to be executed in parallel. Therefore, the satellite image tasks of different task subsets are set to be executed in parallel, which improves the efficiency of the downstream small model in executing tasks, thereby improving the user's satellite image data interaction efficiency and improving the user experience.

[0062] In an optional embodiment, determining the user request according to the data processing instruction and the image interpretation text includes: Extracting first key information from the data processing instruction and second key information from the image interpretation text; Analyze the dependency between the first key information and the second key information; Based on dependencies, user requests are determined.

[0063] It is understandable that in an optional embodiment, the data processing instruction and the image interpretation text can be parsed by a large language model to determine the user request. Specifically, the first key information in the data processing instruction and the second key information in the image interpretation text are extracted, and the dependency between the first key information and the second key information is analyzed. Further, based on the dependency, a comprehensive understanding of the user request is constructed.

[0064] It is understandable that the large language model does not have the ability to understand images. Therefore, it is necessary to interpret the satellite image data into image interpretation text, that is, input the information in the image into the large language model in the form of text, so that the large language module can understand the user's request more accurately and comprehensively based on the image interpretation text. Of course, in an optional embodiment, a model that can understand both images and data processing instructions can also be called, and this application is not limited to this.

[0065] In an optional embodiment, obtaining image interpretation text of satellite image data includes: Receive satellite image data; Preprocessing the satellite image data to obtain target image data; wherein the preprocessing includes image size adjustment and image resolution adjustment; Through the graphic and text description model, the target image data is interpreted graphically and textually to obtain the image interpretation text.

[0066] In a specific embodiment, the user can upload the satellite image data that needs to be processed through the interactive interface. After receiving the satellite image data uploaded by the user, in order to adapt to the input of the model, the acquired satellite image data is preprocessed to obtain the target image data.

[0067] Among them, preprocessing may include but is not limited to image size adjustment and image resolution adjustment. Specifically, the input image size can be adjusted to an image whose width and height are multiples of the preset value of the image, and the resolution is adjusted to an image less than the resolution threshold. For example, the width and height are adjusted to multiples of 64, and the resolution is adjusted to less than 640. The specific adjustment values, preset values ​​and resolution thresholds can be set according to actual business needs, and are not limited to this application.

[0068] Furthermore, the preprocessed target image data is interpreted by the graphic description model to obtain the image interpretation text. In an optional embodiment, after the graphic description model reads the preprocessed target image data, it will re-save the target image data in a preset format (for example, png format) to a new file, and the file storing the target image data will be used as a backup and will not be subsequently processed, and subsequent operations will use the new file for data storage.

[0069] In an optional embodiment, sample data calibrated by an administrator can be obtained to optimize the graphic description model. Of course, the output results of the graphic description model can also be manually corrected, and the corrected data can be used as a sample to optimize the graphic description model, thereby further improving the interactive accuracy of satellite image data.

[0070] Based on the above embodiment, as an optional embodiment, the satellite image data interaction method provided by the present application further includes: Receiving an interactive instruction; wherein the interactive instruction includes at least one of a feedback instruction and a question instruction; Through the large language model, the interactive instructions, user requests and image interpretation texts are parsed to obtain the parsing results; Adjust the execution results based on the analysis results.

[0071] In a specific embodiment, in order to further improve the accuracy of interaction, after the first interaction, interaction instructions are received in real time, wherein the interaction instructions include feedback instructions and questioning instructions. Feedback instructions refer to instructions that allow users to evaluate and provide feedback on the last system execution result during the satellite image data interaction process. Questioning instructions refer to instructions that allow users to further inquire about the last system execution result during the satellite image data interaction process.

[0072] Furthermore, after receiving the interaction instruction, the interaction instruction is parsed again through the large language model. Specifically, the interaction instruction, user request and image interpretation text are input into the large language model so that the large language model can combine and parse the previous and subsequent user requests and interaction instructions based on the image interpretation text, so as to further and more comprehensively construct the user intention and obtain the parsing result.

[0073] Based on the analysis result, the execution result is adjusted. It is understandable that, with the continuous acquisition of interactive instructions, the execution result obtained last time is repeatedly adjusted, and the result that satisfies the user is gradually obtained, thereby improving the user experience.

[0074] In an optional embodiment, the satellite image data interaction method provided in the embodiment of the present application further includes: When a model for processing a specified satellite image task is newly added to the visual basic model library 6, sample data corresponding to the specified satellite image task is obtained; Fine-tune the large language model using sample data.

[0075] In a specific embodiment, the satellite image data interaction method provided in this application can be applied to multiple fields such as geological exploration, environmental monitoring, agricultural development and urban planning, and with the continuous development of satellite technology, it can also be applied to different fields such as natural disaster detection.

[0076] Therefore, in actual use, the satellite image data interaction method provided by the present application can be infinitely adapted to the business needs of various fields. Specifically, a model for processing a specified satellite image task is added to the visual basic model library 6, wherein the model for the specified satellite image task is a model not included in the current visual basic model library 6.

[0077] For example, if the newly added designated satellite image task is fire detection, it is necessary to add a model that can be used to identify fires in the visual basic model library 6 so that it can be retrieved for fire identification later.

[0078] It is understandable that after adding a designated satellite image task, in order to make the large language model understand the newly added satellite image task more accurately, in an optional embodiment, sample data corresponding to the designated satellite image task is obtained, and the large language model is fine-tuned through the sample data. The sample data can be manually annotated sample data, and this application does not limit the acquisition of the sample data.

[0079] It should be noted that Figure 2 The user request parsing module 2 shown in includes an additional training data interface for fine-tuning the model, that is, when a new visual basic model is added to the visual basic model library 6, the new visual basic model is called through the new training data interface.

[0080] Therefore, the interactive system of satellite image data adopts a modular design, so that each basic visual model can be updated and replaced independently, which is convenient for integrating new algorithms and models. It has good scalability and adaptability to meet the ever-changing application needs and technological development.

[0081] Figure 3 The following is a schematic diagram of an interactive interface of an interactive system for satellite image data provided in an embodiment of the present application. In order to facilitate those skilled in the art to better understand the technical solution of the present application, Figure 3 Provide explanation.

[0082] like Figure 3 As shown, in an optional embodiment, in the interactive interface of the interactive system for satellite image data, the user can upload satellite image data by clicking the button to upload the image, or by dragging and dropping. The application does not limit the upload method. In a specific embodiment, the interactive interface allows the user to submit data processing requests, view processing progress, and receive execution results in real time.

[0083] After uploading, Figure 3 As shown, the interactive interface will show that image M is the original image uploaded, and image N is the image N after image and text analysis. Specifically, image N can identify information such as houses, rivers, fields, roads, etc. included in the image, and perform in-depth analysis on it.

[0084] In the interactive interface, according to the task type and specific needs of the user, the scattered information output by the single or multiple visual basic models is integrated and output in the form of a dialogue. Figure 3 As shown, the user can click the Enter Your Instructions dialog box, enter the data processing request, and click the Send button to request a response. If a freeze occurs during the system's identification and processing of the data processing request, you can click the Refresh button to refresh the interface.

[0085] In addition, if Figure 3 As shown in the figure, after the user inputs the data processing request of "what is in the picture", the system gives the answer, and the user can continue to ask "how many planes are there in the picture", that is, continue to receive the user's interactive instructions. Further, the system analyzes all the previous and subsequent request instructions and image interpretation texts to obtain accurate execution results. Finally, it presents them to the user in the interactive GUI interface in the form of natural language or visual images.

[0086] In the above-mentioned embodiments, the interactive method of satellite image data is described in detail. The present application also provides an embodiment corresponding to the interactive device of satellite image data.

[0087] Figure 4A schematic diagram of the structure of a satellite image data interaction device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the device comprises: An acquisition module 40 is used to acquire data processing instructions and image interpretation text of satellite image data; The parsing module 41 is used to parse the data processing instruction and the image interpretation text to determine the user request; The processing module 42 is used for retrieving the target visual basic model for processing the satellite image processing request from the visual basic model library and formulating the satellite image task strategy if the user request is a satellite image processing request; The control module 43 is used to control the target visual basic model to execute the corresponding satellite image task based on the satellite image task strategy, obtain the execution result, and transmit the execution result to the client.

[0088] In addition, the satellite image data interaction device provided in the embodiment of the present application also includes: The task type determination module is used to determine the task type of the satellite imaging task according to the user's request; wherein different task types correspond to one satellite imaging task; A calling module, used for calling the target visual basic model corresponding to each task type respectively when there are multiple task types; The planning module is used to plan the execution order of each target visual basic model and obtain the satellite image mission strategy.

[0089] A dependency determination module is used to determine the dependency between satellite image tasks according to the task type; The task subset processing module divides satellite image tasks with dependencies into a task subset and plans the execution order of satellite image tasks in each task subset; The setting module is used to set the satellite image tasks with the same execution order in each task subset to be executed in parallel.

[0090] An extraction module, used for extracting the first key information in the data processing instruction and the second key information in the image interpretation text; A dependency analysis module, used for analyzing the dependency between the first key information and the second key information; The user request determination module is used to determine the user request based on the dependency.

[0091] A satellite image data receiving module, used for receiving satellite image data; A preprocessing module is used to preprocess satellite image data to obtain target image data; wherein the preprocessing includes image size adjustment and image resolution adjustment; The image-text interpretation module is used to interpret the target image data through the image-text description model to obtain the image interpretation text.

[0092] An interactive instruction receiving module, used to receive interactive instructions; wherein the interactive instructions include at least one of feedback instructions and questioning instructions; The interactive instruction processing module is used to parse the interactive instructions, user requests and image interpretation texts through a large language model to obtain parsing results; The adjustment module is used to adjust the execution result according to the analysis result.

[0093] The fine-tuning module is used to obtain sample data corresponding to the specified satellite image task when a model for processing the specified satellite image task is added to the visual basic model library; and fine-tune the large language model through the sample data.

[0094] Figure 5 A schematic diagram of a satellite image data interaction device provided in another embodiment of the present application is shown in FIG. Figure 5 As shown, the satellite image data interactive device includes: a memory 50 for storing a computer program; The processor 51 is used to implement the steps of the satellite image data interaction method mentioned in the above embodiment when executing the computer program.

[0095] The satellite image data interaction device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0096] Among them, the processor 51 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 51 can be implemented in at least one hardware form of a digital signal processor (Digital Signal Processor, referred to as DSP), a field programmable gate array (Field-Programmable Gate Array, referred to as FPGA), and a programmable logic array (Programmable Logic Array, referred to as PLA). The processor 51 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (Central Processing Unit, referred to as CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 51 may be integrated with a graphics processing unit (Graphics Processing Unit, referred to as GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 51 may also include an artificial intelligence (Artificial Intelligence, referred to as AI) processor, which is used to process computing operations related to machine learning.

[0097] The memory 50 may include one or more computer-readable storage media, which may be non-transitory. The memory 50 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 50 is at least used to store the following computer program 501, wherein, after the computer program is loaded and executed by the processor 51, it can implement the relevant steps of the interactive method of satellite image data disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 50 may also include an operating system 502 and data 503, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 502 may include Windows, Unix, Linux, etc. Data 503 may include but is not limited to relevant data involved in the interactive method of satellite image data, etc.

[0098] In some embodiments, the satellite image data interactive device may further include a display screen 52 , an input / output interface 53 , a communication interface 54 , a power supply 55 , and a communication bus 56 .

[0099] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the interactive device for satellite image data, and may include more or fewer components than those shown in the figure.

[0100] The satellite image data interaction device provided in the embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, the satellite image data interaction method in the above embodiment can be implemented.

[0101] It should be noted that, although the operations are depicted in a specific order in the accompanying drawings, this should not be understood as requiring these operations to be performed in the specific order shown or to be performed sequentially, or requiring all illustrated operations to be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A satellite image data interaction method, characterized in that: The method comprises: Obtain data processing instructions and image interpretation text of satellite image data; Parsing the data processing instruction and the image interpretation text to determine a user request; If the user request is a satellite image processing request, a target visual basic model for processing the satellite image processing request is retrieved from a visual basic model library, and a satellite image task strategy is formulated; Based on the satellite image task strategy, the target visual basic model is controlled to execute the corresponding satellite image task to obtain an execution result; and the execution result is transmitted to the client.

2. The satellite image data interaction method according to claim 1, characterized in that: Retrieve the target visual base model for processing the satellite image processing request and formulate a satellite image mission strategy, including: Determining a task type of the satellite imaging task according to the user request; wherein different task types correspond to one satellite imaging task; When the task type includes multiple ones, respectively calling the target visual basic model corresponding to each task type; Plan the execution order of each of the target visual basic models to obtain the satellite image mission strategy.

3. The satellite image data interaction method according to claim 2, characterized in that: Planning the execution order of each of the target visual base models, including: Determining the dependency relationship between the satellite image tasks according to the task type; Divide the satellite image tasks with the dependency relationship into a task subset; and plan the execution order of the satellite image tasks in each of the task subsets; The satellite image tasks with the same execution order in each of the task subsets are set to be executed in parallel.

4. The satellite image data interaction method according to claim 1, characterized in that: Determining a user request according to the data processing instruction and the image interpretation text includes: Extracting first key information from the data processing instruction and second key information from the image interpretation text; analyzing the dependency between the first key information and the second key information; Based on the dependencies, the user request is determined.

5. The satellite image data interaction method according to claim 1, characterized in that: Obtain image interpretation text of satellite image data, including: Receiving the satellite image data; Preprocessing the satellite image data to obtain target image data; wherein the preprocessing includes image size adjustment and image resolution adjustment; The target image data is interpreted by graphics and text through a graphics and text description model to obtain the image interpretation text.

6. The satellite image data interaction method according to claim 1, characterized in that: The method further comprises: Receiving an interactive instruction; wherein the interactive instruction includes at least one of a feedback instruction and a question instruction; Parsing the interactive instruction, the user request and the image interpretation text through a large language model to obtain a parsing result; The execution result is adjusted according to the analysis result.

7. The satellite image data interaction method according to claim 6, characterized in that: The method further comprises: When a model for processing a specified satellite image task is newly added to the visual basic model library, sample data corresponding to the specified satellite image task is obtained; The large language model is fine-tuned using the sample data.

8. An interactive device for satellite image data, characterized in that: The device comprises: An acquisition module is used to acquire data processing instructions and image interpretation text of satellite image data; A parsing module, used for parsing the data processing instruction and the image interpretation text to determine the user request; A processing module, for retrieving a target visual basic model for processing the satellite image processing request from a visual basic model library if the user request is a satellite image processing request, and formulating a satellite image task strategy; The control module is used to control the target visual basic model to execute the corresponding satellite image task based on the satellite image task strategy, obtain the execution result; and transmit the execution result to the client.

9. An interactive device for satellite image data, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the satellite image data interaction method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the satellite image data interaction method described in any one of claims 1 to 7 are implemented.

Images