SpaceGPT: visual language model authorized remote sensing data service platform

Abstract

Remote sensing satellite data service is limited due to lack of data processing and analysis capabilities, and a current solution depends on manual intervention, so that the manual intervention is low in efficiency, the large-scale processing cost is too high, and human errors are easy to occur. A system is provided to enable a user to extract useful insights through natural language text queries that utilize large language models to understand user intent and elicit corresponding computer vision models that are fine tuned with remote sensing image datasets, such as open vocabulary object detection and object segmentation models with the latest model architecture. Through such revolutionary visual language model interaction established on a cloud native platform, the system can help customers access satellite data and obtain insights faster and more easily than before at any time, and help companies, governments and folk society use satellite images to discover feasible insights of related important phenomena. Such as deforestation, agriculture, climatic changes, biodiversity, and global supply chains.

Description

[0001] abbreviation

[0002] AI

[0003] CLIP Contrastive Language-Image Pre-training

[0004] DETR Detection Converter

[0005] DOTA dataset for object detection in aerial images

[0006] LLM Large Language Model

[0007] OV-DETR Open Vocabulary Detection Transformer

[0008] SAM Segment Anything Model

[0009] VFM Visual Basic Model

[0010] VLM Visual Language Model Technical Field

[0011] The present disclosure generally relates to machine learning models. In particular, the present disclosure relates to visual language models. Background Art

[0012] The analysis and application of aerospace big data belongs to the field of aerospace technology. With the advancement of observation and computer technology, organizations now have access to massive amounts of heterogeneous aerospace data every day. Remote sensing satellite data has great potential in providing insights (e.g., asset / commodity / carbon monitoring) to a variety of customers from governments, businesses to individuals. Common data analysis methods used for satellite data analysis and data services include feature extraction, image classification, segmentation, object detection, change detection, etc.

[0013] The traditional satellite data preprocessing process is limited by insufficient data processing and analysis capabilities, and requires a lot of manual intervention to extract usable information from massive remote sensing data, resulting in low production efficiency, high large-scale processing costs, and prone to human errors. For example, after the 2008 Wenchuan earthquake, more than 190,000 co-seismic surface landslides occurred. The current landslide list is all manually identified and compiled, which is very time-consuming and labor-intensive.

[0014] There is a need in the art for improved techniques for extracting usable information from massive amounts of remote sensing data while achieving a higher level of automation in extracting the information so as to reduce the amount of human intervention involved. Summary of the invention

[0015] In the present disclosure, SpaceGPT is introduced for chatting with users in natural language and processing remote sensing images according to instructions issued or implied by users in the chat in natural language. SpaceGPT is a unified model capable of learning from multiple label modalities in various datasets including remote sensing image datasets (e.g., object detection and object segmentation) with visual language model capabilities connecting LLM and one or more VFMs to enable sending and receiving images during chat. Users can use text queries to ask the model to perform certain image-related tasks. The automation of analyzing remote sensing images is advantageously realized without the intervention or assistance of human experts.

[0016] A first aspect of the present disclosure is to provide a computer-implemented method for using SpaceGPT to chat with a user in natural language and perform image-related tasks mentioned or implied by the user in a query. The image-related tasks are performed on one or more remote sensing images.

[0017] In the method, a platform for bidirectional communication with a user and performing image-related tasks is established in software. The platform includes a prompt manager for communicating and prompting the LLM. The prompt manager is arranged to: forward the query to the LLM so that the LLM identifies the image-related task from the query and determines one or more image processing actions to be performed on the one or more remote sensing images to complete the image-related task; if the LLM determines that the individual image processing operation matches the image processing operation executable by the matching VFM, prompt the LLM to trigger the matching VFM selected from a predetermined set of one or more VFMs to perform the individual image processing operation; and receive a reply from the LLM to the user about any result of the image-related task. After the platform is established, the platform is used to receive a query from the user, process the query and forward the reply to the user. Before the platform is used to process the query, one or more selected VFMs are fine-tuned with one or more remote sensing image data sets. Therefore, advantageously, one or more corresponding image processing operations that can be performed by one or more selected VFMs are suitable or optimized for remote sensing-related image processing. The one or more selected VFMs are selected from a predetermined set of one or more VFMs. It is possible to select all VFMs in the predetermined set.

[0018] Preferably and advantageously, one or more selected VFMs are fine-tuned using one or more remote sensing image datasets in a self-supervised manner.

[0019] In some embodiments, the one or more corresponding image processing operations performed by the one or more selected VFMs include one or more operations selected from an image classification operation, an object detection operation, and an image segmentation operation.

[0020] In some embodiments, the one or more selected VFMs include one or more machine learning models selected from OV-DETR, Grounding Dino, and Segment Anything Model.

[0021] In certain embodiments, the one or more corresponding image processing operations performed by the one or more selected VFMs include one or more operations for detecting or identifying one or more types of hydrological or geomorphic hazards.

[0022] One or more types of hydrological or geomorphic disasters may include floods, landslides, or both.

[0023] Preferably and advantageously, the platform is established in a cloud computing environment.

[0024] In some embodiments, the platform also includes a command interface for interacting with a user. The command interface is arranged to receive queries from the user, forward the received queries to the prompt manager, and forward replies to the user. The command interface is a command interface supported by the LLM. The command interface may also be arranged to support sending and receiving image files during a chat.

[0025] In some embodiments, the LLM is selected as ChatGPT.

[0026] In certain embodiments, the platform further comprises a LLM and a predetermined set of one or more VFMs such that the platform is self-contained with the visual language model.

[0027] A second aspect of the present disclosure is to provide a system for chatting with a user in natural language and performing an image-related task mentioned or implied by the user in a query during the chat. The image-related task is performed on one or more remote sensing images.

[0028] The system includes one or more computers networked together. The one or more computers are configured to execute a computing process for chatting with a user in a natural language and perform image-related tasks as set forth in any embodiment of the disclosed method.

[0029] As shown in the following examples, other aspects of the disclosure are disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 An example of a user interface of SpaceGPT is depicted, in which a normal user communicates with SpaceGPT's LLM in natural language, and the user interface shows that SpaceGPT accepts airport images and returns output images after completing image-related tasks indicated by the user via a query.

[0031] Figure 2A schematic diagram illustrating an exemplary structure of SpaceGPT is depicted, where the structure is a software model.

[0032] Figure 3 A flowchart illustrating typical operation of SpaceGPT is depicted using a non-limiting example of running SpaceGPT.

[0033] Figure 4 Depicted is a workflow illustrating exemplary steps of a method of utilizing SpaceGPT to chat with a user in natural language and perform image-related tasks mentioned or implied by the user in a query during the chat.

[0034] Those skilled in the art will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. DETAILED DESCRIPTION

[0035] The present disclosure relates to a remote sensing data service platform that utilizes a visual language model named SpaceGPT for chatting with users in natural language and processing remote sensing images according to instructions issued or implied by the user in the chat in natural language. Advantageously, even if the user is a technical layman in computer vision and does not know the appropriate computer vision operations that need to be performed when analyzing remote sensing images to achieve a certain high-level purpose, the automation of analyzing remote sensing images can be achieved without the intervention or assistance of human experts. Therefore, the advantages of time efficiency and cost reduction are obtained in analyzing remote sensing images. In order to achieve these advantages, the data service platform utilizes LLM to understand the user's intention when processing images and triggers (or calls, provokes) relevant computer vision AI models fine-tuned on remote sensing image datasets, such as open vocabulary object detection and object segmentation models with state-of-the-art model architectures, to enable users to extract useful insights through natural language text queries.

[0036] Before proceeding further, it is instructive to briefly describe how the average user interacts with SpaceGPT when analyzing remote sensing imagery. Figure 1An example of a user interface 100 of SpaceGPT is depicted, in which an ordinary user communicates with the LLM of SpaceGPT in natural language. On the user interface 100, the user interacts with the LLM in a dialogue using a chat box and instructs SpaceGPT to perform image processing tasks through natural language queries. The user first sends an aerial image 110 of an airport to SpaceGPT, and subsequently asks spaceGPT in natural language with a query 120: "Can you help detect all the aircraft on this airport in the RS image?" The LLM interprets the query 120 and understands that SpaceGPT has been assigned the task of identifying all aircraft appearing on the airport image 110. After this task is completed, SpaceGPT provides the results of the task by displaying an output image 130 on the user interface 100. In the output image 130, the identified aircraft are each labeled and bounded by a bounding box.

[0037] A first aspect of the present disclosure is to provide a computer-implemented method for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat. The image-related tasks are performed on one or more remote sensing images. The disclosed method utilizes SpaceGPT to understand the user's intent when processing one or more remote sensing images in order to determine the image-related tasks. With the help of Figures 2 to 4 The disclosed method is exemplified. Figure 2 A schematic diagram is depicted for illustrating an example structure of a SpaceGPT (referenced as 200) and showing the organization of example elements forming the SpaceGPT 200, wherein Figure 2 The structure shown in is a software model. Figure 3 A flow chart illustrating typical operation of SpaceGPT 200 is depicted, as well as non-limiting examples of operating SpaceGPT 200 . Figure 4 An operational flow 400 illustrating example steps of the disclosed method is depicted. The operational flow 400 includes step 410, step 420, and step 430.

[0038] Step 410 is an initialization step. In step 410, the platform 205 for bidirectional communication with the user and performing image-related tasks is established in software. In particular, the platform 205 includes a prompt manager 240 for communicating and prompting the LLM 210.

[0039] The LLM 210 is capable of understanding natural language in order to identify image-related tasks from queries (labeled 291) made by a user in a chat (labeled 380). The image-related tasks may be explicitly mentioned in the query 291, or may be "implied" or "implicit" in the query 291, such that the LLM 210 needs to first understand the user's intent from the words of the query 291, and then estimate the image-related tasks. In addition, the LLM 210 is also capable of determining one or more image processing actions to be performed on one or more remote sensing images (labeled 382) to complete the image-related tasks. A practical example of the LLM 210 is ChatGPT.

[0040] The prompt manager 240 is arranged (or programmed) to perform at least the following three procedures. First, the prompt manager 240 forwards a query 291 made by a user to the LLM 210. As a result of this forwarding of the query 291, the LLM 210 is triggered or caused to identify an image-related task from the query 291 and determine one or more image processing actions to be performed on one or more remote sensing images 382 to complete the image-related task. Second, if the LLM 210 determines that the individual image processing operation matches an image processing operation executable by a matching VFM, the prompt manager 240 prompts the LLM 210 to cause a matching VFM selected from a predetermined set of one or more VFMs 220 to perform the individual image processing operation. Third, the prompt manager 240 receives a reply 292 from the LLM 210 to the user regarding any results of the image-related task.

[0041] like Figure 2 As shown in , SpaceGPT 200 includes a prompt manager 240, LLM 210, and a predetermined set of one or more VFMs 220. As a result of such integration, SpaceGPT 200 is a unified model capable of learning from multiple label modalities in various datasets (e.g., object detection and object segmentation) with visual language model capabilities connecting LLM 210 (e.g., ChatGPT) and (one or more) VFMs 220 (such as Grounding Dino) to enable sending and receiving images during chat. That is, a user can ask SpaceGPT 200 to perform certain image-related tasks using text queries.

[0042] In practical situations, it is common for the prompt manager 240 to be operated by one service provider, while the LLM 210 and one or more VFMs 220 are operated by (one or more) other service providers or may even be open access software applications. Figure 2As shown in FIG. 1 , it is reasonable that the platform 205 does not include the LLM 210 and (one or more) VFM 220. However, this exclusion is due to non-technical reasons. It is also possible to incorporate the LLM 210 and (one or more) VFM 220 into the platform 205 to form an enlarged platform (such as Figure 2 In some embodiments, the platform 205 also includes a predetermined set of LLMs 210 and one or more VFMs 220, such that the platform 205 is a self-contained visual language model (SpaceGPT 200).

[0043] Typically, the platform 205 also includes a command interface 230 for interacting with the user. The command interface 230 is arranged to receive queries 291 from the user, forward the received queries 291 to the prompt manager 240 and forward replies 292 to the user. It should be noted that the command interface 230 is an LLM-supported command interface that allows text communication between the user and the SpaceGPT 200 in natural language. It should be noted that, in general, the command interface 230 is also arranged to support sending and receiving image files during the chat.

[0044] refer to Figure 4 After performing the initialization step 410, the platform 205 is deployed in step 430 to receive queries 291 from users, process the queries 291 and forward replies 292 to the users.

[0045] It is desirable to customize some selected VFMs in the predetermined set of one or more VFMs 220 for remote sensing-based image processing, so that a higher level of performance can be achieved when performing corresponding image processing operations compared to non-customized versions of the aforementioned selected VFMs. Before using the platform 205 to process the query 291 in step 430, or more practically, before initiating the chat 380, in step 420, one or more selected VFMs 225 are fine-tuned with one or more remote sensing image datasets (as training datasets), wherein the one or more selected VFMs 225 are selected from the predetermined set of one or more VFMs 220. Optionally, all of the one or more VFMs 220 in the predetermined set are selected as the one or more selected VFMs 225. As an advantageous result, one or more corresponding image processing operations that can be performed by the one or more selected VFMs 225 are suitable or optimized for remote sensing-related image processing. It should be noted that step 420 is also an initialization step.

[0046] Preferably and advantageously, one or more selected VFMs 225 are selected such that such VFMs 225 can be fine-tuned in a self-supervised manner. Thus, the time cost of preparing one or more remote sensing image datasets for fine-tuning purposes can be made lower compared to the case of fully-supervised training.

[0047] It is also preferred that the one or more selected VFMs 225 are selected such that the one or more corresponding image processing operations performed by the one or more selected VFMs 225 include one or more operations selected from an image classification operation, an object detection operation, and an image segmentation operation. The latter three operations are typically used for remote sensing-based image processing.

[0048] Examples of VFMs 225 that perform any of these three operations include OV-DETR, Grounding Dino, and SAM. In some embodiments, one or more selected VFMs 225 include one or more machine learning models selected from OV-DETR, Grounding Dino, and SAM. More details of the latter three machine learning models are provided below.

[0049] OV-DETR is an open vocabulary detector based on an end-to-end transformer that formulates the learning objective as a binary matching objective between an input query and a corresponding object. It conditions the transformer decoder based on the input embedding obtained from a pre-trained visual language model such as CLIP. Clips are fine-tuned on remote sensing datasets such as Satlas and DOTA. For more details on OV-DETR, see Y. ZANG, W. LI, K. ZHOU, C. HUANG, and C. C. LOY, "Open Vocabulary DETR with Conditional Matching," arXiv:2203.11876 [cs.CV], 2022, the disclosure of which is incorporated herein by reference.

[0050] Grounding Dino is an open-set object detector that combines a transformer-based detector DINO with grounded pre-training to detect arbitrary objects with human input (such as category names or quoted expressions). It accepts (image, text) pairs as input and outputs 900 (by default) object boxes with similarity scores across all input words. Grounding Dino adopts the following architecture: text backbone, image backbone, feature enhancer, language-guided query selection, and cross-modal decoder. Grounding Dino serves the object detection task with comparable accuracy and generalization ability as OV-DETR, but with different fine-tuning methods on remote sensing datasets. When the results of OV-DETR are not satisfactory, GroundingDino provides an alternative. For more details on Grounding Dino, see S.LIU et al., "Combining DINO with Grounded Pre-training for Open Set Object Detection", arXiv:2303.05499[cs.CV], 2023, which is incorporated herein by reference.

[0051] SAM is a promptable segmentation system with zero-shot generalization capabilities to unfamiliar objects and images without additional training. SAM generates high-quality object masks based on input prompts (such as points or boxes), and can be used to generate masks for all objects in an image. It serves the task of object segmentation on remote sensing images. For more details on SAM, see A.KIRILLOV, "Segment Anything", arXiv:2304.02643 [cs.CV], 2023, the disclosure of which is incorporated herein by reference.

[0052] In addition to image classification operations, object detection operations, and image segmentation operations that are generally considered basic image processing tools, one or more higher-level image processing tools directly related to remote sensing may also be performed in one or more selected VFMs 225. In some embodiments, one or more corresponding image processing operations performed by one or more selected VFMs 225 may include or may also include one or more operations for detecting or identifying one or more types of hydrological or geomorphic disasters. Specifically, each of the one or more operations is configured to detect or identify a corresponding type of hydrological or geomorphic disaster. Examples of hydrological or geomorphic disasters include flood events and landslide events. In some embodiments, one or more types of hydrological or geomorphic disasters include floods, landslides, or both. Typically, the one or more operations described above are operations based on image analysis.

[0053] In summary, one or more selected VFMs 225 obtained after fine-tuning form a remote sensing fine-tuning AI algorithm library 225, which contains AI models fine-tuned on one or more remote sensing image datasets and can complete common tasks on remote sensing images, such as image classification, object detection, and image segmentation.

[0054] Despite Figure 4 4 shows that step 410 of establishing platform 205 precedes step 420 of fine-tuning one or more selected VFMs 225, but the present disclosure is not limited to executing step 410 and step 420 in the above order. It is possible to swap the order of executing step 410 and step 420 so that step 420 is executed before executing step 410.

[0055] It is instructive to provide an example to illustrate the typical operation flow performed in SpaceGPT 200 regarding processing chat 380 and processing query 291. Assume that step 502 has been performed so that the remote sensing fine-tuning AI algorithm library 225 has been established in a predetermined set of one or more VFMs 220. Figure 3 In SpaceGPT 200, a prompt manager 240 receives information about system principles 286 (such as system limitations) and knowledge of a predetermined set of one or more VFMs 220. The knowledge of the predetermined set of one or more VFMs 220 enables the prompt manager 240 to prompt the LLM 210 about the available VFM(s) 220 that can be triggered. When the prompt manager 240 receives a query 291, the prompt manager 240 forwards the query 291 to the LLM 210. Upon receiving the query 291, the LLM 210 identifies or estimates the image-related tasks to be performed by understanding the query 291, and determines an action plan with one or more image processing actions to be performed. The LLM 210 iteratively performs the one or more image processing actions. In each iteration, the LLM 210 determines whether an appropriate VFM needs to be triggered (step 260). If a positive answer is obtained, the aforementioned appropriate VFM is executed (step 261) to give an intermediate answer 262. The intermediate answer 262 is then used in the subsequent image processing actions taken in the next iteration. If in any iteration the LLM 210 concludes that no further VFM will be induced when completing the image-related task, a reply 292 to the user is prepared by the LLM 210 and sent to the user. The reply 292 may contain the last obtained intermediate answer 262. Optionally, the conversation history 281 and the reasoning history 282 in the chat 380 may be used to assist the LLM 210 in completing the image-related task more efficiently or more accurately.

[0056] Other implementation details of the disclosed method or SpaceGPT 200 are described as follows.

[0057] Preferably and advantageously, the platform 205 (or 205a) is built in a cloud computing environment. Since the platform 205 / 205a is cloud-native, it enables users to input desired remote sensing image data or data streams on the cloud or in a user device owned by the user (such as a tablet computer), and perform machine learning analysis through visual ChatGPT interaction to discover actionable insights.

[0058] It will be appreciated by those skilled in the art that platform 205 (or 205a) can be implemented by appropriate programming on a computer system or equivalent. A computer system can be a computer or a group of multiple computers networked together. A separate computer can be a general-purpose computer, a computer equipped with (one or more) dedicated AI computing processors, a desktop computer, a mobile computing device, a computing server, a distributed server (such as implemented in a cloud computing environment), or any computing device. Networking The aforementioned multiple computers can be networked via cables or optical fibers, or can be networked wirelessly. The aforementioned multiple computers can be networked on the Internet.

[0059] Image files of one or more remote sensing images 382 may be uploaded to SpaceGPT 200 by a user via command interface 230. If one or more remote sensing images 382 are stored at a certain location (e.g., stored on a storage device accessible via the Internet), the user may only need to provide SpaceGPT 200 with the address of the location.

[0060] A second aspect of the present disclosure is to provide a system for chatting with a user in natural language and performing an image-related task mentioned or implied by the user in a query during the chat, wherein the image-related task is performed on one or more remote sensing images.

[0061] The system includes one or more computers networked together. The one or more computers are configured to execute a computing process for chatting with a user in a natural language and perform image-related tasks according to any embodiment of the disclosed method detailed above when describing the workflow 400.

[0062] Reference is made above to details relating to one or more computers and a network that networks the computers together.

[0063] Finally, the advantages of SpaceGPT 200 are summarized as follows.

[0064] First, SpaceGPT 200 enables high-accuracy open vocabulary object detection and object segmentation on remote sensing satellite imagery. Using state-of-the-art model architectures, object detection and object segmentation models are trained and fine-tuned on multiple large-scale remote sensing image datasets in a self-supervised manner, which ideally achieves comparable accuracy to closed-set object detection and object segmentation models. It enables the disclosed visual language model 200 to generalize beyond the limited number of base classes labeled during the training phase, and users are able to detect and object segment common objects in remote sensing images beyond the base classes. That is, even if the training dataset used to train SpaceGPT 200 does not contain the target object, the user may be able to successfully detect the target object. SpaceGPT 200 helps remote sensing data analysis avoid the limitations of using a small number of high-quality remote sensing image datasets due to the variability of remote sensing images in scale, resolution, and elevation.

[0065] Secondly, SpaceGPT 200 allows queries using natural language with little or no prior knowledge of the relevant remote sensing satellite data features. SpaceGPT 200 uses a cutting-edge multi-modality VLM with both text understanding and image understanding to allow users to perform object detection and object segmentation tasks by using natural language text queries. LLM 210 is able to understand user intent and trigger corresponding (one or more) computer vision models, such as object detection or object segmentation. This feature of SpaceGPT 200 provides users with a revolutionary big data insight generation experience, even if they are new to remote sensing image processing.

[0066] Third, built on a cloud-native platform, SpaceGPT 200 helps all users, regardless of their level of expertise, access satellite data and gain insights faster and easier than ever before. It also helps companies, governments and civil society use satellite imagery to discover actionable insights on important phenomena such as deforestation, agriculture, climate change, biodiversity and global supply chains.

[0067] Without departing from the spirit or essential features of the present disclosure, the present disclosure may be implemented in other specific forms. Therefore, the present embodiment is considered to be illustrative and not restrictive in all aspects. The scope of the present invention is indicated by the appended claims rather than by the preceding description, and therefore, all changes within the meaning and equivalent range of the claims are intended to be included therein.

Claims

1. A computer-implemented method for chatting with a user in natural language and performing an image-related task mentioned or implied by the user in a query, the image-related task being performed on one or more remote sensing images, the method comprising: A platform is established in software for bidirectionally communicating with the user and performing the image-related task, wherein the platform includes a prompt manager for communicating and prompting a large language model (LLM), the prompt manager being arranged to: forwarding the query to the LLM so that the LLM identifies the image-related task from the query and determines one or more image processing actions to be performed on the one or more remote sensing images to complete the image-related task; If the LLM determines that the individual image processing operation matches an image processing operation that can be performed by the matched visual basis model (VFM), prompting the LLM to cause the matched VFM selected from a predetermined set of one or more VFMs to perform the individual image processing operation; as well as receiving a response from the LLM to the user regarding any results of the image-related task; receiving the query from the user using the platform, processing the query and forwarding the reply to the user; as well as Before the platform is used to process the query, one or more selected VFMs are fine-tuned using one or more remote sensing image datasets so that one or more corresponding image processing operations performed by the one or more selected VFMs are suitable for or optimized for remote sensing related image processing, wherein the one or more selected VFMs are selected from a predetermined set of the one or more VFMs.

2. The method of claim 1, wherein the one or more selected VFMs are fine-tuned using the one or more remote sensing image datasets in a self-supervised manner.

3. The method of claim 1, wherein the one or more corresponding image processing operations performed by the one or more selected VFMs include one or more operations selected from an image classification operation, an object detection operation, and an image segmentation operation.

4. The method of claim 1, wherein the one or more selected VFMs comprise one or more machine learning models selected from OV-DETR, Grounding Dino, and Segment Anything Model.

5. The method of claim 1, wherein the one or more corresponding image processing operations performed by the one or more selected VFMs include one or more operations for detecting or identifying one or more types of hydrological or geomorphic disasters.

6. The method of claim 5, wherein the one or more types of hydrological or geomorphic disasters include floods, landslides, or both. The method according to claim 1 , wherein the platform is established in a cloud computing environment.

8. The method of claim 1, wherein the platform further comprises a command interface for interacting with the user, the command interface being arranged to receive queries from the user, forward the received queries to the prompt manager and forward the replies to the user.

9. The method of claim 8, wherein the command interface is further arranged to support sending and receiving image files during a chat.

10. The method of claim 1, wherein the LLM is selected as ChatGPT.

11. The method of claim 1 , wherein the platform further comprises a predetermined set of the LLM and the one or more VFMs such that the platform is a self-contained visual language model.

12. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 1.

13. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 2.

14. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 3.

15. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 4.

16. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 5.

17. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 6.

18. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 7.

19. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, wherein the image-related tasks are performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to execute a computing process for chatting with the user in natural language and performing the image-related tasks according to the method according to claim 8.

20. A system for chatting with a user in natural language and performing image-related tasks mentioned or implied by the user in a query during the chat, the image-related tasks being performed on one or more remote sensing images, the system comprising one or more computers networked together, wherein the one or more computers are configured to perform chatting with the user in natural language and The method according to claim 9 performs the computational process of the image-related task.

Images