GIS tool chain parameter level intelligent arrangement method and system based on large language model instruction fine tuning

Abstract

The present application relates to the technical field of artificial intelligence, and provides a GIS tool chain parameter level intelligent arrangement method and system based on large language model instruction fine tuning. The method comprises: collecting and preprocessing geographic information tools, guiding a large language model to generate geographic space task instructions and corresponding geographic space task tool chains based on the preprocessed geographic information tools, so as to construct instruction fine tuning data sets and evaluation data sets; using an open source large language model as a basic model, fine tuning the basic model based on the instruction fine tuning data sets and the evaluation data sets by using LoRA fine tuning technology; and automatically completing the arrangement and parameter setting of the geographic space task tool chain by using the fine tuned basic model driven by target geographic task instructions.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for parameter-level intelligent orchestration of GIS toolchains based on fine-tuning of large language model instructions. Background Technology

[0002] Geographic Information Systems (GIS), as a core tool for spatial analysis and decision support, have been widely applied in various fields such as urban planning, traffic management, disaster response, and resource management. Faced with increasingly complex spatial analysis needs, users typically need to connect multiple geospatial tools to form "tool-use chains" containing multiple input and output steps to solve problems. Therefore, automating the geospatial task solving process is crucial for significantly improving the efficiency of GIS applications.

[0003] Traditionally, rule-based systems (such as ArcGIS ModelBuilder) have been used to solidify specific workflows, but their lack of flexibility makes them difficult to adapt to dynamic changes in tasks. In recent years, Large Language Models (LLMs) have demonstrated powerful semantic understanding, reasoning, and code generation capabilities, providing new avenues for solving intelligent geospatial tasks. Researchers have begun exploring the deep integration of LLMs with GIS, resulting in various parallel evolutionary research methods and technical paths aimed at achieving automation at different levels and in different scenarios.

[0004] Deeply embedding LLMs into existing GIS platforms (such as QGIS) has become a mainstream practice for achieving "copilot"-style interaction. This type of research aims to overcome the challenges of integrating AI with complex GIS environments, including tool diversity, heterogeneous data standards, and accurate conversion of natural language to GIS commands. The GIS Copilot plugin guides LLMs to generate accurate PyQGIS code through a structured tool description library and code review module, demonstrating high success rates in basic and intermediate tasks. This work marks an evolution from externally invoked tools to platform-integrated intelligent agents, with the advantage of tightly integrating the computational ecosystem and visualization capabilities of the GIS software itself, enhancing the transparency of analysis and the controllability of results. Utilizing the planning and tool invocation capabilities of commercial LLMs has become a key path to achieving high-performance, general-purpose automated systems. The GeoGPT framework, centered on GPT-4, uses prompting engineering to drive autonomous planning and invocation of a predefined tool pool to complete the entire workflow from data collection to visualization. Such systems demonstrate exceptional accuracy in handling complex workflows involving up to eight steps, highlighting the core role of powerful LLMs in planning complex tasks.

[0005] However, the superior performance of LLM heavily relies on closed-source commercial models such as GPT-4, which presents significant limitations in terms of data privacy, localized deployment, and long-term usage costs. Furthermore, LLM-based methods still face challenges such as tool semantic ambiguity and output illusion. To address the issues of LLM's reliance on and illusion of expertise, Retrieval Augmentation (RAG) technology has been systematically introduced into geospatial task automation frameworks. RAG retrieves relevant information from external knowledge bases (such as authoritative scientific reports, professional literature, or structured tool documents), providing real-time and reliable knowledge support for the LLM's decision-making and generation processes, thereby significantly improving the accuracy and reliability of the output. In the GIS field, RAG is considered a potential solution to address the limitation of verbose tool documents exceeding the model's context, enabling LLM to dynamically retrieve the parameter details and usage examples of the required tools. GeoCogent systematically constructs a multimodal geospatial knowledge base and deeply integrates the RAG mechanism, effectively improving the accuracy of code generation. However, its performance is still limited by retrieval quality and real-time performance, and it cannot fundamentally internalize domain knowledge into the model's stability.

[0006] Despite the progress made, existing research still faces several key challenges: First, most systems capable of handling complex toolchains rely on closed-source commercial LLMs, and the performance of open-source alternatives still needs improvement; second, solving geospatial tasks typically involves multi-tool orchestration and complex parameter configuration, while existing fine-tuning data largely depends on manual design, resulting in limited scale, insufficient diversity, and particularly weak standardization of tool parameters, which restricts the accuracy and generalization ability of models in real-world tasks; third, there is a lack of systematic evaluation benchmarks to comprehensively measure the performance of models in toolchain planning and parameter configuration. Summary of the Invention

[0007] Existing research suffers from limitations such as reliance on manual data design for fine-tuning, resulting in limited scale, insufficient diversity, and particularly weak standardization of tool parameters. These limitations restrict the accuracy and generalization ability of GIS toolchains in practical tasks. This invention proposes a parameter-level intelligent orchestration method and system for GIS toolchains based on large language model instruction fine-tuning.

[0008] In a first aspect, the present invention provides a parameter-level intelligent orchestration method for GIS toolchains based on fine-tuning of large language model instructions, comprising:

[0009] Geographic information tools are collected and preprocessed. Based on the preprocessed geographic information tools, a large language model is used to generate geospatial task instructions and corresponding geospatial task toolchains to construct instruction fine-tuning datasets and evaluation datasets.

[0010] Using an open-source large language model as the base model, the LoRA fine-tuning technique is used to fine-tune and train the base model based on the instruction fine-tuning dataset and the evaluation dataset.

[0011] Driven by the target geographic task instructions, the system automatically completes the arrangement and parameter settings of the geospatial task toolchain using the finely tuned basic model.

[0012] Furthermore, based on the preprocessed geographic information tools, a large language model is used to generate geospatial task instructions and corresponding geospatial task toolchains to construct instruction fine-tuning datasets and evaluation datasets, specifically including:

[0013] Using preprocessed geographic information tools and preset geospatial task templates as contextual prompts, a self-guided strategy guides a large language model to generate geospatial task instructions.

[0014] The geospatial task instructions are taken as input, and combined with the corresponding tool use chain prompts, the large language model is guided to generate a geospatial task toolchain through a self-guided strategy.

[0015] The geospatial task toolchain is paired with the corresponding geospatial task instructions to construct instruction fine-tuning datasets and evaluation datasets.

[0016] Furthermore, the preprocessing includes:

[0017] The geographic information tool is standardized based on a preset structured template, wherein the preset structured template is... ,in Indicates the first The name of the tool, Indicates the first Functional descriptions of each tool Indicates the first Parameter design template for each tool Indicates the first The input data requirements for each tool Indicates the first An example of how to use this tool.

[0018] Furthermore, the preset geospatial task templates include single-tool geospatial task templates, dual-tool geospatial task templates, and three-tool geospatial task templates.

[0019] Furthermore, the single-tool geospatial task template includes single-tool constraint rules, which are used to constrain the large language model to generate geospatial task instructions corresponding to only a single geographic information tool.

[0020] Furthermore, the dual-tool geospatial task template includes dual-tool constraint rules and logical verification rules; the dual-tool constraint rules are used to constrain the large language model to generate geospatial task instructions corresponding to the two geographic information tools; the logical verification rules are used to constrain the geospatial task instructions generated by the large language model to logically use the functions of the two geographic information tools.

[0021] The three-tool geospatial task template includes three-tool constraint rules and logical verification rules; the three-tool constraint rules are used to constrain the large language model to generate geospatial task instructions corresponding to the three geographic information tools; the logical verification rules are used to constrain the geospatial task instructions generated by the large language model to logically use the functions of the three geographic information tools.

[0022] Furthermore, pairing the geospatial task toolchain with the corresponding geospatial task instructions to construct instruction fine-tuning datasets and evaluation datasets also includes:

[0023] If the geospatial task toolchain and the corresponding geospatial task instruction have inconsistent geospatial tool calls or parameter settings, they will be removed.

[0024] Furthermore, using an open-source large language model as the base model, and fine-tuning the base model using LoRA fine-tuning technology based on the instruction fine-tuning dataset and the evaluation dataset, the method further includes:

[0025] When computing resources are less than a preset computing resource threshold, the weight matrix of the open-source large language model is decomposed into a low-rank matrix, and the decomposed matrix is ​​used as a new weight matrix to fine-tune the basic model.

[0026] Secondly, this invention provides a GIS toolchain parameter-level intelligent orchestration system based on large language model instruction fine-tuning, comprising:

[0027] The dataset construction unit is used to collect geographic information tools for preprocessing. Based on the preprocessed geographic information tools, the large language model is guided to generate geospatial task instructions and corresponding geospatial task toolchains to build instruction fine-tuning datasets and evaluation datasets.

[0028] An open-source large language model fine-tuning unit is used to fine-tune and train the base model using the LoRA fine-tuning technique based on the instruction fine-tuning dataset and the evaluation dataset, using an open-source large language model as the base model.

[0029] The task execution unit is used to automatically complete the arrangement and parameter setting of the geospatial task toolchain using the fine-tuned basic model, driven by the target geographic task instructions.

[0030] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in the first aspect.

[0031] The beneficial effects of this invention are as follows:

[0032] The method proposed in this invention introduces a self-guided generation strategy, combining domain knowledge to guide large models to automatically generate "instruction-toolchain" pairs covering diverse scenarios and complexities. This constructs an instruction fine-tuning dataset covering multiple geospatial tool combinations, effectively addressing the scarcity of high-quality training data in the geospatial domain. In particular, it strengthens the structured representation of parameter templates, data types, and value constraints, laying the foundation for the model to learn accurate tool invocation logic and parameter adaptation. Furthermore, an evaluation dataset is constructed, enabling standardized and quantifiable comprehensive evaluation of the model from two dimensions: toolchain accuracy (CTOA) and parameter accuracy (PA). This provides a complete data foundation and evaluation system for the training and performance evaluation of geospatial task agents. Attached Figure Description

[0033] Figure 1 A flowchart illustrating a parameter-level intelligent orchestration method for GIS toolchains based on large language model instruction fine-tuning, provided for an embodiment of the present invention;

[0034] Figure 2 is a structured tool description template provided in an embodiment of the present invention;

[0035] Figure 3 shows a prompt for chain generation using the parameterization tool provided in an embodiment of the present invention;

[0036] Figure 4 Generate prompt words for single-tool geospatial tasks provided in embodiments of the present invention;

[0037] Figure 5 This invention provides a dual-tool geospatial task prompt word generation method for embodiments of the invention.

[0038] Figure 6 Generate prompt words for geospatial tasks using the three tools provided in this embodiment of the invention;

[0039] Figure 7 shows a prompt for the use of the baseline model generation parameterization tool chain in the simulation environment provided by the embodiments of the present invention;

[0040] Figure 8 Example diagrams for calculating road slope using four tools provided in embodiments of the present invention;

[0041] Figure 9 Example diagrams for calculating road turning radii using four tools provided in embodiments of the present invention;

[0042] Figure 10 A structural diagram of a GIS toolchain parameter-level intelligent orchestration system based on large language model instruction fine-tuning is provided for an embodiment of the present invention.

[0043] Figure 11 This is a schematic diagram of the architecture of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] like Figure 1 As shown, this embodiment of the invention provides a parameter-level intelligent orchestration method for GIS toolchains based on large language model instruction fine-tuning, including:

[0046] S1: Collect and preprocess geographic information tools. Based on the preprocessed geographic information tools, guide the large language model to generate geospatial task instructions and corresponding geospatial task toolchains to construct instruction fine-tuning datasets and evaluation datasets. Preprocessing includes standardizing the geographic information tools based on a pre-defined structured template, where the pre-defined structured template is:

[0047]

[0048]

[0049] in Indicates the first The name of the tool, Indicates the first Functional descriptions of each tool Indicates the first Parameter design template for each tool Indicates the first The input data requirements for each tool Indicates the first Examples of tool usage This represents the complete tool suite. Table 1 shows the names, functions, and parameters of the three GIS tools. Taking the tool Points AlongGeometry as an example, its structured description template is shown in Figure 2-(a) and Figure 2-(b), which mainly includes algorithm functions, parameter descriptions (such as Input Layer, Distance, Start Offset, etc.), and data constraints. This structured description helps to reduce the complexity of the input text and alleviate the understanding burden of large language models.

[0050] Table 1. Examples of GeoITC tool descriptions

[0051]

[0052] Specifically, this step constructs two datasets: the GeoITC instruction fine-tuning dataset and the GeoITC-Eval evaluation dataset. These datasets are used for fine-tuning the general-purpose large model, improving its geospatial task solving capabilities. For geospatial task instructions, the collected GIS tools are first standardized and parameterized, forming a standardized format that includes tool functions, structured parameter templates, data constraints, and examples. Based on this, the high-performance large language model is guided to automatically generate diverse geospatial task instructions covering different levels of complexity in a simulated GIS execution environment, and simultaneously generate corresponding, parameterized, complete tool usage chain solutions.

[0053] S2: Use an open-source large language model as the base model, and fine-tune the base model using LoRA fine-tuning technology based on the instruction fine-tuning dataset and the evaluation dataset.

[0054] Specifically, using an open-source, general-domain large model as a foundation, the LoRA fine-tuning technique is employed for model training. This fine-tuning method freezes the original weights of the base model, only introducing trainable low-rank matrices in the sideways of key modules such as model attention for adaptive adjustments. This significantly reduces the number of training parameters, substantially lowering memory consumption and training time. By performing LoRA fine-tuning on the instruction fine-tuning dataset, the model can focus on learning the mapping relationship from natural language descriptions to structured toolchains and parameter configurations, ultimately obtaining a professional geospatial agent (GeoTP). This allows it to accurately grasp the planning and execution capabilities of complex geospatial workflows while maintaining the general language capabilities of the base model.

[0055] S3: Driven by the target geographic task instructions, it automatically completes the arrangement and parameter setting of the geospatial task toolchain using a fine-tuned base model.

[0056] Specifically, driven by natural language task instructions, the finely tuned large language model autonomously completes the geospatial toolchain orchestration and parameter configuration. When inputting instructions, the main objectives of the geospatial analysis task and the corresponding input data conditions must be clearly defined, and the data type and storage path must be clearly labeled to ensure that the task instructions are complete and semantically accurate. There is no need to specify the geospatial tools used, operation steps, parameter settings, or other technical descriptions.

[0057] The method provided in this invention, through fine-tuning with carefully designed parameterized instruction data, can effectively improve the ability of open-source models to plan and accurately configure parameters for complex geospatial workflows, providing a feasible solution for developing low-cost, privately deployable geospatial agents.

[0058] In some embodiments, a large language model is guided by preprocessed geographic information tools to generate geospatial task instructions and corresponding geospatial task toolchains to construct instruction fine-tuning datasets and evaluation datasets, specifically including:

[0059] S1.1: Using preprocessed geographic information tools and preset geospatial task templates as contextual prompts, the large language model is guided to generate geospatial task instructions through a self-guided strategy;

[0060] Specifically, to overcome the bottlenecks of scarce geospatial task instruction data and high manual design costs, this invention proposes an efficient dataset construction method combining a "self-guided generation strategy" and a "simulated environment." Based on the collected QGIS toolset and its structured description templates, a "self-guided strategy" guides a general-purpose large language model to generate geospatial task descriptions covering rich scenarios and with reasonable parameter settings. The large language model used in this step is Deepseek-V3.1. Given its powerful semantic understanding and planning capabilities, it is mainly relied upon to realize geospatial task generation based on the "self-guided strategy."

[0061] S1.2: Taking geospatial task instructions as input, and combining them with corresponding tool use chain prompts, the large language model is guided to generate geospatial task toolchains through a self-guided strategy.

[0062] Specifically, obtaining accurate "command-response" pairings is crucial for building a high-quality command tuning dataset. For geospatial tasks, the core lies in generating a complete solution (geospatial task toolchain) with correct tool sequences and accurate parameter configurations. To this end, this embodiment of the invention adopts an automatic generation strategy based on a large language model. Its core lies in guiding the model to generate parameterized and executable work chains through carefully designed prompt engineering, as shown in Figures 3-(a) and 3-(b). First, the model is provided with a structured description of the tools, enabling the model to accurately understand the tool call interface and laying the foundation for generating reasonable parameter values. Second, after understanding the task objective, the model needs to plan the tool execution sequence and complete the key parameter adaptation work. This requires the model to infer and fill in specific and reasonable parameter values ​​for each tool call based on the task context (e.g., "download SRTM 30-meter DEM of Yosemite National Park area, California"), data characteristics (e.g., output format and coordinate system), and data flow between steps. For example, the natural language description "Yosemite National Park area" is converted into boundary description parameters that meet the tool's input requirements: Define Extent to Download: "Longitude:-119. -119.1, Latitude: 37.5 to 38.3", and the Select DEM to Download: "SRTM 30m" and Output Raster: "yosemite_srtm_30m.tif" are explicitly specified according to the task.

[0063] S1.3: Pair the geospatial task toolchain with the corresponding geospatial task instructions to build instruction fine-tuning datasets and evaluation datasets.

[0064] Specifically, the instruction tuning dataset GeoITC is constructed by combining instruction tasks with a parameterized toolchain:

[0065]

[0066]

[0067] In this paper, "instruction" represents the instruction task, and "output" represents the geospatial task toolchain. To generate instruction fine-tuning data of varying complexity, this invention collected geospatial tasks and their corresponding tool usage chains at three levels, including single-tool, dual-tool, and triple-tool tasks. Specifically, for single-tool tasks, each tool was repeated 35 times to help the large model fully understand its function and usage (18 × 35 = 630). All dual-tool combinations were repeated 20 times, enabling the large model to master the methods of solving geospatial tasks using different tool combinations (18 × 17 ÷ 2 × 20 = 3060). After matching the framework and workflow, some unmatched cases were removed, resulting in 2740 dual-tool instructions. For triple-tool tasks, all triple-tool combinations were repeated 4 times, generating 18 × 17 × 16 ÷ 3 ÷ 2 × 4 = 3264. After removing unmatched cases, 2088 instructions were retained. Detailed statistics of all collected instruction tuning data are shown in Table 2, where the tool number in Table 2 indicates the number of tools required to solve the corresponding geospatial task.

[0068] Table 2 GeoITC Command Fine-tuning Data Statistics

[0069]

[0070] It is understandable that the core goal of instruction fine-tuning is to align the model's output with the dialogue style and knowledge system of human experts or specific domains, thereby significantly improving the model's ability to understand and execute complex tasks based on given instructions. For example, if the model is required to identify areas suitable for building parks in urban planning, ensuring that these areas are within 500 meters of schools and avoid water protection zones, the model not only needs to accurately resolve multiple geographical constraints in the task, but also needs to plan the correct spatial analysis steps—including correctly selecting tools and setting corresponding parameters (such as buffer distance in "buffer analysis," layer processing methods in "overlay analysis," etc.), and finally generating a complete executable toolchain. This ability to understand and configure tool functions and parameters is key to the model's ability to transform user natural language descriptions into professional and reliable operation sequences, and is also the foundation for deploying practical LLMs in professional fields. Currently, self-guided strategies are a widely adopted and efficient method for constructing instruction fine-tuning datasets. This strategy guides the LLM itself to participate in data generation, automatically constructing "instruction-solution" pairs, thereby reducing reliance on manual annotation. The process mainly includes two stages: instruction generation and solution construction, the latter focusing particularly on generating feasible operation sequences containing specific tool calls and parameter configurations. In this embodiment of the invention, this strategy is used to automatically generate supervised data covering diverse geospatial tasks and corresponding toolchains and their parameterized operations using high-performance LLM, providing structured and large-scale training resources for the subsequent accurate planning and execution of the model in complex geospatial scenarios.

[0071] In some embodiments, the preset geospatial task templates include single-tool geospatial task templates, dual-tool geospatial task templates, and three-tool geospatial task templates.

[0072] Specifically, considering the diversity and complexity of geospatial tasks, this invention designs three prompt templates for single-tool, dual-tool, and triple-tool tasks, respectively. In the single-tool task generation template ( Figure 4 The document explicitly requires that instructions generated rely solely on this tool and have complete parameters, including data paths and parameter values. To improve the model's understanding of the correspondence between tasks and parameters, manually constructed example instructions with parameter examples have been added to the prompts.

[0073] The single-tool geospatial task template includes scenario rationality verification rules, single-tool restriction rules, and built-in description length constraint rules. The scenario rationality verification rules are used to determine whether a geospatial task instruction constitutes a feasible and reasonable business scenario, and refuse to generate a geospatial task instruction if the determination is not correct. The single-tool restriction rules are used to constrain the large language model to generate geospatial task instructions corresponding to only a single geographic information tool. The built-in description length constraint rules are used to constrain the text length of the generated geospatial task instructions to be 1 to 2 natural language sentences.

[0074] In dual tools ( Figure 5 ) and three tools ( Figure 6 The task generation template maintains a similar structure, but emphasizes that tools should be combined into a coherent workflow, and parameters between steps should be connectable and transferable. Using this hierarchical prompting system, geospatial task data covering diverse scenarios and complexities was successfully collected, providing high-quality, parameter-rich, and logically rigorous supervision signals for model training.

[0075] The dual-tool geospatial task template includes scenario rationality verification rules, dual-tool restriction rules, logic verification rules, and built-in description length constraint rules. The scenario rationality verification rules are used to determine whether a geospatial task instruction constitutes a feasible and reasonable business scenario, and refuse to generate a geospatial task instruction if the determination is not correct. The dual-tool restriction rules are used to constrain the large language model to generate geospatial task instructions corresponding to two geographic information tools. The logic verification rules are used to constrain the geospatial task instructions generated by the large language model to logically use the functions of two geographic information tools. The built-in description length constraint rules are used to constrain the text length of the generated geospatial task instructions to be 1 to 4 natural language sentences.

[0076] The three-tool geospatial task template includes scenario rationality verification rules, three-tool constraint rules, logic verification rules, and built-in description length constraint rules. The scenario rationality verification rules are used to determine whether a geospatial task instruction constitutes a feasible and reasonable business scenario, and refuse to generate a geospatial task instruction if the determination is not correct. The three-tool constraint rules are used to constrain the large language model to generate geospatial task instructions corresponding to the three geographic information tools. The logic verification rules are used to constrain the geospatial task instructions generated by the large language model to logically use the functions of the three geographic information tools. The built-in description length constraint rules are used to constrain the text length of the generated geospatial task instructions to be 1 to 4 natural language sentences.

[0077] The method provided by the embodiments of the present invention not only alleviates the problem of shortage of geospatial task instruction data, but also ensures the accuracy and executability of generated tasks in tool calls and parameter configuration through structured parameter description and hierarchical prompt design, providing a reliable way for the system to improve the planning and parametric tool usage capabilities of large language models in the professional GIS field.

[0078] In some embodiments, pairing a geospatial mission toolchain with corresponding geospatial mission instructions to construct instruction fine-tuning datasets and evaluation datasets further includes:

[0079] If the geospatial task toolchain and the corresponding geospatial task command have inconsistent geospatial tool calls or parameter settings, they will be removed.

[0080] Specifically, since large language models cannot actually perform GIS operations, a structured output format is designed to simulate this process. Specifically, the model output must include two parts: first, an assessment of task feasibility; if solvable, a high-level solution framework must be generated, outputting a geospatial task toolchain containing detailed tool sequences, parameter configurations, and data flows. To improve the reliability of the generated results, this design incorporates a self-verification mechanism: the planned tool sequence and functional intent within the framework must be consistent with the specific calls and parameter settings logic of each step in the workflow. Any inconsistency indicates an illusion or logical error in the output and should be discarded. Research shows that this step-by-step reasoning approach of "plan first, then refine" effectively improves the accuracy and reliability of the model response, thereby systematically obtaining high-quality, verifiable geospatial task toolchains.

[0081] In some embodiments, GeoITC-Eval evaluation data was collected to assess the language model's ability to solve geospatial tasks. Similar to the instruction fine-tuning data, three levels of geospatial tasks were collected: for tasks requiring only a single tool, 270 tasks were collected (18 tools, each repeated 15 times); for tasks requiring two tools, the 18 available tools were paired to form 9 unique pairs, this process was repeated 10 times, resulting in 90 tasks; for tasks requiring three tools, the 18 available tools were grouped into 6 unique triplets, this process was repeated 10 times, resulting in 60 tasks. Unlike instruction fine-tuning data collected solely through Deepseek-V3.1, the evaluation data underwent professional verification. Specifically, the generated geospatial task details, framework, and each step of the workflow were reviewed by two GIS experts with master's degrees. This ensures that the evaluation data can be used to effectively evaluate agent performance. Table 3 lists the statistical results of all collected evaluation data.

[0082] Table 3. Statistical information for GeoITC-Eval, GeoITC-EvalProlong, and GeoITC-EvalExtend

[0083]

[0084] Furthermore, this embodiment of the invention also collects two extended datasets to evaluate the performance of the GeoTP model, fine-tuned using the method of this invention, in scenarios beyond the training data. Specifically, the first extended toolchain length dataset, GeoITC-EvalProlong, contains complex tasks requiring four to five geospatial tools to complete. Through GeoITC-EvalProlong, GeoTP can be tested on tasks more complex than the training data. In addition, three external geospatial tools are collected; the second extended tool dataset, GeoITC-EvalExtend, is constructed based on tools not used during the GeoITC training process. Table 4 provides information on these three tools.

[0085] Specifically, three levels of geospatial tasks were collected: for geospatial tasks requiring a single tool, 15 instructions were collected (each repeated 5 times by 3 tools); for geospatial tasks requiring two tools, 15 instructions were collected, each based on one external tool and one of 18 training tools; and 15 instructions requiring three tools were collected, again based on one external tool and two of the 18 training tools. In this way, GeoITC-EvalExtend can be used to test GeoTP's performance on unfamiliar geospatial tools, demonstrating its transferability.

[0086] Table 4 Information on three external geospatial tools

[0087]

[0088] In some embodiments, S2 further includes:

[0089] When computing resources are less than a preset computing resource threshold, the weight matrix of the open-source large language model is decomposed into a low-rank matrix, and the decomposed matrix is ​​used as a new weight matrix to fine-tune the basic model.

[0090] Specifically, considering the limited computing power of local devices, which cannot support full parameter fine-tuning of large language models, a parameter-efficient fine-tuning strategy must be adopted to fully utilize local computing resources—that is, optimizing the language model while minimizing computational and storage requirements during training. The LoRA method is chosen due to its excellent performance demonstrated in recent instruction fine-tuning research. Compared to full parameter fine-tuning of large language models, LoRA significantly reduces the number of trainable parameters. For the weight matrix of a large pre-trained language model... We employ the Low-Rank Adaptive (LoRA) method to freeze the matrix. And through low-rank decomposition To constrain its update volume ,in , Given two trainable parameters, and rank For linear layers Its corrected forward propagation is It is important to note that the LoRA method employed is scalable and can adapt to different system resources. When computational resources are significantly insufficient, the matrix can be reduced. The LoRA configuration is adjusted by the number of elements—these elements will be decomposed into low-rank matrices A and B. Alternatively, the matrix can be reduced... and rank .

[0091] This invention selects an open-source large language model as the base model and injects geospatial tool usage capabilities through the LoRa (Locally Automated Refinement) technique. Experiments show that GeoTP performs excellently on multiple tasks, significantly outperforming similar models not only in known tool tasks but also demonstrating strong adaptability in unseen tool generalization and ultra-long workflow planning.

[0092] To fully quantify the quality of the large language model generation workflow trained using the method of this invention, this embodiment verifies the quality through experiments.

[0093] 1. Evaluation Indicators

[0094] The evaluation is conducted from two dimensions: toolchain accuracy and parameter accuracy. Toolchain accuracy (ToolCorrectness and Order) assesses whether the model can plan the use of all necessary tools and whether the tool invocation order is reasonable. This dimension focuses on the overall structure and logical flow of the workflow, requiring that the generated workflow be completely consistent with the expected workflow in terms of the number of tools, identifiers, and order. Parameter accuracy (PA) verifies whether the parameters set by the model for the tools correctly match the tool definitions, including the existence of all necessary parameters, the conformity of parameter value data types, and the compliance of value ranges. In the quantitative evaluation, ToolCorrectness and Order Accuracy (TCOA) is used to calculate the proportion of toolchain samples with correct tool usage order out of the total sample.

[0095] Parameter accuracy is used to calculate the proportion of toolchain samples with completely reasonable parameter settings out of the total number of samples. The calculation formula is:

[0096]

[0097] In the formula,

[0098]

[0099]

[0100]

[0101]

[0102] in, Represents the total number of samples. Set a completely reasonable number of toolchain samples for the parameters. This represents the j-th toolchain generated, and for each tool in the toolchain, the parameter reasonableness standard. The following three conditions must be met simultaneously: The parameter names in the generated toolchain Parameter names in the toolset A perfect match; the data type of each parameter value conforms to the tool's definition requirements, and the parameter values ​​conform to the parameter value range specified by the tool.

[0103] The formula for calculating toolchain accuracy is:

[0104]

[0105] In the formula,

[0106]

[0107]

[0108]

[0109]

[0110] Among them, among them, Represents the total number of samples. The number of samples where the toolchain is completely correct; the standard for a completely correct toolchain. The following two conditions must be met: Generate toolchain Number of tools With expected toolchain The number of tools in each position is exactly the same, and the tool name at each position exactly matches the expected toolchain.

[0111] 2. Baseline Model

[0112] Choosing a suitable large language model (LLM) is crucial for elucidating the role of instruction fine-tuning methods in enhancing the capabilities of LLMs in geospatial tasks. Meta AI has released a mature, general-purpose large language model, Llama3-8B, with 8 billion parameters. This model boasts high inference efficiency and superior performance, making it suitable for real-time applications requiring rapid response and widely used in large language model augmentation research. Furthermore, to compare the performance of different large language models, this embodiment analyzes the performance of Deepseek-V3.1 and Qwen3-max on a collected evaluation dataset. This comparison helps determine whether, in geospatial tasks, the performance of a smaller-parameter open-source model, after LoRA fine-tuning using the collected instruction fine-tuning dataset, can surpass that of the most intelligent open-source large language models currently available.

[0113] Since the GeoITC instruction fine-tuning dataset collected during LoRA fine-tuning has already injected all available QGIS tool information into the model of this invention, only the geospatial task needs to be input into the model of this invention during the evaluation process. However, in order to make an effective and fair comparison, a strategy applicable to the general domain benchmark LLM is also designed. Specifically, for the Deepseek-V3.1 and Qwen3-max models, a system prompt is designed to provide them with the collected GIS tools and their corresponding descriptions, as shown in Figures 7-(a) and 7-(b).

[0114] 3 Results Analysis

[0115] 3.1 GeoITC-Eva

[0116] Table 5 shows the performance results of the baseline model and the present invention on the GeoITC-Eval evaluation dataset. To evaluate the model's capabilities under tasks of varying complexity, the number of tools required for each task type (1 to 3 tools) was calculated. Overall, the model trained with fine-tuning instructions using the method of the present invention significantly outperformed other baseline models in most cases, especially in toolchain accuracy (TCOA). Specifically, in single-tool tasks, the TCOA of the present invention reached 91.5%, far exceeding DeepSeek-v3.1 (62.6%) and qwen3-max (54.1%), indicating that it has approached expert level in basic tool invocation and sequence planning; its parameter accuracy (PA) was also as high as 90.0%, indicating that the model can accurately match parameter types and values, further verifying the effectiveness of the parameterized tool description in the instruction data. In multi-tool tasks (2-3 tools), the performance of all models decreased with increasing task complexity, but the present invention still maintained its leading position. Especially in toolchain planning (TCOA: 65.6% with 2 tools, 58.3% with 3 tools), it significantly outperforms the similarly sized LLaMA-8B and the general model qwen3-max, indicating that training with structured toolchain data enables the model to perform better multi-step inference and tool combination. It is worth noting that although DeepSeek-v3.1 has slightly higher parameter accuracy than this invention on some tasks (such as 3-tool PA), its toolchain accuracy remains consistently lower, further demonstrating that instruction fine-tuning data designed for geospatial tasks plays a crucial role in improving the ability to use specialized tools. In summary, the superior performance of this invention in both toolchain planning and parameter configuration verifies the effectiveness of the proposed parameterized, structured instruction fine-tuning dataset (GeoITC) in training professional domain models, significantly improving the tool invocation and workflow generation capabilities of open-source models in complex geospatial tasks.

[0117] Table 5. Performance results of the baseline model and the present invention on the GeoITC-Eval evaluation dataset.

[0118]

[0119] 3.2 GeoITC-EvalProlong

[0120] In more complex task scenarios (with 4 and 5 tools), this embodiment further evaluated the performance of the present invention and baseline models on the GeoITC-EvalProlong dataset. It should be noted that the present invention was only exposed to tasks using a maximum of 3 tools during the training phase; therefore, such tasks place higher demands on its generalization and combinatorial reasoning capabilities. As shown in Table 6, in tasks requiring 4 tools, the present invention achieved a toolchain accuracy (CTOA) of 66.7%, significantly higher than all other baseline models, with DeepSeek-v3.1, the second-best performer, only achieving 20.0%. More notably, the present invention performed exceptionally well in parameter accuracy (PA), reaching 76.7%, far exceeding other models. This indicates that in complex multi-step tasks, the present invention not only maintains a high degree of rationality in tool invocation order but also maintains strong consistency and accuracy in parameter configuration. In tasks with 5 tools, the overall performance of all models further declined, but the present invention still maintained a relative advantage (CTOA and PA both at 50.0%). It is worth noting that its CTOA still significantly outperforms DeepSeek-v3.1 (40.0%) and qwen3-max (45.0%), further demonstrating that through training with structured and parameterized toolchain data, the model can still exhibit a certain level of generalization and inference ability when faced with tasks exceeding training complexity. Overall, this invention maintains its lead in more complex tasks, especially in terms of stable parameter configuration, indicating that the instruction fine-tuning data designed in this invention not only helps the model master the use of basic tools but also enhances its robustness in planning and execution in long-chain, multi-parameter scenarios. This further verifies the effectiveness of the data construction method proposed in this invention in improving the model's ability to handle complex geospatial tasks.

[0121] Table 6. Experimental results of the baseline model and the present invention on the GeoITC-EvalProlong evaluation dataset.

[0122]

[0123] 3.3 GeoITC-EvalExtend

[0124] Finally, to examine the model's generalization and transfer capabilities when faced with external tools not encountered during training, the invention and baseline models were evaluated on the GeoITC-EvalExtend dataset, with results shown in Table 7. All tasks in this dataset were built based on tools not used during model training. Experimental results show that the invention comprehensively outperforms all baseline models in both toolchain accuracy (CTOA) and parameter accuracy (PA). In particular, in tasks requiring 2 and 3 tools, the invention's CTOA (both 53.3%) is significantly higher than the second-best performing qwen3-max (40.0%). This fully demonstrates that after training with the structured and parameterized instruction data designed in this invention, the model can not only correctly plan the execution order of unseen tools but also possess strong tool combination reasoning capabilities. More importantly, the invention also demonstrates robust performance in parameter accuracy (60.0% for single tool and 40.0% for three tools) and significantly outperforms other models in multi-tool tasks. This further demonstrates that the model learns not through rote memorization of specific tool parameters, but rather by acquiring the general ability to infer parameter types, constraints, and assign appropriate values ​​based on tool descriptions, thus enabling rapid adaptation to new tools. Compared to its performance on tasks within the training set distribution (Table 5), this invention maintains a similar performance level on external tool tasks, with a more pronounced advantage in multi-tool scenarios. This result indicates that the instruction fine-tuning data constructed in this invention can effectively help the model understand the general functional logic and parameter specifications of geospatial tools, thereby enabling it to possess true tool generalization ability, rather than being limited to combinations of known tools. This further verifies the effectiveness of the data construction method and the robustness of the model in this invention.

[0125] Table 7. Experimental results of the baseline model and the present invention on the GeoITC-EvalExtend evaluation dataset.

[0126]

[0127] 4 Case Studies

[0128] This embodiment demonstrates, through two real-world geospatial task examples, how the present invention understands user natural language commands and generates logically rigorous and parameter-complete executable workflows. These two examples fully verify that the model trained using the structured, parameterized instruction data (GeoITC) proposed in this invention can not only perform correct tool planning but also achieve accurate parameter inference and adaptation.

[0129] like Figure 8As shown, this workflow demonstrates a process from data acquisition, data preprocessing (coordinate transformation), feature extraction, turning radius, etc., which requires discretized road geometry features. It automatically fills in a reasonable sampling distance parameter (Distance=20) in the "point placement along the line" step, and ensures the accuracy of subsequent geometric calculations by inserting a "reprojection" step. This proves its logical planning ability and parameter context reasoning ability in multi-step tasks.

[0130] like Figure 9 As shown, this case involves three types of operations: vector data processing, raster value extraction, and attribute field calculation. The toolchain generated by this invention perfectly decomposes the instructions into four strictly sequential and coherent data flow steps. Crucially, in multiple stages, the model outputs highly specialized and reasonable parameters, such as the length of line segment divisions and the specific vertex indices to be extracted. This case further demonstrates that this invention is not simply a memorization tool, but rather reflects a profound understanding of the calculation principle of "road slope" and the ability to instantiate this understanding into concrete, executable workflows and parameters.

[0131] In summary, this invention proposes a framework for geospatial tool orchestration and parameter selection. It enhances the geospatial tool usage capabilities of an open-source large language model through instruction fine-tuning, enabling it to understand geospatial tasks described in natural language and autonomously generate executable tool usage chains. To this end, an innovative self-guiding strategy guided by seed tasks was designed, automatically constructing a large-scale, parameterized geospatial task instruction fine-tuning dataset, GeoITC, in a simulated environment. Then, LoRA technology was used for efficient parameter fine-tuning, resulting in the geospatial tool orchestration model of this invention. Through constructing a multi-dimensional evaluation benchmark, GeoITC-Eval, and conducting experiments, this invention outperforms the baseline model on most tasks, achieving a toolchain accuracy of 91.5% on single-tool tasks. For new tools and complex multi-tool (4-5 tools) tasks, this invention still maintains toolchain accuracy of over 53.3% and parameter accuracy of over 50.0%, respectively, with a parameter accuracy as high as 76.7% on 4-tool tasks. Experiments show that: 1) The dataset construction method proposed in this invention can effectively encapsulate the planning logic and parameter knowledge of geospatial workflows; 2) By constructing professional datasets and carrying out instruction fine-tuning, the open-source large model can master the ability to call professional domain tools and adapt parameters in a way that is no less than or even surpasses that of general large models.

[0132] like Figure 10 As shown, this embodiment of the invention also provides a GIS toolchain parameter-level intelligent orchestration system based on large language model instruction fine-tuning, including:

[0133] The dataset construction unit is used to collect geographic information tools for preprocessing. Based on the preprocessed geographic information tools, the large language model is guided to generate geospatial task instructions and corresponding geospatial task toolchains to build instruction fine-tuning datasets and evaluation datasets.

[0134] The open-source large language model fine-tuning unit is used to fine-tune and train the base model using an open-source large language model as the base model, based on the instruction fine-tuning dataset and the evaluation dataset, and employing the LoRA fine-tuning technique.

[0135] The task execution unit is used to automatically complete the orchestration and parameter setting of the geospatial task toolchain, driven by the target geographic task instructions and using a fine-tuned basic model.

[0136] It should be noted that the GIS toolchain parameter-level intelligent orchestration system provided in this embodiment of the invention is for implementing the methods described above. Its specific functions can be found in the above method embodiments, and will not be repeated here.

[0137] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include a processor 1101, a communications interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communications interface 1102, and memory 1103 communicate with each other via the communication bus 1104. The processor 1101 can call logical instructions in the memory 1103 to execute a GIS toolchain parameter-level intelligent orchestration method, which includes the methods described in embodiments S1 to S3 above.

[0138] Furthermore, when the logical instructions in the aforementioned memory 1103 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A parameter-level intelligent orchestration method for GIS toolchains based on fine-tuning of large language model instructions, characterized in that, include: Geographic information tools are collected and preprocessed. Based on the preprocessed geographic information tools, a large language model is used to generate geospatial task instructions and corresponding geospatial task toolchains to construct instruction fine-tuning datasets and evaluation datasets. Using an open-source large language model as the base model, the LoRA fine-tuning technique is used to fine-tune and train the base model based on the instruction fine-tuning dataset and the evaluation dataset. Driven by the target geographic task instructions, the system automatically completes the arrangement and parameter settings of the geospatial task toolchain using the finely tuned basic model.

2. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 1, characterized in that, Based on preprocessed geographic information tools, a large language model is guided to generate geospatial task instructions and corresponding geospatial task toolchains, in order to construct instruction fine-tuning datasets and evaluation datasets, specifically including: Using preprocessed geographic information tools and preset geospatial task templates as contextual prompts, a self-guided strategy guides a large language model to generate geospatial task instructions. The geospatial task instructions are taken as input, and combined with the corresponding tool use chain prompts, the large language model is guided to generate a geospatial task toolchain through a self-guided strategy. The geospatial task toolchain is paired with the corresponding geospatial task instructions to construct instruction fine-tuning datasets and evaluation datasets.

3. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 1 or 2, characterized in that, The preprocessing includes: The geographic information tool is standardized based on a preset structured template, wherein the preset structured template is... ,in Indicates the first The name of the tool, Indicates the first Functional descriptions of each tool Indicates the first Parameter design template for each tool Indicates the first The input data requirements for each tool Indicates the first An example of how to use this tool.

4. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 2, characterized in that, The preset geospatial task templates include single-tool geospatial task templates, dual-tool geospatial task templates, and three-tool geospatial task templates.

5. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 4, characterized in that, The single-tool geospatial task template includes single-tool constraint rules, which are used to constrain the large language model to generate geospatial task instructions corresponding to only a single geographic information tool.

6. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 4, characterized in that, The dual-tool geospatial task template includes dual-tool constraint rules and logical verification rules; the dual-tool constraint rules are used to constrain the large language model to generate geospatial task instructions corresponding to the two geographic information tools; the logical verification rules are used to constrain the geospatial task instructions generated by the large language model to logically use the functions of the two geographic information tools. The three-tool geospatial task template includes three-tool constraint rules and logical verification rules; the three-tool constraint rules are used to constrain the large language model to generate geospatial task instructions corresponding to the three geographic information tools; the logical verification rules are used to constrain the geospatial task instructions generated by the large language model to logically use the functions of the three geographic information tools.

7. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 2, characterized in that, Pairing the geospatial task toolchain with the corresponding geospatial task instructions to construct instruction fine-tuning datasets and evaluation datasets, the method also includes: If the geospatial task toolchain and the corresponding geospatial task instruction have inconsistent geospatial tool calls or parameter settings, they will be removed.

8. The GIS toolchain parameter-level intelligent orchestration method based on large language model instruction fine-tuning according to claim 1, characterized in that, Using an open-source large language model as the base model, and fine-tuning the base model using LoRA fine-tuning technology based on the instruction fine-tuning dataset and the evaluation dataset, the training also includes: When computing resources are less than a preset computing resource threshold, the weight matrix of the open-source large language model is decomposed into a low-rank matrix, and the decomposed matrix is ​​used as a new weight matrix to fine-tune the basic model.

9. A GIS toolchain parameter-level intelligent orchestration system based on large language model instruction fine-tuning, characterized in that, include: The dataset construction unit is used to collect geographic information tools for preprocessing. Based on the preprocessed geographic information tools, the large language model is guided to generate geospatial task instructions and corresponding geospatial task toolchains to build instruction fine-tuning datasets and evaluation datasets. An open-source large language model fine-tuning unit is used to fine-tune and train the base model using the LoRA fine-tuning technique based on the instruction fine-tuning dataset and the evaluation dataset, using an open-source large language model as the base model. The task execution unit is used to automatically complete the arrangement and parameter setting of the geospatial task toolchain using the fine-tuned basic model, driven by the target geographic task instructions.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1 to 8.

Images