Lightweight LLM-based automatic business process generation method, apparatus and device

Abstract

The invention relates to a lightweight LLM-based automatic business process generation method, apparatus and device. The method comprises the steps of establishing a business process generation data set for satellite imaging; the data set is adopted to carry out fine tuning on an automatic business process generation model based on the lightweight LLM; inputting a to-be-processed satellite imaging process description into the fine-tuned automatic business process generation model, and generating a satellite imaging business process in combination with a thinking chain and a ReAct prompt method; and performing process node definition, process node implementation and service process implementation on the satellite imaging service process, and generating a corresponding BPMN file in an XML format for visual display. According to the method, by combining the thinking chain and the ReAct prompt method, the problem that the current language model is insufficient in flow generation capacity is effectively solved, the adaptability and the processing capacity of the model are remarkably improved, and the generation capacity of a large language model for a complex business flow is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and equipment for automatically generating business processes based on lightweight LLM. Background Art

[0002] A business process is a series of activities completed by different people in sequence to achieve a specific value goal. It is a set of repeatable steps used by an enterprise to achieve its goals. A business process usually contains multiple workflow paradigms and related data, organizations, and systems. Using business processes can help organizations improve customer satisfaction and increase agility in responding to rapid market changes. At the same time, process-oriented organizations break down the barriers of structural departments and avoid functional silos. Good business processes are essential to achieving goals and improving business operations.

[0003] The first step of business process management usually requires domain professionals (usually system developers) to perform process modeling. Due to the lack of relevant experience and domain knowledge, describing the actual business process into a process model that can be recognized by the business process system is a rather tedious and complicated process. At present, in terms of business process generation, the vast majority of business processes applied in the real world are manually designed by human experts, which requires rich experience and domain knowledge. However, as the complexity and diversity of tasks increase, manually designing business processes will be a very complex and time-consuming task. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device and equipment for automatic business process generation based on lightweight LLM to address the above technical problems.

[0005] An automatic business process generation method based on lightweight LLM, the method comprising:

[0006] Set the data set structure and format for the business process generation task applicable to satellite imaging; each data entry in the data set includes: instructions, inputs, and outputs.

[0007] According to the dataset structure and format, a semi-automatic method of manual annotation and large-scale model generation is used to establish a business process for satellite imaging to generate datasets.

[0008] The automatic business process generation model based on the lightweight LLM is fine-tuned according to the data set; the automatic business process generation model based on the lightweight LLM includes a pre-trained lightweight LLM model.

[0009] A pipeline description of the satellite imagery to be processed is performed for preprocessing.

[0010] The obtained preprocessing results are input into the fine-tuned automatic business process generation model, and the satellite imaging business process for symbolic representation of business process modeling is generated by combining the thought chain and ReAct prompting methods.

[0011] Define process nodes according to the satellite imaging business process.

[0012] According to the prompt, the input and output data formats of the process nodes are determined through the associated structure, the node content is converted into a code function, and the data processing program is implemented using the associated compiler of the code function.

[0013] The various nodes of the satellite imaging business process are logically connected in series, thereby connecting the entire business process in series and generating the corresponding BPMN file in XML format for visual display.

[0014] In one embodiment, the process description of the satellite image to be processed is pre-processed, including:

[0015] Determine whether the process description of the satellite imaging to be processed is image information or text information.

[0016] If it is image information, image preprocessing related operations are performed, and the corresponding object recognition results are extracted and sent to the user.

[0017] If it is text information, the text information is filtered and relevant points are extracted, and the results are returned to the user.

[0018] In one embodiment, fine-tuning the automatic business process generation model based on the lightweight LLM according to the data set includes:

[0019] According to the dataset, a low-rank adaptive large language model is used to fine-tune the automatic business process generation model based on lightweight LLM.

[0020] In one embodiment, the pre-trained lightweight LLM model is but not limited to Internlm-2-7b-chat, Llama-2-7b-chat or CodeLlama-7b-Instruct.

[0021] In one embodiment, the satellite imaging business process is generated as it is generated with process node definition and process node implementation.

[0022] In one embodiment, according to the structure and format of the data set, a semi-automatic method of manual annotation and large-scale model generation is used to establish a business process for satellite imaging to generate a data set, including:

[0023] Several data samples are manually constructed according to the description of the satellite imaging task, and then the LLM model is prompted to generate a natural language description of the business process in XML format and receive it as input.

[0024] To assess the quality of the generated descriptions, the outputs were manually curated and reviewed, and the degree of match between the descriptions and the input business processes was visually assessed.

[0025] In one embodiment, the hints used in the reasoning process are created by combining an XML version of the business process provided in the output for each data entry in the dataset with the task description in the input.

[0026] An automatic business process generation device based on lightweight LLM, the device comprising:

[0027] The data set structure determination module is used to set the data set structure and format applicable to the business process generation task of satellite imaging; each data entry in the data set includes: instructions, input and output.

[0028] The dataset generation module is used to establish a business process for satellite imaging to generate datasets based on the dataset structure and format, using a semi-automatic method of manual annotation and large-scale model generation.

[0029] The automatic business process generation model fine-tuning module is used to fine-tune the automatic business process generation model based on the lightweight LLM according to the data set; the automatic business process generation model based on the lightweight LLM includes a pre-trained lightweight LLM model.

[0030] The satellite imaging business process generation module is used to preprocess the process description of the satellite imaging to be processed; the preprocessing results are input into the fine-tuned automatic business process generation model, and the satellite imaging business process for business process modeling symbolic representation is generated by combining the thinking chain and ReAct prompt method.

[0031] The satellite imaging business process execution module is used to define process nodes according to the satellite imaging business process; determine the input and output data formats of the process nodes through the associated structure according to the prompts, convert the node content into a code function, and use the associated compiler of the code function to implement the data processing program; logically connect the various nodes of the satellite imaging business process in series, thereby connecting the entire business process in series, and generate the corresponding BPMN file in XML format for visual display.

[0032] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements any step of the above-mentioned automatic business process generation method based on lightweight LLM when executing the computer program.

[0033] The above-mentioned method, device and equipment for automatic business process generation based on lightweight LLM, the method establishes a business process generation data set for satellite imaging; fine-tunes the automatic business process generation model based on lightweight LLM according to the data set; the automatic business process generation model based on lightweight LLM includes a pre-trained lightweight LLM model; the process description of the satellite imaging to be processed is input into the fine-tuned automatic business process generation model, and the satellite imaging business process is generated by combining the thinking chain and ReAct prompt method; the satellite imaging business process is defined by the process node, the process node is implemented, and the business process is implemented, and the corresponding XML format BPMN file is generated for visual display. By combining the thinking chain and the ReAct prompt method, this method effectively solves the problem of insufficient process generation capability faced by the current language model, significantly improves the adaptability and processing capability of the model, and effectively improves the generation capability of large language models for complex business processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of an automatic business process generation method based on lightweight LLM in one embodiment;

[0035] Figure 2 is an exemplary schematic diagram of an entry in a data set in one embodiment;

[0036] Figure 3 A flowchart of a drone patrol service in one embodiment;

[0037] Figure 4 It is a schematic diagram of the overall process framework of an automatic business process generation method based on lightweight LLM in one embodiment;

[0038] Figure 5 A schematic diagram of a BPGen Agent generation process in another embodiment;

[0039] Figure 6 A schematic diagram of a case analysis of a business process generation process in another embodiment;

[0040] Figure 7 A schematic diagram of task completion rate analysis in another embodiment;

[0041] Figure 8 A schematic diagram of node correctness analysis in another embodiment;

[0042] Fig. 9 It is a structural block diagram of an automatic business process generation device based on lightweight LLM in one embodiment;

[0043] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] Large language model, English full name: large language models, abbreviated as LLM.

[0046] Chain of Thought, full English name: Chain of Thought, abbreviated as CoT.

[0047] ReAct, full English name: Reason and Act.

[0048] The present application proposes an automatic business process generation method based on lightweight LLM, referred to as BPGen method.

[0049] In one embodiment, Figure 1 As shown, a method for automatic business process generation based on lightweight LLM is provided, and the method includes the following steps:

[0050] Step 100: Set a data set structure and format suitable for a satellite imaging business process generation task; each data entry in the data set includes: instructions, inputs, and outputs.

[0051] Specifically, since this method fully exploits the power of LLM, it is necessary to incorporate structural and natural language elements into each data entry. This method proposes a new dataset schema customized for business process generation. Each entry in the dataset consists of three parts: "instructions", "inputs", and "outputs". While the original work generally suggests that the "input" is optional, in our dataset we combine the XML version of the BP provided in the "output" with the task description in the "input" to create hints that are used later in the reasoning process. Figure 2 An example of an entry in a dataset is shown, showing the business process of drone patrol. The first box is the instruction, which is common to all samples in the dataset and is fixed. The second box is the input part, which provides a natural language description of the business process. The third box is the output element, which represents the business process in XML format.

[0052] Since business processes are usually described through graph-based representations, e.g. Figure 3The business process diagram in the above example is shown, so it is difficult for LLM to understand and generate new business processes. Therefore, we believe that many LLMs have learned a large amount of code data during the pre-training process and have strong code generation capabilities. Therefore, when describing the function represented by each process node, some specific codes are used to represent it, thereby promoting the understanding of the actual application of business process logic. For example, complex business processes often involve complex control logic, including conditional branches, loops, or the execution of sub-workflows.

[0053] Traditional methods usually involve human developers designing graph-based representations to describe control flows (e.g., Zapier; n8n), but their ability to express complex workflows is limited and not suitable for LLM understanding and generation. Since programming languages ​​support complex control logic and, more importantly, LLM is learned during the pre-training phase, we use programming languages ​​to describe business nodes and related logic in workflows. As a high-level programming language, it can provide rich primitives and features to describe complex control logic, thereby providing stronger expressive power. As shown in Table 1, Figure 3 The code implementation of the two nodes "check target" and "Return information" in the business process.

[0054] Table 1 Code implementation of business process nodes

[0055]

[0056] Step 102: According to the data set structure and format, a semi-automatic method of manual annotation and large-scale model generation is used to establish a business process for satellite imaging to generate a data set.

[0057] Step 104: fine-tune the automatic business process generation model based on the lightweight LLM according to the data set; the automatic business process generation model based on the lightweight LLM includes a pre-trained lightweight LLM model.

[0058] Specifically, generating business processes from process descriptions is a fundamental goal in the field of workflow application automation and execution. Its main goal is to transform a given process description into a business process that can be executed by a process engine. The structure of the process is represented by the Business Process Modeling Notation (BPMN), which enables the process engine to autonomously perform complex tasks according to the logical flow indicated by the business process. The business process generation task is defined as the function of mapping the process description to the corresponding business process. The ability requirements related to business process generation include task understanding, process planning, tool calling ability, and code generation ability, which are summarized in Table 2.

[0059] Table 2 Capabilities related to BP generation

[0060]

[0061]

[0062] Task understanding is very important for business process generation because it is the first step to construct the process description into a business process. Without a clear understanding of the business description, it is impossible to generate a business process that successfully completes the specified task and meets user needs. Especially when the task description of the business process is provided in an unconstrained natural language form, it is particularly challenging to extract relevant information (such as process goals, service functions, and related constraints) for generating the business process.

[0063] Process planning: decompose complex processes into multiple sub-processes and organize them in BP generation. Process planning requires converting abstract semantic instructions into detailed operable sub-processes and organizing them in the right order. Specifically, it decomposes high-level goals into simpler manageable sub-processes and organizes their control flows and information flows into business process structures.

[0064] Tool invocation: Features related to tool invocation are crucial in complex business process generation tasks because they allow leveraging the capabilities and relevant expertise of existing tools. Considering the complexity of business process generation tasks and the problems of large models such as hallucinations, using specialized tools can directly obtain the results of related tasks and effectively improve the quality of business process generation.

[0065] Code-related: Code-related features play a key role in the business process generation task, ensuring that the generated process is both executable and desirable to the user. The ability to generate code helps translate abstract instructions into actionable code that can be easily executed by agents or systems, and the code can be interpreted in natural language for developers, making the generated business process more transparent and explainable.

[0066] There are still many challenges in applying LLM to the task of business process generation. First, in terms of training the BPGen model, the ability to generate business processes involves key considerations of the interaction between the model performance and its constituent elements.

[0067] The automatic business process generation model based on lightweight LLM includes three projects: basic lightweight LLM model, training dataset and training pipeline. The overall process framework is as follows: Figure 4As shown. This method selected Internlm-2-7b-chat, Llama-2-7b-chat, and CodeLlama-7b-Instruct, and used smaller models with only 7B parameters, which makes this method replicable on consumer-grade hardware without having to rely on larger and more complex models provided through APIs. All of these models are based on the Transformer architecture adopted in the literature 1 (Vaswani A. Attention is all you need [J]. Advances in Neural Information Processing Systems, 2017.). Additional fine-tuning of these models enables customization to meet specific needs, such as domain specialization, in our case the generation of business processes.

[0068] Step 106: Preprocess the process description of the satellite image to be processed.

[0069] Step 108: Input the obtained preprocessing results into the fine-tuned automatic business process generation model, combine the thought chain and ReAct prompt method, and generate a satellite imaging business process for business process modeling symbolic representation.

[0070] Specifically, we combine MindChain and ReAct's BP generation framework BPGen to alleviate the common problems of hallucination and data bias in the phantom LLM generation process.

[0071] LLM is pre-trained on a large number of general datasets and can be very effective for common tasks with zero or few shots. However, the results on specialized data are often unsatisfactory. In the presence of specialized data, LLM can be fine-tuned to further improve the model performance. For fine-tuning on our business process generation task, the data corpus specifically contains task-specific input-output pairs. The data collection covers a variety of formats, each designed to meet different desired capabilities. The goal of this fine-tuning phase is to train LLM to interpret user prompts and generate appropriate outputs across a variety of formats such as code, XML, or restricted languages.

[0072] Step 110: Define process nodes according to the satellite imaging business process.

[0073] Step 112: Determine the input and output data formats of the process node through the associated structure according to the prompt, convert the node content into a code function, and use the associated compiler of the code function to implement the data processing program.

[0074] Step 114: logically connect the nodes of the satellite imaging business process in series, thereby connecting the entire business process in series, and generate a corresponding BPMN file in XML format for visual display.

[0075] Through the above steps, we have a business process LLM that can generate a proprietary domain BPMN format. At present, although some existing large language models have certain capabilities to call and execute tools when generating business processes, many of them are limited to single-step calling tools or relatively simple tasks. For complex tasks that require multi-step calls or contain logical judgments, the performance will be greatly reduced. The main reason is that in the process of generating and executing complex BPs, the input text of the model is too long, which distracts attention and causes the task that should be executed to be forgotten or confused in the middle. In response to the above problems, this paper proposes a BPGen Agent method based on Chain of Ideas (CoT) and ReAct (Reason+Action) (I verify it by executing and generating at the same time during generation, so as to generate a process that can actually run.), which can help the model track the currently generated business process and complete the entire business process through LLM tool calls. Specifically, the three-step method of "process node definition-process node implementation-business process implementation" is adopted.

[0076] like Figure 5 As shown, the process node definition determines which service node is selected to be added to the business process.

[0077] The implementation of a process node determines its input / output data format through an associated structure, converts the node content into a code function, and implements the data processing program using the associated compiler of the code function.

[0078] The realization of a business process refers to the logical connection of the various nodes of the appeal in series, thereby connecting the entire business process in series to achieve the goal that needs to be achieved. This approach helps the model better focus on the task at hand without being distracted by the length of the text caused by complex tasks. Moreover, a complex business process task can be decomposed into a logical sequence of multiple meta-tasks that can be completed by a single tool.

[0079] Figure 6Business process visualization process generated for BPGen Agent. For this case task, BPGen Agent generated a workflow with 6 nodes, including a logical control node and 5 service task nodes. For example, when a user asks "Please process the data of this reconnaissance terminal, it is necessary to know the information related to the intelligence data and the reconnaissance target information obtained. BPGen Agent first defines the relevant task nodes of the business process. For example, it determines that the input is satellite detection data and image data, and then pre-processes the data and analyzes the target area. In this way, it is possible to determine which task nodes can meet user needs and can be further added to the business process.

[0080] The BPGen Agent framework has broad application prospects. It can realize the automatic generation of command control chains and the automated execution of tasks in industrial production, and plays an important role in Robotic Process Automation (RPA). If the Multi-Agent collaborative scheduling technology is further implemented at the multi-agent layer, the framework can be further expanded to form an autonomous, orderly, and decentralized application structure with other task environments, tool sets, and basic models. With the traceability of blockchain technology, the correctness and standardization of the business process execution process can also be guaranteed. In addition, when migrating to new task scenarios or adding new tools, new task knowledge or tool knowledge can be added to the BPGenAgent framework with the help of retrieval generation technology, thereby improving the model's generalization ability for new tasks or new tools.

[0081] In the above-mentioned automatic business process generation method based on lightweight LLM, the method establishes a business process generation data set for satellite imaging; fine-tunes the automatic business process generation model based on lightweight LLM according to the data set; the automatic business process generation model based on lightweight LLM includes a pre-trained lightweight LLM model; the process description of the satellite imaging to be processed is input into the fine-tuned automatic business process generation model, and the satellite imaging business process is generated by combining the CoT and ReAct prompting methods; the satellite imaging business process is defined by process nodes, implemented by process nodes, and implemented by business processes, and the corresponding BPMN file in XML format is generated for visual display. By combining the CoT and ReAct prompting methods, this method effectively solves the problem of insufficient process generation capabilities faced by current language models, significantly improves the adaptability and processing capabilities of the model, and effectively improves the generation capabilities of large language models for complex business processes.

[0082] In one embodiment, step 106 includes: determining whether the process description of the satellite imaging to be processed is image information or text information; if it is image information, performing image preprocessing related operations, extracting corresponding object recognition results and sending them to the user; if it is text information, filtering the text information and extracting relevant points, and returning the results to the user.

[0083] Specifically, to perform different actions based on whether the data information belongs to image information or text information, it is necessary to understand the meaning of each business process. Therefore, BPGen Agent defines a judgment code implementation, the purpose of which is to determine whether the business process is image processing or text processing; to determine what operation to perform next. If the business process belongs to an image source, BPGen Agent performs operations related to image preprocessing, extracts the corresponding object recognition results and sends them to the user. If the business process is a text source, it is necessary to filter the text information and extract relevant points, and return the results to the user. The implementation of specific process nodes is expressed in code. After defining and arranging the relevant task nodes, BPGenAgent will generate the corresponding BPMN file in XML format for visual display.

[0084] In one of the embodiments, the automatic business process generation model based on the lightweight LLM is fine-tuned according to the data set, including: fine-tuning the automatic business process generation model based on the lightweight LLM using a low-rank adaptive large language model according to the data set.

[0085] Specifically, to perform the fine-tuning task, we further fine-tune the model using the constructed dataset. LLM fine-tuning is a computationally intensive task, so we adopt the parameter efficient fine-tuning (PEFT) method, specifically the low-rank adaptive large language model (LoRA). The core idea of ​​LoRA is to freeze the pre-trained model weights, add a bypass to the original model framework, and use matrix operations to first increase and then reduce the dimension of the parameters. During training, only the parameters of the original model need to be fixed to train the reduced and increased dimensional matrices. Since the training rank decomposition matrix is ​​injected into each layer of the Transformer architecture, the number of trainable parameters for downstream tasks is greatly reduced.

[0086] Specifically, for LoRA parameters, LoRA restricts the weight matrix updates in the model to a low-rank subspace through reparameterization techniques. This means that most of the pre-trained weights of the model remain unchanged, and only a few parameters determined by the low-rank matrix are updated.

[0087] The lightweight LLM-based automatic business process generation model used in this example was fine-tuned by LoRA, and the fine-tuning process was completed on an NVIDIA A100 with a total video memory of 40GB. All basic hyperparameters have been used except for the batch size of 512 and the micro-batch size of 4. This example uses a learning rate of 2e-4 and a validation once, setting the size to 10%. Considering that the constructed dataset consists of less than a thousand samples, the number of epochs was increased to 3 to obtain satisfactory results.

[0088] In one embodiment, the pre-trained lightweight LLM model is but not limited to Internlm-2-7b-chat, Llama-2-7b-chat or CodeLlama-7b-Instruct.

[0089] In one embodiment, the satellite imaging business process is generated as it is generated with process node definition and process node implementation.

[0090] In one embodiment, step 102 includes: manually constructing a number of data samples based on the description of the satellite imaging mission, and then prompting the LLM model to generate a natural language description of the business process in XML format and receiving it as input; in order to evaluate the quality of the generated description, the output results are manually selected and reviewed, and the degree of match between the description and the input business process is intuitively evaluated.

[0091] Specifically, recent studies have highlighted the superior capabilities of modern LLMs, especially GPT, in multiple domains such as question answering, text generation, and code generation. GPT models are suitable for text generation tasks and can be used to generate descriptions of the provided input. With this in mind, this method uses a semi-automatic approach of manual annotation and large-scale model generation for dataset construction. Specifically, several data samples are first manually constructed based on the descriptions of satellite imaging tasks, and then the model is prompted to generate a natural language description of the business process in XML format, which is received as input. The one-shot prompt for generating the provided business process description using the GPT4 model is the same as discussed in [9]. We also give additional information, including the maximum word count and the required compatibility with BPMN20., as well as an overall summary that the description must represent a task that can be clearly described in natural language. The method finally leverages the language capabilities inherent in the GPT4 model to create a description of the business process. To evaluate the quality of the generated descriptions, we manually curate and audit the output and then visually evaluate how well the description matches the input BP. Our evaluation is positive, so we finally form the CBPG dataset with a total of about 600 entries.

[0092] In one embodiment, the hints used in the reasoning process are created by combining an XML version of the business process provided in the output for each data entry in the dataset with the task description in the input.

[0093] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0094] In a verification example, in order to evaluate and compare the performance of three basic models (i.e., Internlm2-chat, LlamaChat, and CodeLlama) with parameter number 7b and the fine-tuned version of the present application, we tested the test set using different metrics.

[0095] (1) Task definition and evaluation indicators

[0096] All three models were tested with the input being a description of a satellite imaging mission. The expected output is a business process in XML format and compliant with the BPMN standard, providing an easily understandable visual graphical representation.

[0097] The tasks used include multiple tasks such as target area imaging, terrain imaging, and meteorological environment imaging, which are all included in the dataset.

[0098] Since the satellite imaging process dataset is mainly used to describe the model's ability to represent complex processes, this embodiment proposes to comprehensively measure the performance of the model through four indicators: BPMN generation time, BPMN generation syntax correctness, BPMN generation task completion status, and BPMN generation node correctness. The specific indicators are as follows:

[0099] <s:1> Time: Evaluating the temporal performance of the model is crucial when considering the entire process from LLM to actual business process execution.

[0100] Syntax correctness (SC): Use the Camunda platform to evaluate syntax accuracy and check the syntax correctness of the XML schema and the entire business process.

[0101] Task completion rate (TCR): evaluates whether the generated process can complete the task without considering the syntactic correctness. This indicator reflects the model's ability to generate executable business processes and summarize and refine the processes.

[0102] <s:1> Node correctness (NC): The accuracy of the process nodes is calculated by comparing the business process node set output by the model with the manually labeled business process node set.

[0103] (2) Experimental results

[0104] Table 3 shows the execution time comparison when performing Generate Business Processes on the models. From Table 3, it can be seen that Internlm2 is generally faster than CodeLlama when using the one-shot prompt method, but with a smaller standard deviation, considering the models that were not fine-tuned. In particular, LlamaChat does not terminate during the generation process, so the average execution time cannot be calculated. For the models fine-tuned with the dataset, it can be seen that the one-shot prompt method is slightly slower than the zero-shot prompt method because they have to handle more complex inputs. The mean and standard deviation of the zero-shot prompt and the one-shot prompt after fine-tuning for Internlm2 and LlamaChat are similar. However, the one-shot prompt generation time of CodeLlama after fine-tuning increases significantly, and the standard deviation also increases. After fine-tuning, LlamaChat obtains a significant performance improvement because it can complete the task of generating business processes, but it is still slower in most cases.

[0105] Table 3 Time comparison

[0106]

[0107] In terms of syntactic correctness, all outputs of the three basic models were evaluated using Camunda, and the corresponding fine-tuned versions were evaluated using our dataset, including zero-shot and one-shot prompts, and the evaluation results are shown in Table 4. As can be seen from Table 4, due to the special format of BPMN and the corresponding structural method, the basic model with zero-shot prompts obtained very limited results, and only on some simple business processes, syntactically correct business processes can be generated in very limited cases. In contrast, by adding examples (i.e., one-shots) to the prompts, the performance is significantly improved, and Internlm2 and CodeLlama have significantly stronger generation capabilities than LlamaChat. It is worth noting that the fine-tuned models also achieved similar results and significantly improved the performance of syntactic correctness. In this case, even with zero-shot prompts, all models can generate syntactically correct business processes for most tasks. However, it can be seen that the one-shot prompt method after fine-tuning does not improve the accuracy as much as before, probably because the examples have already entered the parameters of LLM through fine-tuning.

[0108] Table 4 Syntax Correctness Comparison

[0109] Internlm2 CodeLlama LlamaChat Base model, zero-shot 12.31％ 0％ 0％ –Base model, single sample 74.23％ 75.31％ 35.14％ Fine-tuning the model, zero-shot 84.41％ 83.18％ 69.26％ Fine-tuning model, single sample 86.16％ 84.92％ 68.28％

[0110] Figure 7 and Figure 8 The task completion rate and node correctness rate of the generated business process are analyzed respectively. For a given complex task, the model plans the task and obtains the percentage of samples representing the correct execution process in XML format, which reflects the model's ability to generate task requirements and summarize tasks. The node correctness rate is a comparison between the generated business process nodes and the actual correct nodes, indicating the correctness of the selection of the specified business process nodes and the logical rationality of the node execution. Since the basic model under zero-sample prompts performs poorly under the first two indicators, the basic model under zero-sample prompts is not considered in the evaluation of the subsequent two indicators.

[0111] from Figure 7 and Figure 8It can be seen that the model has significant advantages in indicators after fine-tuning the dataset. For example, the task completion rate and node correctness rate of the fine-tuned Internlm2 under one prompt are 83.66% and 84.73% respectively, indicating that the model has strong business process understanding and planning capabilities. However, the researchers also found an interesting observation. When the LlamaChat and CodeLlama fine-tuning models use one prompt, the performance of the model decreases slightly during fine-tuning. This may be due to the overfitting characteristics given by the model prompt. This once again shows that the design of the prompt plays an important role in guiding LLM to produce the correct solution results.

[0112] Then, this application evaluates the performance of the fine-tuned model in combination with the proposed BPGen Agent method. The input is the original complex task instruction. The model needs to generate a business process based on the instruction and then give the final answer. This application evaluates the performance of the model by syntactic correctness (SC), task completion rate (TCR) and node correctness (NC). Tables 5 and 6 show the performance comparison results of the BPGen method proposed in this application and other methods on the CBPG dataset. The difference between the two tables is whether there is a single sample prompt. The ReAct technology is based on a large language model and directly generates and executes tasks through Reason+Action. Considering that the performance of LlamaChat is relatively poor in both basic performance and fine-tuning performance, the subsequent evaluation mainly uses the fine-tuned Internlm2 and CodeLlama models for testing. The models in Tables 5 and 6 also default to the tuned models.

[0113] From the results in the two tables, we can see that the large language model using generative methods has a significant impact on performance. With the use of model generative methods, the results of most evaluation indicators have also improved. This is because with the assistance of generative methods, the model usually has stronger understanding and reasoning capabilities, can better capture the complexity of input data, and thus improve the accuracy of task generation and execution.

[0114] Table 5 Performance comparison of related methods on the CBPG dataset (–zero sample)

[0115]

[0116] Table 6 Performance comparison of related methods on CBPG dataset (single sample)

[0117]

[0118] The BPGen method has significant advantages in fine-tuning models and adopting ReAct on all model sizes and all indicators. For example, in the CodeLlama model under one-time prompting, the TCR and NC of the BPGen method are 93.23% and 92.46% respectively, indicating that the model has strong business process generation capabilities and subsequent execution node capabilities. As the main performance evaluation indicator, the performance of the BPGen method is improved by about 32% over the other two methods, and the growth rate is higher than other indicators, which fully proves the effectiveness of the BPGen method.

[0119] It can also be seen from the table that Internlm2 is better than CodeLlama in understanding the syntax of BPMN, but worse than CodeLlama in generating it. It can also be seen from the table that under the zero-shot prompt, the node correctness of the CodeLlama model in the BPGen method reached 92.18%, but the task success rate was slightly lower because the model could not configure the correct parameters for the nodes in the business process. The business process completion rate was significantly improved, which in turn led to a better task generation rate.

[0120] In one embodiment, Fig. 9 As shown, an automatic business process generation device based on lightweight LLM is provided, including: a data set structure determination module, a B module and a C module, wherein:

[0121] The data set structure determination module is used to set the data set structure and format applicable to the business process generation task of satellite imaging; each data entry in the data set includes: instructions, inputs and outputs;

[0122] The dataset generation module is used to establish a business process for satellite imaging to generate datasets based on the dataset structure and format, using a semi-automatic method of manual annotation and large-scale model generation;

[0123] An automatic business process generation model fine-tuning module, used for fine-tuning the automatic business process generation model based on the lightweight LLM according to the data set; the automatic business process generation model based on the lightweight LLM includes a pre-trained lightweight LLM model;

[0124] The satellite imaging business process generation module is used to preprocess the process description of the satellite imaging to be processed; the preprocessing results are input into the fine-tuned automatic business process generation model, and the satellite imaging business process for business process modeling symbolic representation is generated by combining CoT and ReAct prompting methods;

[0125] The satellite imaging business process execution module is used to define process nodes according to the satellite imaging business process; determine the input and output data formats of the process nodes through the associated structure according to the prompts, convert the node content into a code function, and use the associated compiler of the code function to implement the data processing program; logically connect the various nodes of the satellite imaging business process in series, thereby connecting the entire business process in series, and generate the corresponding BPMN file in XML format for visual display.

[0126] In one of the embodiments, the satellite imaging business process generation module is also used to determine whether the process description of the satellite imaging to be processed is image information or text information; if it is image information, perform image preprocessing related operations, extract corresponding object recognition results and send them to the user; if it is text information, filter the text information and extract relevant points, and return the results to the user.

[0127] In one of the embodiments, the automatic business process generation model based on the lightweight LLM is fine-tuned according to the data set, including: fine-tuning the automatic business process generation model based on the lightweight LLM using a low-rank adaptive large language model according to the data set.

[0128] In one of the embodiments, the pre-trained lightweight LLM model in the automatic business process generation model fine-tuning module is but not limited to Internlm-2-7b-chat, Llama-2-7b-chat or CodeLlama-7b-Instruct.

[0129] In one embodiment, the generation of the satellite imaging business process in the satellite imaging business process execution module is performed in parallel with the process node definition and process node implementation.

[0130] In one of the embodiments, the dataset generation module is also used to manually construct several data samples based on the description of the satellite imaging task, and then prompt the LLM model to generate a natural language description of the business process in XML format and receive it as input; in order to evaluate the quality of the generated description, the output results are manually selected and reviewed, and the degree of match between the description and the input business process is intuitively evaluated.

[0131] In one embodiment, the hints used in the reasoning process in the satellite imaging business process execution module are created by combining the XML version of the business process provided in the output for each data entry in the dataset with the task description in the input.

[0132] For the specific limitations of the automatic business process generation device based on lightweight LLM, please refer to the limitations of the automatic business process generation method based on lightweight LLM above, which will not be repeated here. Each module in the above-mentioned automatic business process generation device based on lightweight LLM can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0133] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an automatic business process generation method based on a lightweight LLM is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0134] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for automatic business process generation based on lightweight LLM, characterized in that: The method comprises: Setting a data set structure and format for a business process generation task applicable to satellite imaging; each data entry in the data set includes: instructions, inputs, and outputs; According to the structure and format of the data set, a semi-automatic method of manual annotation and large-scale model generation is used to establish a business process for satellite imaging to generate data sets; Fine-tuning the automatic business process generation model based on the lightweight LLM according to the data set; the automatic business process generation model based on the lightweight LLM includes a pre-trained lightweight LLM model; Preprocessing of the satellite imagery pipeline description to be processed; The obtained preprocessing results are input into the fine-tuned automatic business process generation model, and the satellite imaging business process for business process modeling symbolic representation is generated by combining the thought chain and ReAct prompting method; Defining process nodes according to the satellite imaging business process; Determine the input and output data formats of the process nodes through the associated structure according to the prompts, convert the node content into code functions, and use the associated compiler of the code function to implement the data processing program; The various nodes of the satellite imaging business process are logically connected in series, thereby connecting the entire business process in series and generating the corresponding BPMN file in XML format for visual display.

2. The method for automatic business process generation based on lightweight LLM according to claim 1, characterized in that: Preprocessing of the satellite imagery pipeline description to be processed includes: Determine whether the process description of the satellite imaging to be processed is image information or text information; If it is image information, then perform image preprocessing related operations, extract the corresponding object recognition results and send them to the user; If it is text information, the text information is filtered and relevant points are extracted, and the results are returned to the user.

3. The method for automatic business process generation based on lightweight LLM according to claim 1, characterized in that: Fine-tune the automatic business process generation model based on lightweight LLM according to the dataset, including: According to the dataset, a low-rank adaptive large language model is used to fine-tune the automatic business process generation model based on the lightweight LLM.

4. The method for automatic business process generation based on lightweight LLM according to claim 1, characterized in that: The pre-trained lightweight LLM models are but not limited to Internlm-2-7b-chat, Llama-2-7b-chat or CodeLlama-7b-Instruct.

5. The method for automatic business process generation based on lightweight LLM according to claim 1, characterized in that: The satellite imaging business process is generated as it is being generated with process node definition and process node implementation.

6. The method for automatically generating a business process based on lightweight LLM according to claim 1, characterized in that: According to the structure and format of the dataset, a semi-automatic method of manual annotation and large-scale model generation is used to establish a business process for satellite imaging to generate datasets, including: Manually construct several data samples based on the description of the satellite imaging task, then prompt the LLM model to generate a natural language description of the business process in XML format and receive it as input; To assess the quality of the generated descriptions, the outputs were manually curated and reviewed, visually assessing how well the descriptions matched the input business processes.

7. The method for automatic business process generation based on lightweight LLM according to claim 1, characterized in that: The hints used in the reasoning process are created by combining the XML version of the business process provided in the output for each data entry in the dataset with the task description in the input.

8. An automatic business process generation device based on lightweight LLM, characterized in that: The device comprises: A data set structure determination module is used to set a data set structure and format suitable for a business process generation task for satellite imaging; each data entry in the data set includes: instructions, inputs and outputs; A data set generation module, for establishing a business process for satellite imaging to generate a data set based on the data set structure and format, using a semi-automatic method of manual annotation and large-scale model generation; An automatic business process generation model fine-tuning module, used to fine-tune the automatic business process generation model based on the lightweight LLM according to the data set; the automatic business process generation model based on the lightweight LLM includes a pre-trained lightweight LLM model; The satellite imaging business process generation module is used to pre-process the process description of the satellite imaging to be processed; the pre-processing results are input into the fine-tuned automatic business process generation model, and the satellite imaging business process for business process modeling symbolic representation is generated by combining the thought chain and ReAct prompt method; The satellite imaging business process execution module is used to define process nodes according to the satellite imaging business process; determine the input and output data formats of the process nodes through the associated structure according to the prompt, convert the node content into a code function, and use the associated compiler of the code function to implement the data processing program; logically connect the various nodes of the satellite imaging business process in series, thereby connecting the entire business process in series, and generate a corresponding BPMN file in XML format for visual display.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method for automatic business process generation based on lightweight LLM according to any one of claims 1 to 7 is implemented.

Images