Large language model driving system

Abstract

The invention discloses a large language model driving system which comprises a process module, an execution module, a process management module and a state observation module. The process module is used for setting an operation process according to an actual service and sending the operation process to the execution module; the execution module is used for receiving the operation process and driving a large language model to execute according to the operation process; the process management module is used for managing the execution process of the large language model; and the state observation module is used for monitoring the execution state of the large language model in the execution process in real time. The system comprises a module for determining the execution of the large language model according to the operation process, and also comprises a process management module for managing the execution process and a state observation module for monitoring the execution state in real time, so that driving and management can be realized at the same time, and the whole body and the whole execution process of the large language model are grasped; and the efficiency, quality and normal operation are ensured.

Description

Technical Field

[0001] This application relates to the technical field of large language model-driven technologies, and specifically relates to a large language model-driven system. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models have become a hot topic in current research and applications, and higher requirements have been put forward for their driving frameworks or driving systems. In terms of model deployment and optimization, such as how to efficiently run large language models on different platforms to achieve high-efficiency inference, and in addition, many studies have also been carried out in aspects such as model interpretability and security.

[0003] However, little attention has been paid to aspects such as the overall operation, operation process, and operation status of large language models, and these aspects will also affect the efficiency and quality of large language models. Therefore, a framework or system that can drive and control the overall operation process of large language models is needed to ensure the normal operation of large language models. Summary of the Invention

[0004] In view of this, this application provides a large language model-driven system, which is a framework or system for driving and controlling the overall operation process of large language models, so as to ensure the normal operation of large language models.

[0005] To achieve the above objectives, the following solutions are proposed:

[0006] In a first aspect, a large language model-driven system includes: a process module, an execution module, a process management module, and a status observation module;

[0007] The process module is used to set an operation process according to the actual business and send the operation process to the execution module;

[0008] The execution module is used to receive the operation process and drive the large language model to execute according to the operation process;

[0009] The process management module is used to manage the execution process of the large language model;

[0010] The status observation module is used to monitor the execution status during the execution process of the large language model in real time.

[0011] Preferably, the process module includes multiple process sub-modules, and each process sub-module corresponds to a business scenario;

[0012] The process of the process module setting the operation process according to the actual business includes:

[0013] Determine the business scenario of the actual business;

[0014] Match the business scenario with each of the process sub - modules to determine the process sub - module corresponding to the business scenario of the actual business, so that the process sub - module depicts the operation process of the actual business.

[0015] Preferably, the process of the process sub - module depicting the operation process of the actual business includes:

[0016] Analyze the actual business to extract each business keyword corresponding to the actual business;

[0017] Package the business scenario of the actual business with each of the business keywords to obtain business content;

[0018] Based on the business content, obtain the response data corresponding to the actual business;

[0019] Depict the operation process of the actual business according to the response data.

[0020] Preferably, the process of the execution module driving the large - language model to execute according to the operation process includes:

[0021] Determine the node types of each node in the process module;

[0022] Determine the start node and the end node according to the node types;

[0023] Based on the start node and the end node, correspond the operation process with each of the other nodes in the process module except the start node and the end node to determine the node order;

[0024] After the correspondence, drive the large - language model to execute the operation process according to the node order.

[0025] Preferably, the start node includes an input terminal, an output terminal, and a start flag unit;

[0026] The end node includes an input terminal, an output terminal, and an end flag unit;

[0027] Except for the start node and the end node, each of the other nodes in the process module includes an input terminal, an output terminal, and a processing unit;

[0028] The processing unit includes a question - answering configuration toolkit, a large - language model library, a length configuration toolkit for input information and output information, and an interactive text configuration toolkit.

[0029] Preferably, the process management module manages the execution process of the large - language model, including:

[0030] Monitor the execution process of the large language model in real time and determine each execution result during the execution process;

[0031] Extract each piece of data of the large language model during the execution process in real time;

[0032] Store each of the execution results and each piece of data.

[0033] Preferably, the process of the state observation module monitoring the execution state during the execution process of the large language model includes:

[0034] Monitor in real time whether any one or more of the preset states are reached during the execution process of the large language model;

[0035] If it is monitored that any one or more of the preset states are reached during the execution process of the large language model, determine the state data corresponding to each reached state;

[0036] Judge in real time whether the end state among the preset states is reached;

[0037] If so, determine the execution result;

[0038] Perform state feedback based on the execution result and the state data.

[0039] Preferably, the performing state feedback based on the execution result and the state data includes:

[0040] Judge whether the execution result is successful;

[0041] If the execution result is successful, feedback the execution result and the state data;

[0042] If the execution result is not successful, trace the problem according to the execution result to determine the error message, and feedback the error message and the state data.

[0043] Preferably, the preset state is a waiting state, a ready state, an execution state, a data filling state or an end state.

[0044] Preferably, the large language model driving model further includes a memory management module;

[0045] The memory management module is used to manage the input information, output information of the large language model, and the roles corresponding to the input information and output information respectively.

[0046] As can be seen from the above technical solutions, the present application provides a large language model-driven system, including a process module, an execution module, a process management module, and a status observation module; the process module is used to set an operation process according to the actual business and send the operation process to the execution module; the execution module is used to receive the operation process and drive the large language model to execute according to the operation process; the process management module is used to manage the execution process of the large language model; the status observation module is used to monitor the execution status during the execution process of the large language model in real time. In the present application, by providing a system that includes a module for setting an operation process according to the actual business, a module for determining the execution of the large language model according to the operation process, a process management module for managing the execution process, and a status observation module for monitoring the execution status in real time, it is possible to achieve both driving and management, grasp the overall and entire execution process of the large language model, and ensure its efficiency, quality, and normal operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative efforts.

[0048] Figure 1 FIG. is an optional system block diagram of a large language model-driven system provided by an embodiment of the present application;

[0049] Figure 2 FIG. is a schematic structural diagram of a process module provided by an embodiment of the present application;

[0050] Figure 3 FIG. is a schematic diagram of a framework construction process provided by an embodiment of the present application;

[0051] Figure 4 FIG. is a schematic diagram of an execution process provided by an embodiment of the present application;

[0052] Figure 5 FIG. is an optional system block diagram of another large language model-driven system provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0054] With the rapid development of artificial intelligence technology, large language models have become a hot topic in current research and applications, and higher requirements have been put forward for their driving frameworks or systems. In terms of model deployment and optimization, such as how to efficiently run large language models on different platforms to achieve high-efficiency inference, and a lot of research has also been invested in aspects such as model interpretability and security.

[0055] However, little attention has been paid to aspects such as the overall operation, operation process, and operation status of large language models, and these aspects will also affect the efficiency and quality of large language models. Therefore, a framework or system that can drive and control the overall operation process of large language models is needed to ensure the normal operation of large language models.

[0056] Therefore, an embodiment of the present invention provides a large language model driving system. An optional system block diagram is as Figure 1 shown. The system includes a process module, an execution module, a process management module, and a status observation module.

[0057] Among them, the process module (Flow) is used to set the operation process according to the actual business and send the operation process to the execution module. The actual business can be set according to the application scenario of the system or can be set independently by the user. This embodiment does not limit this. For example, asking for prices, finding locations, summarizing articles, etc. Each actual business will correspond to a business process, that is, an operation process, which can be understood as the process or logic for the large language model to perform reasoning, retrieval, and analysis based on the input question or text content. After setting is completed, the operation process needs to be sent to the execution module.

[0058] The execution module (Executor) is used to receive the operation process sent by the process module and drive the large language model to execute according to the operation process. It can be understood that the execution module plays a driving and controlling role, and it can enable the large language model to run according to the operation process.

[0059] The process management module (Flow Context) is used to manage the execution process of the large language model. When the large language model executes according to the operation process, there will be multiple processes or steps. Then each process or step is managed by this process management module, such as storing step status, whether the step is correct, step results, problems that occur in the step, etc., to achieve comprehensive management, so that the entire process of the large language model can be mastered.

[0060] The state observation module (Observer) is used to monitor the execution state during the execution process of the large language model in real time. This is because different states will occur in different steps of the execution process. For example, a certain step may be executed successfully or failed, and the current state of the large language model, such as normal state or incorrect state. In this way, it can be timely discovered whether there are problems with the large language model, whether it needs to be paused, whether it can continue to run, whether it needs to be corrected, etc., to ensure the efficiency, quality, and normal operation of the large language model.

[0061] In terms of connection, the process module is connected to the execution module, and both the process management module and the state observation module are connected to the execution module.

[0062] As can be seen from the above technical solution, the present application provides a large language model-driven system, including a process module, an execution module, a process management module, and a state observation module; the process module is used to set the operation process according to the actual business and send the operation process to the execution module; the execution module is used to receive the operation process and drive the large language model to execute according to the operation process; the process management module is used to manage the execution process of the large language model; the state observation module is used to monitor the execution state during the execution process of the large language model in real time. In the present application, by providing a system that includes a module for setting the operation process according to the actual business, a module for determining the execution of the large language model according to the operation process, a process management module for managing the execution process, and a state observation module for real-time monitoring of the execution state, it is possible to achieve both driving and management at the same time, grasp the overall and entire execution process of the large language model, and ensure its efficiency, quality, and normal operation.

[0063] In the system provided by the embodiment of the present invention, the process module includes a plurality of process sub-modules, and each process sub-module corresponds to a business scenario;

[0064] The process of the process module setting the operation process according to the actual business includes:

[0065] Determine the business scenario of the actual business;

[0066] Match the business scenario with each of the process sub-modules to determine the process sub-module corresponding to the business scenario of the actual business, so that the process sub-module can depict the operation process of the actual business.

[0067] Specifically, the multiple process sub-modules (Flow-1, Flow-2, Flow-…, Flow-n) in the process module are combined in sequence, such as Figure 2As shown, it has penetrability, which can meet the interaction between multiple business scenarios, as well as the selection and determination of different business scenarios. Each set process sub-module corresponds to a business scenario, which can meet the needs of different actual businesses in different business scenarios. Compared with the non-layered setting in the prior art, through the combination of multiple layers of process sub-modules in this application, collaboration can be achieved, providing more possibilities, being more refined and specific, and making the execution results of the large language model more accurate.

[0068] Among them, the process of the process sub-module depicting the operation process of the actual business may include the following steps:

[0069] S1: Analyze the actual business to extract each business keyword corresponding to the actual business.

[0070] S2: Package the business scenario of the actual business and each business keyword to obtain business content.

[0071] S3: Based on the business content, obtain the response data corresponding to the actual business.

[0072] S4: Depict the operation process of the actual business according to the response data.

[0073] Specifically, the above steps S1 to S3 are the depicted operation processes. The data in each step of the operation processes corresponding to different business scenarios are different. That is to say, different business scenarios have different business keywords, so different business contents will be obtained, and thus different response data will be obtained. The response data is the output and feedback of the large language model according to the actual business or the input information.

[0074] In an example, the actual business is that the user inquires about the sea freight price from Guangzhou to Shanghai. Then the actual business is the sea freight inquiry business. Extract each business keyword corresponding to this sea freight inquiry business, including Guangzhou, Shanghai, sea freight, sea freight direction, and sea freight price. Then package the sea freight inquiry business and these business keywords to obtain a piece of business content. Then, according to this business content, the database API for the sea freight inquiry business can be called through the interface of the large language model, so as to extract the response data, that is, the specific price. Then the large language model outputs and feedbacks this price.

[0075] The following embodiments will detail the process of the execution module in this application driving the large language model to execute according to the operation process.

[0076] Determine the node types of each node in the process module;

[0077] Determine the start node and the end node according to the node types;

[0078] Based on the start node and the end node, the running process is corresponded to each of the other nodes in the process module except the start node and the end node to determine the node order.

[0079] After the correspondence, the large language model is driven to execute the running process in the node order.

[0080] Specifically, the process module includes multiple nodes, and each node corresponds to a node type. These nodes include a start node, an end node, and some other nodes. The start node refers to the node where the running process starts, and the end node refers to the node where the running process ends. Then, for a running process, there will also be other nodes before the start node and the end node. In this application, each step of the running process is sorted out in the process module in the form of nodes, which can improve the execution success rate of the large language model and prevent the large language model from deviating from the originally set "track". In the execution module, a multi-thread pool can be adopted to increase the possibility of parallelism of the running process and reduce the waiting time during the execution and reasoning processes.

[0081] In an existing LangChain execution framework, based on the ReAct idea (combining reasoning and acting), an intelligent agent Agent is constructed to enable a large language model (LLM) to understand intentions, plan processes, and execute logic. The construction process is as Figure 3 shown, including:

[0082] Define tools: Inherit the BaseTool class in the langchain execution framework, implement the _run or _arun method, and define the name, description, and parameter specifications of the tool; Define the global Prompt: Define the only Prompt template in the current execution Agent session; Define the global LLM: Define and implement an LLM docking; Assemble the Agent: Assemble the tool, Prompt, and LLM into an Agent; Execute the Agent: Fill in the user's question and execute the tool according to the LLM reasoning to obtain the result.

[0083] The execution process can be as Figure 4 shown, including:

[0084] LLM reasoning: Input the user's question through the assembled Agent, and obtain the result through LLM reasoning or select the tool to be executed; Call the tool: Determine whether to call the tool after LLM reasoning. If the tool is not called, return the result given by the LLM; Execute the tool: Understand the user's intention through the LLM, extract the parameters required by the tool, and execute the logic carried by the tool.

[0085] In the existing technology, only a global Prompt is used to implement, and it cannot implement the actual operations in multiple different business scenarios. Therefore, the only execution process is not applicable, and the obtained execution results are inaccurate, which easily causes serious hallucinations in the large language model. In contrast, in this application, by setting the operation process according to the actual business and corresponding it to the nodes, it is possible to resist the hallucinations of the large language model during the execution process as much as possible, enhance memory, and improve accuracy.

[0086] In addition, the termination link in the above-mentioned existing technology ends rather hastily, while this application can achieve a highly flexible function and does not limit the data format of the execution results.

[0087] Furthermore, each node in the process module includes an input terminal and an output terminal. At the same time, different nodes also have other units. The start node also includes a start flag unit, the end node also includes an end flag unit, and other nodes except the start node and the end node also include a processing unit. The processing unit includes a question-and-answer configuration toolkit, a large language model library, a length configuration toolkit for input information and output information, and an interactive text configuration toolkit.

[0088] The question-and-answer configuration toolkit refers to the configurable tool range, corresponding to the business scenario, such as a sea freight inquiry tool, a weather query tool, etc., which can be regarded as the tools used to call the API. The large language model library contains various large language models, and the large language model driven by the execution module can be randomly selected from this large language model library, so as to cover various business scenarios and various languages. The input information and output information refer to the information input into the large language model and the information output by the large language model, and the length configuration tool refers to the tool for configuring the information structure length. The interactive text configuration toolkit can be used to configure the Prompt.

[0089] Therefore, the fine ranking among multiple process sub-modules in the process module can more precisely define the range for the large language model to select tools in the above-mentioned various toolkits, freely select tools, and reduce the probability of incorrect selection.

[0090] The following describes the process management module for managing the execution process of the large language model:

[0091] Monitor the execution process of the large language model in real time to determine each execution result in the execution process;

[0092] Extract each piece of data in the execution process of the large language model in real time;

[0093] Store each of the execution results and each piece of data.

[0094] Specifically, during the execution of the large language model, it will be divided into some steps or sub - processes. Each step or sub - process will correspond to an execution result, and some data will also be generated during the entire execution process. Then the process management module needs to store these execution results and data.

[0095] In this application, the status observation module monitors the execution status during the execution of the large language model in real - time. Specifically, it may include:

[0096] Monitor in real - time whether any one or more of the preset statuses are reached during the execution of the large language model;

[0097] If it is monitored that any one or more of the preset statuses are reached during the execution of the large language model, determine the status data corresponding to each reached status;

[0098] Judge in real - time whether the end status among the preset statuses is reached;

[0099] If so, determine the execution result;

[0100] Perform status feedback based on the execution result and the status data.

[0101] Among them, the step of performing status feedback based on the execution result and the status data includes:

[0102] Judge whether the execution result is successful;

[0103] If the execution result is successful, feedback the execution result and the status data;

[0104] If the execution result is not successful, trace the problem according to the execution result to determine the error message, and feedback the error message and the status data. Subsequently, the large language model driving system of this application or the large language model therein can be adjusted and corrected according to the error message.

[0105] Specifically, the preset statuses are waiting status, ready status, execution status, data filling status, or end status. The status during the execution of the large language model can reflect the execution situation and degree of the large language model. Therefore, this system sets up a status observation module for real - time monitoring.

[0106] During the real - time monitoring process, the status observation module supports the input of external information, which can be achieved based on the interaction mode, visualization mode, and can also provide a debugging function to achieve the ability of dynamic information injection. For example, in the information supplemented by the user in human - machine interaction, following the above - mentioned example of the shipping inquiry service, if the information input by the user lacks the destination, it needs to be fed back to the user for supplementation, and then continue to monitor after supplementation.

[0107] Furthermore, the system may further include a Memory management module, which is used to manage the input information, output information of the large language model, and the roles corresponding to the input information and output information respectively. The structural block diagram of the entire system may be as shown in Figure 5 the figure.

[0108] In addition, in each embodiment of the present disclosure, each functional module may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a live broadcast device, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present disclosure.

[0109] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0110] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0111] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A large language model driven system, characterized in that: include: Process module, execution module, process management module, status observation module; The process module is used to set the operation process according to the actual business and send the operation process to the execution module; The execution module is used to receive the operation process and drive the large language model to execute according to the operation process; The process management module is used to manage the execution process of the large language model; The state observation module is used to monitor the execution state of the large language model in real time during its execution.

2. The system according to claim 1, characterized in that The process module includes multiple process sub-modules, each of which corresponds to a business scenario; The process module sets and runs the process according to the actual business, including: Determine the business scenario of the actual business; The business scenario is matched with each of the process sub-modules to determine a process sub-module corresponding to the business scenario of the actual business, so that the process sub-module can describe the operation process of the actual business.

3. The system according to claim 2, characterized in that The process submodule describes the process of the actual business operation process, including: Parsing the actual business to extract various business keywords corresponding to the actual business; Packing the business scenario of the actual business with each of the business keywords to obtain business content; Based on the business content, obtaining response data corresponding to the actual business; According to the response data, the operation process of the actual business is described.

4. The system according to claim 1, characterized in that The process in which the execution module drives the large language model to execute according to the operation process includes: Determine the node type of each node in the process module; Determine a start node and an end node according to the node type; Based on the start node and the end node, the running process is matched with each other node in the process module except the start node and the end node to determine the node order; Correspondingly, the large language model is driven to execute the operation process according to the node sequence.

5. The system according to claim 4, characterized in that The start node includes an input terminal, an output terminal and a start flag unit; The end node includes an input terminal, an output terminal and an end mark unit; Except for the start node and the end node, each other node in the process module includes an input terminal, an output terminal and a processing unit; The processing unit includes a question and answer configuration toolkit, a large language model library, an input information and output information length configuration toolkit, and an interactive text configuration toolkit.

6. The system according to claim 1, characterized in that The process management module manages the execution process of the large language model, including: Monitoring the execution process of the large language model in real time, and determining various execution results in the execution process; Extracting various pieces of data during the execution of the large language model in real time; Each of the execution results and each of the data are stored.

7. The system according to claim 1, characterized in that The process of the state observation module monitoring the execution state of the large language model in real time during execution includes: Real-time monitoring of whether the large language model reaches any one or more of the preset states during execution; If it is monitored that any one or more of the preset states are reached during the execution of the large language model, then state data corresponding to each reached state is determined; Determine in real time whether the end state of each preset state is reached; If so, determine the execution result; Status feedback is performed based on the execution result and status data.

8. The system according to claim 7, characterized in that The performing status feedback based on the execution result and the status data includes: Determine whether the execution result is successful; If the execution result is successful, the execution result and the status data are fed back; If the execution result is unsuccessful, the problem is traced according to the execution result to determine the error information, and the error information is fed back with the status data.

9. The system according to claim 7, characterized in that The preset state is a waiting state, a ready state, an execution state, a data filling state or an end state.

10. The system according to any one of claims 1 to 9, characterized in that: The large language model driven model also includes a memory management module; The memory management module is used to manage the input information, output information and the roles corresponding to the input information and output information of the large language model.

Images