Intelligent agent construction method, system, medium and device based on large language model

Abstract

The present application provides a method, system, medium and device for constructing an intelligent agent based on a large language model, and relates to the technical field of constructing a decision-making intelligent agent. The method selects a suitable large language model and establishes a series of prompt engineering texts for the large language model, so as to utilize the large language model and the prompt engineering to generate target code on demand, and the code can be run to call the class library functions of the network model library, the reinforcement learning algorithm and the simulation training environment, and finally realize the auxiliary construction and training of the decision-making intelligent agent, and solve the problems in the existing decision-making intelligent agent construction and training process.

Description

Technical Field

[0001] The present application relates to the technical field of decision-making intelligent agent construction, and in particular to an intelligent agent construction method, system, medium and device based on a large language model. Background Art

[0002] The difficulties in building and training decision-making agents mainly include: first, it is difficult to clarify business problems and accurately refine inputs and outputs, which highly relies on the designer's deep understanding of the business and requires continuous correction and optimization; second, designing a neural network structure and reinforcement learning algorithm that adapts to the business is extremely challenging, and it is necessary to consider multiple factors such as business needs and data volume. During network construction and training, it may be necessary to repeatedly tune the network to avoid overfitting and underfitting, which depends largely on the experience and technology of the personnel; third, the application specifications of the simulation training system that usually supports the training of decision-making agents are complex, and the development code calls the simulation training system to support the training of decision-making agents, which requires high professionalism of designers and is relatively inefficient. Summary of the invention

[0003] The present application provides a method, system, medium and device for constructing an intelligent agent based on a large language model. By selecting a suitable large language model, a series of prompt engineering texts are established for the large language model, thereby supporting efficient and accurate decision-making intelligent agent construction and training on demand.

[0004] The first aspect of the embodiment of the present application provides a method for constructing an intelligent agent based on a large language model, the method comprising:

[0005] Determine the target large language model;

[0006] Based on the description documents and example codes corresponding to the network model library, a first prompt engineering text corresponding to the network model is established;

[0007] Based on the description documents and example codes corresponding to the reinforcement learning algorithm library, establish the second prompt engineering text corresponding to the reinforcement learning algorithm;

[0008] Based on the interface documentation, class library functions and code examples of the simulation training environment, a third prompt engineering text corresponding to the simulation training environment is established;

[0009] Based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text, a complete demonstration prompt engineering text for different business needs is constructed;

[0010] A target prompt engineering text corresponding to the target business is generated based on user input information, and the target prompt engineering text is input into the target large language model, so that the target large language model performs demand analysis on the target prompt engineering text to obtain demand labels, and based on the demand labels and the complete demonstration prompt engineering text, a target code corresponding to the target business is generated, and the target code is used to construct a corresponding decision-making agent model and train the decision-making agent model.

[0011] Optionally, the description document and example code corresponding to the network model library include description information and code information corresponding to a variety of network models;

[0012] Based on the description documents and example codes corresponding to the network model library, the first prompt engineering text corresponding to the network model is established, including:

[0013] Based on the description information corresponding to various network models, establish network model classification and feature documents, network model architecture description documents, network model parameter setting documents and network model function definition documents; wherein the network model classification and feature documents are used to describe the name, characteristics, applicable scenarios, advantages and limitations of each network model; the network model architecture description document is used to describe the hierarchical structure, neuron connection mode and data flow of each network model; the network model parameter setting document is used to describe the parameter name, possible value range, recommended value and impact on model performance of each network model; the network model function definition document is used to describe the function name, functional description, input parameters, output results and algorithm logic of each network model;

[0014] Based on the code information corresponding to the various network models, a network model function call example document is established; the network model function call example document is used to describe the example usage scenarios, call codes and code comments of each network model;

[0015] Based on the network model classification and characteristic document, the network model architecture description document, the network model parameter setting document, the network model function definition document and the network model function call example document, a first prompt engineering text corresponding to the network model is generated.

[0016] Optionally, the description document and example code corresponding to the reinforcement learning algorithm library include description information and code information corresponding to a plurality of reinforcement learning algorithms;

[0017] Based on the description documents and example codes corresponding to the reinforcement learning algorithm library, establish the second prompt engineering text corresponding to the reinforcement learning algorithm, including:

[0018] Based on the description information corresponding to multiple reinforcement learning algorithms, establish reinforcement learning algorithm library function description documents and model training hyperparameter tuning experience documents; wherein, the reinforcement learning algorithm library function description documents are used to describe the algorithm name, principle, advantages and disadvantages, applicable scenarios, performance comparison, function name, function description, input parameters, output results and algorithm logic of each reinforcement learning algorithm; the model training hyperparameter tuning experience documents are used to describe the hyperparameters, tuning methods, tuning experience summary and application cases of the network model using each reinforcement learning algorithm;

[0019] Based on the code information corresponding to the various reinforcement learning algorithms, a reinforcement learning algorithm class library function call example document is established; the reinforcement learning algorithm class library function call example document is used to explain the example usage scenarios, call codes and code comments of each reinforcement learning algorithm;

[0020] Based on the reinforcement learning algorithm class library function description document, the model training hyperparameter tuning experience document and the reinforcement learning algorithm class library function call example document, a second prompt engineering text corresponding to the reinforcement learning algorithm is generated.

[0021] Optionally, based on the interface documentation, class library functions and code examples of the simulation training environment, a third prompt engineering text corresponding to the simulation training environment is established, including:

[0022] Based on the interface document of the simulation training environment, establish a simulation training environment introduction document, a simulation training environment configuration document, a simulation training environment interface detailed description document and a simulation training environment data format specification document; wherein the simulation training environment introduction document is used to describe the functional modules, characteristics, scope of application and advantages of each simulation environment; the simulation training environment configuration document is used to describe the hardware requirements, software dependencies, environment variable settings and configuration steps of each simulation environment; the simulation training environment interface detailed description document is used to describe the interface name, function, input parameter format and requirements, output data format and meaning of each simulation environment; the simulation training environment data format specification document is used to describe the input data format, output data format, data type and data dimension of each simulation environment;

[0023] Based on the class library functions of the simulation training environment, a classification summary document of the class library functions of the simulation training environment and an internal implementation analysis document of the class library functions of the simulation training environment are established; wherein the classification summary document of the class library functions of the simulation training environment is used to describe the function categories, function names and function functions of each simulation environment; the internal implementation analysis document of the class library functions of the simulation training environment is used to describe the function codes, key code comments, flow charts and function execution processes of each simulation environment;

[0024] Based on the code examples of the simulation training environment, a simulation training environment application example document is established, wherein the simulation training environment application example document is used to describe the example scenarios, implementation steps, code examples and application results of each simulation environment application;

[0025] Based on the simulation training environment introduction document, the simulation training environment configuration document, the simulation training environment interface detailed description document, the simulation training environment data format specification document, the simulation training environment class library function classification summary document, the simulation training environment class library function internal implementation analysis document and the simulation training environment application example document, a third prompt engineering text corresponding to the simulation training environment is generated.

[0026] Optionally, based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text, a complete demonstration prompt engineering text for different business needs is constructed, including:

[0027] Check and supplement the business requirements of different decision-making agents, the first prompt engineering text, the second prompt engineering text, and the third prompt engineering text to obtain complete prompt information;

[0028] The complete prompt information is sorted based on the target functions corresponding to different business needs to generate complete demonstration prompt engineering texts for different business needs.

[0029] Optionally, generating a target prompt engineering text corresponding to the target business based on the user input information includes:

[0030] Obtaining demand information, technical implementation constraint information and / or implementation requirement information input by the user; wherein the demand information is used to describe the functional objectives and expected results corresponding to the target business, as well as the scenario information in the historical real scenario corresponding to the target business; the technical implementation constraint information is used to describe the user's preference information for decision-making agent construction and training; the implementation requirement information is used to describe the user's basic demand information for the target code;

[0031] Based on the demand information, the technical implementation constraint information and / or the implementation requirement information, a target prompt engineering text corresponding to the target business is generated.

[0032] Optionally, the method further comprises:

[0033] Creating corresponding independent class libraries for each network model in the network model library and each reinforcement learning algorithm in the reinforcement learning algorithm library, respectively, and defining class library functions used by the network model or the reinforcement learning algorithm in the independent class libraries, explaining the parameters, return values ​​and usage examples of the class library functions, and obtaining the description documents corresponding to the network model library and the reinforcement learning algorithm library;

[0034] Based on the example codes of constructing different historical decision-making agents using the class library functions, the example codes corresponding to the network model library and the reinforcement learning algorithm library are sorted out.

[0035] Based on the same inventive concept, the second aspect of the embodiment of the present application provides an agent construction system based on a large language model, the system comprising:

[0036] A large language model selection module, used to determine a target large language model;

[0037] A first construction module is used to establish a first prompt engineering text corresponding to the network model based on the description document and example code corresponding to the network model library;

[0038] A second building module is used to establish a second prompt engineering text corresponding to the reinforcement learning algorithm based on the description document and example code corresponding to the reinforcement learning algorithm library;

[0039] A third construction module is used to establish a third prompt engineering text corresponding to the simulation training environment based on the interface documentation, class library functions and code examples of the simulation training environment;

[0040] A fourth construction module, configured to construct a complete demonstration prompt engineering text for different business requirements based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text;

[0041] The decision-making agent model construction plan generation module is used to generate a target prompt engineering text corresponding to the target business based on user input information, and input the target prompt engineering text into the target large language model so that the target large language model performs demand analysis on the target prompt engineering text to obtain a demand label, and based on the demand label and the complete demonstration prompt engineering text, generates a target code corresponding to the target business, and the target code is used to build a corresponding decision-making agent model and train the decision-making agent model.

[0042] Based on the same inventive concept, the third aspect of an embodiment of the present application provides a storage medium, which stores machine executable instructions. When the machine executable instructions are executed by a processor, the method for constructing an intelligent agent based on a large language model proposed in the first aspect of the present application is implemented.

[0043] Based on the same inventive concept, the fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the method for constructing an intelligent agent based on a large language model as proposed in the first aspect of the present application is implemented.

[0044] Compared with the prior art, this application has the following advantages:

[0045] The embodiment of the present application provides a method for constructing an intelligent agent based on a large language model. First, the target large language model is determined; based on the description document and example code corresponding to the network model library, the first prompt engineering text corresponding to the network model is established; based on the description document and example code corresponding to the reinforcement learning algorithm library, the second prompt engineering text corresponding to the reinforcement learning algorithm is established; based on the interface document, class library function and code example of the simulation training environment, the third prompt engineering text corresponding to the simulation training environment is established. Then, based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text, a complete demonstration prompt engineering text for different business needs is constructed. Finally, based on the user input information, a target prompt engineering text corresponding to the target business is generated, and the target prompt engineering text is input into the target large language model, so that the target large language model performs demand analysis on the target prompt engineering text to obtain a demand label, and based on the demand label and the complete demonstration prompt engineering text, a target code corresponding to the target business is generated, and the target code is used to construct a corresponding decision-making intelligent agent model and train the decision-making intelligent agent model. Therefore, by selecting a suitable large language model and establishing a series of prompt engineering texts for the large language model, the large language model and prompt engineering can be used to generate target code on demand. Running the code can correspondingly call the network model library, reinforcement learning algorithm and simulation training environment class library functions, and ultimately realize the auxiliary construction and training of decision-making agents, solving the problems in the existing decision-making agent construction and training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0047] Figure 1 is a flow chart of a method for constructing an intelligent agent based on a large language model in one embodiment of the present application;

[0048] Figure 2 It is a functional module diagram of an intelligent agent construction system based on a large language model in one embodiment of the present application;

[0049] Figure 3 It is a structural schematic diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] Decision-making agents are algorithmic models that are constructed using artificial intelligence methods and use neural networks in their structure. By inputting tasks and conditions into the decision-making agent, it can quickly output decision-making solutions. Decision-making agents have great application potential in business areas that require fast, accurate, and dynamic optimization, such as emergency resource allocation, financial investment, power dispatching, or urban traffic flow control. The construction of decision-making agents is a multi-faceted process that requires comprehensive analysis of task characteristics, complexity, and sample conditions, and the selection of appropriate neural network structures and combinations. It requires a high level of experience and technical skills from researchers.

[0052] At present, the difficulties in building and training decision-making agents mainly include: 1. It is difficult to clarify business problems and accurately refine inputs and outputs, which highly relies on the designer's deep understanding of the business and requires continuous correction and optimization; 2. Designing a neural network structure and reinforcement learning algorithm that adapts to the business is extremely challenging, and it is necessary to consider multiple factors such as business needs and data volume. During network construction and training, it may be necessary to repeatedly tune to avoid overfitting and underfitting, which depends largely on the experience and technology of personnel; 3. The application specifications of the simulation training system that usually supports the training of decision-making agents are complex. The development code calls the simulation training system to support the training of decision-making agents, which requires high professionalism of designers and is relatively inefficient.

[0053] In the related technologies, in the simulation training environment scheduling and scenario design, some studies have used large language models to quickly build simulation scenarios, and fine-tuned the large language models by building a simulation knowledge base, so that the large language models can better adapt to specific simulation scenario requirements and improve their accuracy and practicality in specific fields. However, the fine-tuning of large language models faces many difficulties, such as the difficulty in obtaining and building fine-tuning data, the difficulty in ensuring the effectiveness after fine-tuning, the need to rely on certain hardware resources for fine-tuning of large language models, and the time-consuming training.

[0054] Prompt engineering is an excellent way to avoid fine-tuning large language models and achieve better application, including methods such as thought chaining. By using cleverly designed input prompts, including the logical guidance of thought chaining, the large language model can be prompted to generate more accurate, valuable and logical answers, which can give full play to the advantages of the rich knowledge reserve of the large language model without complex and difficult fine-tuning operations.

[0055] For example, in the medical field, assuming there is a well-trained large language model, when a doctor needs to diagnose a rare disease, the model can be helped by carefully designing prompts. The input prompt is: "Based on the following symptoms and test results, analyze the possible rare diseases: [detailed symptom description], [related test data], please reason according to the logic of medical diagnosis, and refer to the characteristics of common rare diseases to give a possible range of diseases." Through such prompts, the large language model can use its extensive knowledge, combine the logic of the medical field and the characteristics of common diseases, and give some possible diagnostic directions to provide valuable references for doctors. Although the model is not fine-tuned for this medical field, it can understand and handle professional medical problems to a certain extent through accurate prompt engineering. The prompt engineering here includes elements such as tasks (analyzing possible rare diseases), context (detailed symptom descriptions and related test data), examples (may imply reasoning according to a specific medical diagnosis logic), and roles (similar to the role of medical experts), which guide the large language model to output results that meet constraints and are relatively accurate and effective.

[0056] In view of this, the inventor of this application believes that the large language model is a good tool, but fine-tuning the large language model is not a suitable technical route and has poor feasibility. Therefore, prompt engineering is chosen to fully explore and utilize the capabilities of the trained large language model by designing appropriate prompt engineering texts that meet the background requirements of application needs, so as to serve the construction and training of decision-making intelligent agents.

[0057] Therefore, this application proposes a method for building an intelligent agent based on a large language model. By selecting a suitable large language model and establishing a series of prompt engineering texts for the large language model, the large language model and prompt engineering are utilized to generate target code on demand. Running the code can correspond to calling the network model library, reinforcement learning algorithm and simulation training environment class library functions, and finally quickly and effectively realize the auxiliary construction and training of the decision-making intelligent agent.

[0058] Please refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for constructing an intelligent agent based on a large language model proposed in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0059] S101: Determine a target large language model.

[0060] The big language model is based on deep learning technology, especially the neural network based on the Transformer architecture. It learns language patterns, grammar, and semantics by processing large amounts of text data, so that it can understand and generate human language. It is often used in text generation and creation, chatbots and customer service, language translation, and programming code assistance.

[0061] In this embodiment, first of all, it is necessary to select a suitable large language model to assist in building a decision-making agent according to the relevant requirements of the large language model application. It should be noted that this application does not train or fine-tune the large language model, but only performs prompt engineering, thereby utilizing the capabilities of the large language model itself to generate corresponding target codes (and annotations, etc.) for decision-making agent construction and training.

[0062] Specifically, when determining the target large language model, the main factors to consider include the following aspects:

[0063] 1. Parameter scale of the large language model. Since this method needs to be run on a computing device, the parameter scale of the selected large language model should be adapted to the capabilities of the computing device. If the computing power (including computing power, memory, etc.) is high, the number of parameters of the selected large language model can be larger.

[0064] 2. Input scale of large language models. Since this method does not train or fine-tune a large language model, but only uses prompt engineering, and the prompt engineering text in this application involves a lot of content, it is necessary to select a large language model with a large input text or data volume, such as a large language model that can accept tens of millions of Chinese characters (or equivalent data volume) among the currently released large language models.

[0065] 3. Areas that large language models are good at. This includes choosing a large language model that is good at the corresponding language (for example, you can choose a large language model that is good at Chinese to output Chinese results), a corresponding programming language (for example, you can choose Python), and a large language model that is good at writing and understanding scientific problems (such as including various algorithms, etc.) and supports a large amount of output at one time (or can input a large amount of input data in batches and has a strong memory function).

[0066] There are some large language models that perform well in programming and scientific problem understanding, which can be selected based on the above considerations. For example, Pangu Coder2 is not only familiar with common algorithms, but also proficient in using various APIs (Application Programming Interfaces) to solve advanced mathematics problems, and in addition to English, Pangu Coder2 also performs well in Chinese; StarCoder is also a large language model for code, which performs well in popular programming benchmarks, has a context length of more than 8,000 tokens, and can handle more inputs than other open source large language models; for example, CodeX also has powerful code generation capabilities and has received widespread attention and application in the industry.

[0067] S102: Based on the description document and example code corresponding to the network model library, a first prompt engineering text corresponding to the network model is established.

[0068] In this embodiment, the network model library contains various network models for selection in the subsequent construction of decision-making agents. The description document corresponding to the network model library is used to define and explain the relevant functions of various network models, for example, to explain the meaning of the function parameters, to explain the return value of the function, and to explain the use of the function, etc. The example code is used to call the relevant functions of various network models to use these functions to build different network models.

[0069] Furthermore, based on the description documents and example codes corresponding to the network model library, by organizing them in the form of separate files, a first prompt engineering text corresponding to the network model is established to describe relevant knowledge and usage demonstrations of the network model.

[0070] For example, a "network model classification and characteristics document" is established to summarize and summarize the name, characteristics, applicable scenarios, advantages and limitations of the network model in the form of a table, and store it in the form of an Excel file, so that the subsequent large language model can more conveniently and quickly select and call the network model when building a decision-making intelligent agent.

[0071] S103: Based on the description document and example code corresponding to the reinforcement learning algorithm library, a second prompt engineering text corresponding to the reinforcement learning algorithm is established.

[0072] In this embodiment, the reinforcement learning algorithm library contains various reinforcement learning algorithms, which can be selected for the subsequent construction and training of decision-making agents. Similarly, the description document corresponding to the reinforcement learning algorithm library is used to define and explain the relevant functions of various reinforcement learning algorithms, and the example code is used to call the relevant functions of various reinforcement learning algorithms to use these functions to train decision-making agents.

[0073] Furthermore, based on the description documents and example codes corresponding to the reinforcement learning algorithm library, a second prompt engineering text corresponding to the reinforcement learning algorithm is established by organizing them in the form of separate files, which is used to describe the relevant knowledge and usage demonstration of the reinforcement learning algorithm.

[0074] For example, establish a "reinforcement learning algorithm library function description document" to describe the algorithm name, principle, advantages and disadvantages, applicable scenarios, performance comparison, function name, function description, input parameters, output results and algorithm logic in the form of code (and add detailed comments), and store it in the form of .py files (Python's most basic source code file format, used to store Python code in plain text), so that the subsequent large language model can more conveniently and quickly select and call the reinforcement learning algorithm when training decision-making agents.

[0075] S104: Based on the interface documents, class library functions and code examples of the simulation training environment, a third prompt engineering text corresponding to the simulation training environment is established.

[0076] In this implementation, the interface document of the simulation training environment is used to describe the interface function names, interface functions, and input parameter types of different simulation training environments, including class library functions for entity calling, entity editing, engagement rule calling and editing, scenario deployment, simulation environment initialization, training process control, and process data collection. Code examples are used to call related functions of various simulation training environments to assist in the training of decision-making agents.

[0077] The reason why the training of decision-making agents based on deep reinforcement learning requires a simulation training environment is that most actual application scenarios cannot be used directly for training. As a mapping of the actual environment, the simulation training environment can simulate the actual situation and create safe, efficient and repeatable conditions for agent training. During the training process, the decision-making agent first receives the current state generation plan and sends it to the simulation training environment. The simulation training environment runs the plan and produces results, and records the data for training the decision-making agent. Usually, the parameters of the decision-making agent neural network are randomly assigned at the beginning, and the output decision plan is of low quality. Through continuous interaction with the simulation training environment, continuous training and optimization are carried out until a well-trained decision-making agent is obtained. Therefore, a simulation training environment that can effectively train an agent needs to map the entities, entity operation rules and evaluation rules in the real application environment to help the agent continuously improve its decision-making and performance during training.

[0078] In this implementation, based on the interface documentation, library functions and code examples of the simulation training environment, a third prompt engineering text corresponding to the simulation training environment is established by organizing them in the form of separate files, which is used to describe the relevant knowledge and usage demonstrations of the simulation training environment.

[0079] For example, a "Simulation Training Environment Introduction Document" is established to elaborate on the functional modules, characteristics, scope of application, advantages, etc. of the simulation training environment in text form, and store it in the form of a .txt file (a plain text file format). This makes it easier and faster for subsequent large language models to select and call the simulation training environment when training decision-making agents.

[0080] S105: Based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text, a complete demonstration prompt engineering text for different business requirements is constructed.

[0081] In this implementation, the first prompt engineering text, the second prompt engineering text and the third prompt engineering text are separate prompt engineering texts for the network model, the reinforcement learning algorithm and the simulation training environment respectively. The complete demonstration prompt engineering text is a targeted and complete prompt engineering text constructed according to the specific functions of the business requirements and the thinking logic of function realization for different business needs.

[0082] In specific implementation, through folder management and file organization, all complete demonstration prompt project texts are placed in one folder, and various files under a complete demonstration prompt project text for specific business needs will be placed in a subfolder in the folder. The subfolder is further divided into folders to place different codes and documents in S102-S104. It should be noted that the folder and file naming rules must be consistent and must meet data retrieval and usage requirements.

[0083] For example, if business 1 needs to implement red-blue game confrontation, the complete demonstration prompt project text generated for business 1 will include files related to network models, reinforcement learning algorithms, and simulation training environments, so as to demonstrate which network model, reinforcement learning algorithm, and simulation training environment to use for simulation interactive training under this business requirement. If the requirement of business 2 is to assist in generating simulation training environment code, the complete demonstration prompt project text generated for business 2 will include files related to the simulation training environment, such as simulation training environment introduction documents, simulation training environment interface detailed description documents, and other simulation training environment-related code documents. At this time, documents related to the network model are not needed, and the prompt project text will not mention the network model, otherwise it will affect the auxiliary programming efficiency and effect of this function.

[0084] S106: Generate a target prompt engineering text corresponding to the target business based on the user input information, and input the target prompt engineering text into the target large language model, so that the target large language model performs demand analysis on the target prompt engineering text to obtain demand labels, and generates a target code corresponding to the target business based on the demand labels and the complete demonstration prompt engineering text. The target code is used to construct a corresponding decision-making agent model and train the decision-making agent model.

[0085] Assuming that a system for building and training decision-making agents based on a large language model is established, and this system is used to implement decision-making agents that meet specific business needs, then the above steps S102-S105 can be understood as the establishment of basic knowledge, which serves as a general database of the system to support the operation of the system, and step S106 is to establish and apply target prompt engineering texts for specific business problems.

[0086] In this implementation, a functional module for collecting business requirements is constructed (the functional module has functions such as human-computer interaction, data collection and storage, etc.), the user's input information is collected, and the business requirement prompt engineering text is generated accordingly, that is, the target prompt engineering text corresponding to the target business. Then, the target prompt engineering text is parsed using the target large language model, for example, some requirement labels are obtained, and the target code of the decision-making agent construction plan is generated by comparing and matching the requirement labels with the complete demonstration prompt engineering text for different business requirements. The target code can be run to call the class library function, and the neural network model code, the selected reinforcement learning algorithm code, and the code for the interactive interface control with the simulation environment are generated as needed to support the auxiliary construction and training of the decision-making agent.

[0087] In addition, the large language model can also annotate the generated code as needed, and finally deliver the generated target code and related documents to the designers for verification and use.

[0088] This application selects a suitable large language model and establishes a series of prompt engineering texts for the large language model, so as to utilize the large language model and prompt engineering to generate target code on demand. Running the code can call the network model library, reinforcement learning algorithm and simulation training environment class library functions, and finally quickly and effectively realize the auxiliary construction and training of decision-making agents. The series of prompt engineering texts constructed by this application can clearly describe the background, specific requirements and expected output format of the problem, so that the large language model can better understand the task and use its general knowledge and language processing capabilities to generate effective execution code to build and train decision-making agents.

[0089] Optionally, the preparation process of the description documents and example codes corresponding to the above network model library and reinforcement learning algorithm library specifically includes:

[0090] S201: Create corresponding independent class libraries for each network model in the network model library and each reinforcement learning algorithm in the reinforcement learning algorithm library, define class library functions used by the network model or reinforcement learning algorithm in the independent class libraries, explain the parameters, return values ​​and usage examples of the class library functions, and obtain the corresponding description documents for the network model library and the reinforcement learning algorithm library.

[0091] In this implementation, it is necessary to first collect, organize, and supplement the network model library related to the construction of decision-making agent neural networks, including multi-layer perceptron (MLP), convolutional neural network (CNN), recurrent neural network (RNN) and its variants (such as LSTM, GRU, etc.). At the same time, it is necessary to collect, organize, and supplement the reinforcement learning algorithm library including DQN, AC, PG, PPO, SAC, A3C, TD3, etc.

[0092] Then, create an independent class library for each network model and reinforcement learning algorithm, in which: define related functions, including model initialization function (used to set basic parameters such as the number of layers and neurons), input and output setting function (specify the input data format, dimension and output result form), model evaluation function (calculate evaluation indicators such as accuracy and recall rate), model training function (define optimization algorithm, learning rate and other parameters in training), and write detailed instructions and comments for each class library, including function parameter description, return value explanation and usage examples, etc., to finally obtain the corresponding instruction documents for the network model library and reinforcement learning algorithm library.

[0093] S202: Based on the use of library functions to construct instance codes of different historical decision-making agents, the instance codes corresponding to the network model library and the reinforcement learning algorithm library are sorted out.

[0094] In this implementation, the example codes corresponding to the network model library and the reinforcement learning algorithm library in this application can be established by organizing and applying the class library functions to construct example codes of different historical decision-making agents.

[0095] In addition, you can also directly download the open source network model library and reinforcement learning algorithm library, and then use them after modification.

[0096] The preparation process of the interface documentation, class library functions and code examples of the simulation training environment is similar. By designing, organizing and supplementing the simulation training environment interface, write interface documentation for each interface, indicate the interface function name, interface function, input parameter type and meaning, output result format and possible exceptions. Among them, the simulation training environment interface includes class library functions for entity calling, entity editing, engagement rule calling and editing, hypothetical deployment, simulation environment initialization, training process control, process data collection and other functions. By organizing past code examples, add detailed comments to each code example to explain the code logic and key steps.

[0097] Optionally, in the above step S102, the description document and example code corresponding to the network model library include description information and code information corresponding to multiple network models; step S102 establishes a first prompt engineering text corresponding to the network model based on the description document and example code corresponding to the network model library, specifically including:

[0098] S102-1: Based on the description information corresponding to multiple network models, establish a network model classification and feature document, a network model architecture description document, a network model parameter setting document and a network model function definition document; among them, the network model classification and feature document is used to describe the name, characteristics, applicable scenarios, advantages and limitations of each network model; the network model architecture description document is used to describe the hierarchical structure, neuron connection method and data flow of each network model; the network model parameter setting document is used to describe the parameter name, possible value range, recommended value and impact on model performance of each network model; the network model function definition document is used to describe the function name, functional description, input parameters, output results and algorithm logic of each network model.

[0099] In this implementation, based on the description information corresponding to various network models, multiple management documents are established from different aspects such as the characteristics, architecture, parameters, functions and function calls of the network model to demonstrate the network model knowledge. Specifically, it includes:

[0100] 1) Establish "Network Model Classification and Characteristics Document"

[0101] - Format: Presented in tabular form.

[0102] - Storage: Can be saved as Excel file.

[0103] - Elements: network model name, characteristics, applicable scenarios, advantages and limitations, etc.

[0104] - Note: Make sure the classification is clear and accurate, and the feature description is concise and clear.

[0105] 2) Create a "Network Model Architecture Description Document"

[0106] - Form: Draw an architectural diagram and provide a text description.

[0107] - Storage: Can be saved as Excel file.

[0108] - Elements: model hierarchy, neuron connection method, data flow, etc.

[0109] - Note: The architecture diagram should be concise and easy to understand, and the text explanation should be precise.

[0110] 3) Establish a "network model parameter setting document"

[0111] - Format: presented in Excel spreadsheet.

[0112] - Storage: Excel files.

[0113] - Elements: parameter name, possible value range, recommended value, and description of its impact on model performance.

[0114] - Note: The value range should be reasonably recommended based on a large number of experiments and experience.

[0115] 4) Create a "network model function definition document"

[0116] - Form: In code form, with detailed comments.

[0117] - Storage: Save as .py file.

[0118] - Elements: function name, functional description, input parameters, output results, algorithm logic.

[0119] - Notes: The code is standardized, the comments are clear, and the logic is rigorous.

[0120] S102-2: Based on the code information corresponding to the various network models, a network model function call example document is established; the network model function call example document is used to describe example usage scenarios, call codes, and code comments for each network model.

[0121] In this implementation, the model calling example is established mainly based on the code information corresponding to the various network models. Specifically, it includes:

[0122] 5) Create a "network model function call example document"

[0123] - Form: Contains multiple complete calling example codes, each with detailed comments.

[0124] - Storage: .py files.

[0125] - Elements: Example scenario description, calling code, code comments explaining what each line does.

[0126] - Note: Examples should be representative and practical.

[0127] S102-3: Generate a first prompt engineering text corresponding to the network model based on the network model classification and feature document, the network model architecture description document, the network model parameter setting document, the network model function definition document and the network model function call example document.

[0128] In this implementation, based on the instruction documents and example codes corresponding to the network model library, multiple management documents are established by classifying, summarizing and organizing them in the form of separate files, and then the first prompt engineering text corresponding to the network model is obtained, which is used to describe the relevant knowledge and usage demonstrations of the network model.

[0129] Optionally, in the above step S103, the description document and example code corresponding to the reinforcement learning algorithm library include description information and code information corresponding to multiple reinforcement learning algorithms; step S103 establishes a second prompt engineering text corresponding to the reinforcement learning algorithm based on the description document and example code corresponding to the reinforcement learning algorithm library, specifically including:

[0130] S103-1: Based on the description information corresponding to various reinforcement learning algorithms, establish reinforcement learning algorithm library function description documents and model training hyperparameter tuning experience documents; among them, the reinforcement learning algorithm library function description documents are used to explain the algorithm name, principle, advantages and disadvantages, applicable scenarios, performance comparison, function name, function description, input parameters, output results and algorithm logic of each reinforcement learning algorithm; the model training hyperparameter tuning experience document is used to explain the hyperparameters, tuning methods, tuning experience summary and application cases of the network model using each reinforcement learning algorithm.

[0131] In this implementation, based on the description information corresponding to various reinforcement learning algorithms, multiple management documents are established from different aspects such as the characteristics, parameters, functions and function calls of the reinforcement learning algorithms to demonstrate the knowledge of the reinforcement learning algorithms. Specifically, they include:

[0132] 1) Establish "Reinforcement Learning Algorithm Library Function Description Document"

[0133] - Form: In code form, with detailed comments.

[0134] - Storage: Save as .py file.

[0135] - Elements: algorithm name, principle, advantages and disadvantages, applicable scenarios, performance comparison, function name, functional description, input parameters, output results, and algorithm logic.

[0136] - Notes: The code is standardized, the comments are clear, and the logic is rigorous.

[0137] 2) Establish "Model Training Hyperparameter Tuning Experience Document"

[0138] - Format: mainly text, combined with cases and diagrams.

[0139] - Storage: .txt file or Markdown file.

[0140] - Elements: hyperparameter introduction, tuning methods, experience summary, case analysis.

[0141] - Note: Experience is practical and can be used as a reference.

[0142] S103-2: Based on the code information corresponding to the various reinforcement learning algorithms, establish a reinforcement learning algorithm class library function call example document; the reinforcement learning algorithm class library function call example document is used to illustrate the example usage scenarios, calling codes and code comments of each reinforcement learning algorithm.

[0143] In this implementation, examples of reinforcement learning algorithm calls are established mainly based on the code information corresponding to a variety of reinforcement learning algorithms. Specifically, they include:

[0144] 3) Create a "Reinforcement Learning Algorithm Library Function Call Example Document"

[0145] - Form: Contains multiple complete calling example codes, each with detailed comments.

[0146] - Storage: .py files.

[0147] - Elements: Example scenario description, calling code, code comments explaining what each line does.

[0148] - Note: Examples should be representative and practical.

[0149] S103-3: Generate a second prompt engineering text corresponding to the reinforcement learning algorithm based on the reinforcement learning algorithm library function description document, model training hyperparameter tuning experience document, and reinforcement learning algorithm library function call example document.

[0150] In this implementation, based on the instruction documents and example codes corresponding to the reinforcement learning algorithm, multiple management documents are established by classifying, summarizing and organizing them in the form of separate files, and then the second prompt engineering text corresponding to the reinforcement learning algorithm is obtained, which is used to describe the relevant knowledge and usage demonstration of the reinforcement learning algorithm.

[0151] Optionally, the above step S104 establishes a third prompt engineering text corresponding to the simulation training environment based on the interface document, class library function and code example of the simulation training environment, specifically including:

[0152] S104-1: Based on the interface document of the simulation training environment, establish a simulation training environment introduction document, a simulation training environment configuration document, a simulation training environment interface detailed description document and a simulation training environment data format specification document; wherein, the simulation training environment introduction document is used to describe the functional modules, characteristics, scope of application and advantages of each simulation environment; the simulation training environment configuration document is used to describe the hardware requirements, software dependencies, environment variable settings and configuration steps of each simulation environment; the simulation training environment interface detailed description document is used to describe the interface name, function, input parameter format and requirements, output data format and meaning of each simulation environment; the simulation training environment data format specification document is used to describe the input data format, output data format, data type and data dimension of each simulation environment.

[0153] In this implementation, based on the interface documents corresponding to various simulation training environments, multiple management documents are established from different aspects such as the characteristics, functions, data, functions and function calls of the reinforcement learning algorithm to demonstrate the knowledge of the simulation training environment. Specifically, it includes:

[0154] 1) Establish a "Simulation Training Environment Introduction Document"

[0155] - Form: detailed elaboration in text form.

[0156] - Storage: .txt files.

[0157] - Elements: functional modules, characteristics, scope of application, advantages, etc. of the simulation environment.

[0158] - Notes: The language is easy to understand and the key points are highlighted.

[0159] 2) Establish a "Simulation Training Environment Configuration Document"

[0160] - Format: Detailed text description, optionally accompanied by diagrams.

[0161] - Storage: .txt files.

[0162] - Elements: hardware requirements, software dependencies, environment variable settings, configuration steps, etc.

[0163] - Note: The steps are clear and explained step by step in order.

[0164] 3) Establish a "Simulation Training Environment Interface Detailed Description Document"

[0165] - Format: Provide a detailed text description of each interface, with input and output examples.

[0166] - Storage: .txt files or Markdown files.

[0167] - Elements: interface name, function, input parameter format and requirements, output data format and meaning.

[0168] - Notes: The description is accurate and the examples are representative.

[0169] 4) Establish "Simulation Training Environment Data Format Specification Document"

[0170] - Format: Text description with examples.

[0171] - Storage: Markdown files.

[0172] - Elements: specifications such as input data format, output data format, data type, data dimension, etc.

[0173] - Note: Be clear in the specifications and avoid ambiguity.

[0174] S104-2: Based on the class library functions of the simulation training environment, establish a simulation training environment class library function classification summary document and a simulation training environment class library function internal implementation analysis document; wherein, the simulation training environment class library function classification summary document is used to describe the function category, function name and function function of each simulation environment; the simulation training environment class library function internal implementation analysis document is used to describe the function code, key code comments, flow charts and function execution process of each simulation environment.

[0175] In this implementation, the instructions for using the simulation training environment are mainly established based on the class library functions corresponding to the various simulation training environments. Specifically, they include:

[0176] 5) Establish a "Simulation Training Environment Class Library Function Classification Summary Document"

[0177] - Format: List the categories in a table format.

[0178] - Storage: Excel file or Markdown file.

[0179] - Elements: function category, function name, brief description of function.

[0180] - Notes: Reasonable classification, easy to find.

[0181] 6) Establish "Internal Implementation Analysis Document of Simulation Training Environment Class Library Function"

[0182] - Form: Parsed through code comments and flow charts.

[0183] - Storage: .py files and PDF files (flowcharts).

[0184] - Elements: function code, key code comments, flow chart, function execution process description.

[0185] - Notes: The comments are detailed and the flowchart accurately reflects the logic.

[0186] S104-3: Based on the code examples of the simulation training environment, a simulation training environment application instance document is established. The simulation training environment application instance document is used to describe the instance scenarios, implementation steps, code examples and application results of each simulation environment application.

[0187] In this implementation, examples of calling a simulation training environment are established mainly based on code examples corresponding to a variety of simulation training environments. Specifically, the examples include:

[0188] 7) Establish “Simulation Training Environment Application Example Document”

[0189] - Format: Provide templates and examples of various scenario designs, with text descriptions.

[0190] - Storage: .txt file or Markdown file.

[0191] - Elements: example scenario description, implementation steps, code examples, and result analysis.

[0192] - Note: The examples are typical and practical.

[0193] S104-4: Generate the third prompt engineering text corresponding to the simulation training environment based on the simulation training environment introduction document, simulation training environment configuration document, simulation training environment interface detailed description document, simulation training environment data format specification document, simulation training environment class library function classification summary document, simulation training environment class library function internal implementation analysis document and simulation training environment application example document.

[0194] In this implementation, based on the interface documentation, library functions and code examples of the simulation training environment, multiple management documents are established by classifying, summarizing and organizing them in the form of separate files, and then the third prompt engineering text corresponding to the simulation training environment is obtained, which is used to describe the relevant knowledge and usage demonstrations of the simulation training environment.

[0195] In addition, data annotation specifications and sample documents can be established to explain and demonstrate the requirements for data annotation, so that the explanation of all knowledge is kept in a unified format, which is convenient for large language models to understand.

[0196] Establish "Data Annotation Specifications and Sample Documents"

[0197] - Format: Code example presentation with text description.

[0198] - Storage: .txt file or Markdown file.

[0199] - Elements: data cleaning steps, annotation categories, annotation rules, and annotation examples.

[0200] - Notes: The specifications are clear and the examples are representative.

[0201] It should be noted that the above classification methods, file formats (txt and markdown are more commonly used, and can also be directly converted into binary files) and included contents are only examples. When used specifically, they can be adjusted according to actual needs.

[0202] Optionally, the above step S105 constructs a complete demonstration prompt project text for different business needs based on the first prompt project text, the second prompt project text and the third prompt project text, specifically including:

[0203] S105-1: Check and supplement the business requirements of different decision-making intelligent entities, the first prompt engineering text, the second prompt engineering text and the third prompt engineering text to obtain complete prompt information.

[0204] In this implementation, the business requirements of different decision-making agents are checked and supplemented, mainly including: collecting relevant information on the business requirements constructed by the decision-making agents, including but not limited to requirement documents, technical requirements, requirement specifications, implementation plans, work summaries, technical summaries, etc., systematically organizing the collected information, and supplementing and improving unclear or incomplete requirement descriptions.

[0205] Check and supplement the first prompt project text and the second prompt project text, mainly including: 1) Summarize the existing neural network model-related codes and documents, clarify the model type (such as multi-layer perceptron, convolutional neural network, etc.) and architecture (number of layers, number of neurons, connection method, etc.), and check whether the comments are clear and accurate for key functions such as model initialization, forward propagation, and back propagation in the code. If there are deficiencies, supplement and improve them. 2) Organize the code and instructions related to model training, confirm the class libraries and functions called therein, clarify the reinforcement learning algorithm used, the optimization algorithm selected for decision-making agent parameter training (such as stochastic gradient descent, Adagrad, etc.), the learning rate, the number of iterations and other parameters, and add detailed comments for unclear parameter assignments and adjustments.

[0206] Check and supplement the third prompt project text, mainly including: collecting the configuration of the simulation training environment and calling related codes, organizing the settings of parameters such as data loading path, preprocessing method, and result saving path, and supplementing relevant comments if necessary.

[0207] In addition, the selection and use of parallel training strategies are also included. If the decision-making agent adopts a parallel training strategy, collect relevant code and documents. If a parallel computing framework is used (such as Tensorflow's distributed training, PyTorch's DDP, etc.), clarify the division of data and computing tasks, check whether the description of parallel settings and coordination logic is clear, and add comments where there are deficiencies.

[0208] S105-2: The complete prompt information is sorted based on the target functions corresponding to different business needs, and a complete demonstration prompt project text for different business needs is generated.

[0209] In this implementation, the above-mentioned parts that have been supplemented and sorted are classified and organized according to the functions and logic of different business requirements, and a complete demonstration prompt project text with a clear structure, easy to understand and consult, including multiple files, is formed for a specific business requirement. At the same time, the codes and documents for all historical business requirements are sorted and supplemented to form multiple complete demonstration prompt project texts.

[0210] Optionally, in the above step S106, generating a target prompt engineering text corresponding to the target business based on the user input information specifically includes:

[0211] S106-1: Obtain demand information, technical implementation constraint information and / or implementation requirement information input by the user; wherein the demand information is used to describe the functional goals and expected results corresponding to the target business, as well as the scenario information in the historical real scenarios corresponding to the target business; the technical implementation constraint information is used to describe the user's preference information for the construction and training of decision-making agents; and the implementation requirement information is used to describe the user's basic demand information for the target code.

[0212] In this implementation, the information input by the user mainly includes three aspects: business requirement information of the target business, technical implementation constraint information (i.e., the user's construction preferences, such as preferred model structure, reinforcement learning algorithm, parallel training method, etc.) and some other implementation requirement information (such as code description requirements, tuning requirements, etc.).

[0213] Specifically, business requirement information includes: 1) Collecting textual description information related to the business problem to be solved by the decision-making agent, including the main objectives and expected results of the problem, and clearly defining the technical fields involved, specific technical indicators, performance requirements and possible technical limitations. 2) Collecting requirement document information, including but not limited to requirement documents or requirement specifications (such as technical contracts), and other relevant business requirement documents, and supplementing the explanation of the relationship and role with the core requirements. 3) Collecting real-world business problem scenario information, explaining the entities, rules, parameters related to the business problem, and the calculation equations, methods, parameter ranges, etc. related to the entities and rules.

[0214] Technical implementation constraint information includes: 1) Neural network preference, clearly indicating the preferred network structure category, such as multi-layer perceptron, convolutional neural network, etc., as well as possible combinations, preferred number of network layers, parameters and hyperparameters, etc. 2) Preference for imitation examples, clearly indicating the completed examples you want to imitate, and briefly explaining the reasons. 3) Reinforcement learning algorithm preference, specifically indicating the preferred reinforcement learning algorithm, such as A2C, A3C, etc., and explaining the applicable scenarios and expected results. 4) Parallel training requirement determination, clearly determining whether parallel training is required. 5) Parallel training setting preference, if parallel training is required, you can set the number of parallel trainings, explain the basis for selecting this number, and describe how parallel training works with the selected reinforcement learning algorithm.

[0215] Implementation requirements include: 1) Code description document requirements, which clearly state the detailed requirements for code description documents, including the content covered (such as code functions, logic, input and output formats, etc.), the level of detail of the description, and the richness of usage examples. 2) Omission assessment and prompt requirements, which indicate the evaluation criteria for code analysis omissions and the decision basis for whether further prompts are needed. 3) Program feature design requirements, which specify special requirements on code tendencies, such as the prominence of specific functions, the application of specific design patterns, etc. 4) Code segment presentation requirements, which determine whether the code needs to be presented in segments. 5) Interactive tuning requirements, which indicate whether continuous interactive tuning is required, as well as the focus and expected goals of the tuning.

[0216] It should be noted that in order to build a decision-making intelligent entity that meets the target business, the above-mentioned business demand information must be collected, and the technical implementation constraint information and implementation requirement information can be selectively set according to actual conditions. For example, text prompts of the above three aspects can be displayed on the system human-computer interaction page, so that designers can fill in according to actual conditions.

[0217] S106-2: Generate a target prompt engineering text corresponding to the target business based on the demand information, technical implementation constraint information and / or implementation requirement information.

[0218] In this implementation, after collecting the demand information, technical implementation constraint information and / or implementation requirement information input by the designer, the corresponding business demand prompt engineering text, technical implementation constraint prompt engineering text and other implementation requirement prompt engineering texts are automatically generated, and then the target prompt engineering text corresponding to the target business is comprehensively formed.

[0219] For example, a business requirement prompt engineering text is generated through specifications, and the text content will describe the information corresponding to the business requirement, including: business requirement description, business objects, business input, business output, business-related rules and other business-related content. The business requirement prompt engineering text can help the target large language model understand the business requirements. At the same time, the generated complete and standardized business requirement prompt engineering text, combined with the user's own description, can make it easier for the target large language model to understand the requirements.

[0220] In this embodiment, the prompt engineering texts such as the neural network model library, the reinforcement learning algorithm library, the interface documents of the simulation training environment, the class library functions and code examples are ready, and the knowledge and demonstration prompt engineering text database and the complete demonstration prompt engineering text database are deployed. By collecting the input of the designer, the business demand prompt engineering text, the technical implementation constraint prompt engineering text and other implementation requirement prompt engineering text are obtained, and stored in the prompt engineering text database responded by the large language model. Then, using the LangChain framework (a development framework for the large language model), the target large language model is run, and the target large language model will read the prompt engineering text from the prompt engineering text database, parse the requirements, and generate the target code. The target code can be called by running the class library function, and the neural network model code, the selected reinforcement learning algorithm code and the code for the interactive interface control with the simulation environment are generated on demand to support the auxiliary construction and training of the decision-making agent.

[0221] Please refer to Figure 2 Based on the same inventive concept, the second aspect of the embodiment of the present application provides an agent construction system based on a large language model. The agent construction system 200 based on a large language model includes:

[0222] A large language model selection module 201 is used to determine a target large language model;

[0223] A first construction module 202 is used to establish a first prompt engineering text corresponding to the network model based on the description document and example code corresponding to the network model library;

[0224] The second construction module 203 is used to establish a second prompt engineering text corresponding to the reinforcement learning algorithm based on the description document and example code corresponding to the reinforcement learning algorithm library;

[0225] The third construction module 204 is used to establish a third prompt engineering text corresponding to the simulation training environment based on the interface document, class library functions and code examples of the simulation training environment;

[0226] A fourth construction module 205 is used to construct a complete demonstration prompt engineering text for different business requirements based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text;

[0227] The decision agent model construction plan generation module 206 is used to generate a target prompt engineering text corresponding to the target business based on user input information, and input the target prompt engineering text into the target large language model so that the target large language model can perform demand analysis on the target prompt engineering text to obtain demand labels, and generate target codes corresponding to the target business based on the demand labels and the complete demonstration prompt engineering text. The target codes are used to build the corresponding decision agent model and train the decision agent model.

[0228] Optionally, the description document and example code corresponding to the network model library include description information and code information corresponding to a plurality of network models; the first construction module 202 includes:

[0229] The first summary submodule is used to establish a network model classification and feature document, a network model architecture description document, a network model parameter setting document and a network model function definition document based on the description information corresponding to the various network models; wherein the network model classification and feature document is used to describe the name, characteristics, applicable scenarios, advantages and limitations of each network model; the network model architecture description document is used to describe the hierarchical structure, neuron connection mode and data flow of each network model; the network model parameter setting document is used to describe the parameter name, value range, recommended value and impact on model performance of each network model; the network model function definition document is used to describe the function name, function description, input parameters, output results and algorithm logic of each network model;

[0230] The second summary submodule is used to establish a network model function call example document based on the code information corresponding to the multiple network models; the network model function call example document is used to explain the example usage scenarios, call codes and code comments of each network model;

[0231] The first prompt engineering text generation submodule is used to generate the first prompt engineering text corresponding to the network model based on the network model classification and feature document, the network model architecture description document, the network model parameter setting document, the network model function definition document and the network model function call example document.

[0232] Optionally, the description document and example code corresponding to the reinforcement learning algorithm library include description information and code information corresponding to a plurality of reinforcement learning algorithms; the second construction module 203 includes:

[0233] The first summary submodule is used to establish a reinforcement learning algorithm class library function description document and a model training hyperparameter tuning experience document based on the description information corresponding to multiple reinforcement learning algorithms; wherein the reinforcement learning algorithm class library function description document is used to describe the algorithm name, principle, advantages and disadvantages, applicable scenarios, performance comparison, function name, function description, input parameters, output results and algorithm logic of each reinforcement learning algorithm; the model training hyperparameter tuning experience document is used to describe the hyperparameters, tuning methods, tuning experience summary and application cases of the network model using each reinforcement learning algorithm;

[0234] The second summary submodule is used to establish a reinforcement learning algorithm class library function call example document based on the code information corresponding to the multiple reinforcement learning algorithms; the reinforcement learning algorithm class library function call example document is used to explain the example usage scenarios, call codes and code comments of each reinforcement learning algorithm;

[0235] The second prompt engineering text generation submodule is used to generate the second prompt engineering text corresponding to the reinforcement learning algorithm based on the reinforcement learning algorithm library function description document, model training hyperparameter tuning experience document and reinforcement learning algorithm library function call example document.

[0236] Optionally, the third building block 204 includes:

[0237] The first summary submodule is used to establish a simulation training environment introduction document, a simulation training environment configuration document, a simulation training environment interface detailed description document and a simulation training environment data format specification document based on the interface document of the simulation training environment; wherein the simulation training environment introduction document is used to describe the functional modules, characteristics, applicable scope and advantages of each simulation environment; the simulation training environment configuration document is used to describe the hardware requirements, software dependencies, environment variable settings and configuration steps of each simulation environment; the simulation training environment interface detailed description document is used to describe the interface name, function, input parameter format and requirements, output data format and meaning of each simulation environment; the simulation training environment data format specification document is used to describe the input data format, output data format, data type and data dimension of each simulation environment;

[0238] The second summary submodule is used to establish a classification summary document of the class library functions of the simulation training environment and an internal implementation analysis document of the class library functions of the simulation training environment based on the class library functions of the simulation training environment; wherein the classification summary document of the class library functions of the simulation training environment is used to describe the function category, function name and function function of each simulation environment; the internal implementation analysis document of the class library functions of the simulation training environment is used to describe the function code, key code comments, flow chart and function execution process of each simulation environment;

[0239] The third summary submodule is used to establish a simulation training environment application instance document based on the code examples of the simulation training environment, and the simulation training environment application instance document is used to describe the instance scenarios, implementation steps, code examples and application results of each simulation environment application;

[0240] The third prompt engineering text generation submodule is used to generate the third prompt engineering text corresponding to the simulation training environment based on the simulation training environment introduction document, the simulation training environment configuration document, the simulation training environment interface detailed description document, the simulation training environment data format specification document, the simulation training environment class library function classification summary document, the simulation training environment class library function internal implementation analysis document and the simulation training environment application example document.

[0241] Optionally, the fourth building block 205 includes:

[0242] A leak checking and supplementing submodule is used to check and supplement the business requirements of different decision-making agents, the first prompt engineering text, the second prompt engineering text and the third prompt engineering text to obtain complete prompt information;

[0243] The complete demonstration prompt project text generation submodule is used to organize the complete prompt information based on the target functions corresponding to different business needs, and generate complete demonstration prompt project texts for different business needs.

[0244] Optionally, the decision agent model construction solution generation module 206 includes:

[0245] The user demand acquisition submodule is used to obtain the demand information, technical implementation constraint information and / or implementation requirement information input by the user; wherein the demand information is used to describe the functional objectives and expected results corresponding to the target business, as well as the scenario information in the historical real scenario corresponding to the target business; the technical implementation constraint information is used to describe the user's preference information on the construction and training of the decision-making intelligent agent; the implementation requirement information is used to describe the user's basic demand information for the target code;

[0246] The target prompt engineering text generation submodule is used to generate the target prompt engineering text corresponding to the target business based on demand information, technical implementation constraint information and / or implementation requirement information.

[0247] Optionally, the system further comprises:

[0248] A description document generation module is used to create a corresponding independent class library for each network model in the network model library and each reinforcement learning algorithm in the reinforcement learning algorithm library, and define the class library functions used by the network model or reinforcement learning algorithm in the independent class library, explain the parameters, return values ​​and usage examples of the class library functions, and obtain the description documents corresponding to the network model library and the reinforcement learning algorithm library;

[0249] The example code generation module is used to build example codes of different historical decision-making agents based on the application of library functions, and to organize the example codes corresponding to the network model library and reinforcement learning algorithm library.

[0250] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0251] On the third aspect, based on the same inventive concept, an embodiment of the present application provides a storage medium, which stores machine executable instructions. When the machine executable instructions are executed by a processor, the method for constructing an intelligent agent based on a large language model proposed in the first aspect of the present application is implemented.

[0252] It should be noted that the specific implementation of the storage medium of the embodiment of the present application refers to the specific implementation of the intelligent agent construction method based on the large language model proposed in the first aspect of the aforementioned embodiment of the present application, and will not be repeated here.

[0253] In the fourth aspect, based on the same inventive concept, refer to Figure 3 An embodiment of the present application provides an electronic device 300, including a processor 301 and a memory 302; the memory 302 stores machine executable instructions that can be executed by the processor 301, and the processor 301 is used to execute the machine executable instructions to implement the intelligent agent construction method based on a large language model as proposed in the first aspect of the present application.

[0254] It should be noted that the specific implementation of the electronic device 300 in the embodiment of the present application refers to the specific implementation of the intelligent agent construction method based on the large language model proposed in the first aspect of the aforementioned embodiment of the present application, and will not be repeated here.

[0255] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0256] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0257] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0258] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0259] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0260] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0261] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0262] The above is a detailed introduction to the intelligent agent construction method, system, medium and device based on a large language model provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for constructing an intelligent agent based on a large language model, characterized in that: The method comprises: Determine the target large language model; Based on the description documents and example codes corresponding to the network model library, a first prompt engineering text corresponding to the network model is established; Based on the description documents and example codes corresponding to the reinforcement learning algorithm library, establish the second prompt engineering text corresponding to the reinforcement learning algorithm; Based on the interface documentation, class library functions and code examples of the simulation training environment, a third prompt engineering text corresponding to the simulation training environment is established; Based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text, a complete demonstration prompt engineering text for different business needs is constructed; Generate a target prompt engineering text corresponding to the target business based on user input information, and input the target prompt engineering text into the target large language model, so that the target large language model performs demand analysis on the target prompt engineering text to obtain a demand label, and generate a target code corresponding to the target business based on the demand label and the complete demonstration prompt engineering text, wherein the target code is used to construct a corresponding decision-making agent model and train the decision-making agent model; Among them, the first prompt engineering text is used to describe the relevant knowledge and usage demonstration of the network model, the second prompt engineering text is used to describe the relevant knowledge and usage demonstration of the reinforcement learning algorithm, and the third prompt engineering text is used to describe the relevant knowledge and usage demonstration of the simulation training environment. The complete demonstration prompt engineering text is constructed according to the functions and function implementation logic of business requirements.

2. The method according to claim 1, characterized in that The description documents and example codes corresponding to the network model library include description information and code information corresponding to a variety of network models; Based on the description documents and example codes corresponding to the network model library, the first prompt engineering text corresponding to the network model is established, including: Based on the description information corresponding to various network models, establish network model classification and feature documents, network model architecture description documents, network model parameter setting documents and network model function definition documents; wherein the network model classification and feature documents are used to describe the name, characteristics, applicable scenarios, advantages and limitations of each network model; the network model architecture description document is used to describe the hierarchical structure, neuron connection mode and data flow of each network model; the network model parameter setting document is used to describe the parameter name, possible value range, recommended value and impact on model performance of each network model; the network model function definition document is used to describe the function name, functional description, input parameters, output results and algorithm logic of each network model; Based on the code information corresponding to the various network models, a network model function call example document is established; the network model function call example document is used to describe the example usage scenarios, call codes and code comments of each network model; Based on the network model classification and characteristic document, the network model architecture description document, the network model parameter setting document, the network model function definition document and the network model function call example document, a first prompt engineering text corresponding to the network model is generated.

3. The method according to claim 1, characterized in that The documentation and example codes corresponding to the reinforcement learning algorithm library include description information and code information corresponding to various reinforcement learning algorithms; Based on the description documents and example codes corresponding to the reinforcement learning algorithm library, establish the second prompt engineering text corresponding to the reinforcement learning algorithm, including: Based on the description information corresponding to multiple reinforcement learning algorithms, establish reinforcement learning algorithm library function description documents and model training hyperparameter tuning experience documents; wherein, the reinforcement learning algorithm library function description documents are used to describe the algorithm name, principle, advantages and disadvantages, applicable scenarios, performance comparison, function name, function description, input parameters, output results and algorithm logic of each reinforcement learning algorithm; the model training hyperparameter tuning experience documents are used to describe the hyperparameters, tuning methods, tuning experience summary and application cases of the network model using each reinforcement learning algorithm; Based on the code information corresponding to the various reinforcement learning algorithms, a reinforcement learning algorithm class library function call example document is established; the reinforcement learning algorithm class library function call example document is used to explain the example usage scenarios, call codes and code comments of each reinforcement learning algorithm; Based on the reinforcement learning algorithm class library function description document, the model training hyperparameter tuning experience document and the reinforcement learning algorithm class library function call example document, a second prompt engineering text corresponding to the reinforcement learning algorithm is generated.

4. The method according to claim 1, characterized in that: Based on the interface documentation, library functions and code examples of the simulation training environment, establish the third prompt engineering text corresponding to the simulation training environment, including: Based on the interface document of the simulation training environment, establish a simulation training environment introduction document, a simulation training environment configuration document, a simulation training environment interface detailed description document and a simulation training environment data format specification document; wherein the simulation training environment introduction document is used to describe the functional modules, characteristics, scope of application and advantages of each simulation environment; the simulation training environment configuration document is used to describe the hardware requirements, software dependencies, environment variable settings and configuration steps of each simulation environment; the simulation training environment interface detailed description document is used to describe the interface name, function, input parameter format and requirements, output data format and meaning of each simulation environment; the simulation training environment data format specification document is used to describe the input data format, output data format, data type and data dimension of each simulation environment; Based on the class library functions of the simulation training environment, a classification summary document of the class library functions of the simulation training environment and an internal implementation analysis document of the class library functions of the simulation training environment are established; wherein the classification summary document of the class library functions of the simulation training environment is used to describe the function categories, function names and function functions of each simulation environment; the internal implementation analysis document of the class library functions of the simulation training environment is used to describe the function codes, key code comments, flow charts and function execution processes of each simulation environment; Based on the code examples of the simulation training environment, a simulation training environment application example document is established, wherein the simulation training environment application example document is used to describe the example scenarios, implementation steps, code examples and application results of each simulation environment application; Based on the simulation training environment introduction document, the simulation training environment configuration document, the simulation training environment interface detailed description document, the simulation training environment data format specification document, the simulation training environment class library function classification summary document, the simulation training environment class library function internal implementation analysis document and the simulation training environment application example document, a third prompt engineering text corresponding to the simulation training environment is generated.

5. The method according to claim 1, characterized in that Based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text, a complete demonstration prompt engineering text for different business needs is constructed, including: Check and supplement the business requirements of different decision-making agents, the first prompt engineering text, the second prompt engineering text, and the third prompt engineering text to obtain complete prompt information; The complete prompt information is sorted based on the target functions corresponding to different business needs to generate complete demonstration prompt engineering texts for different business needs.

6. The method according to claim 1, characterized in that Generate target prompt engineering text corresponding to the target business based on user input information, including: Obtaining demand information, technical implementation constraint information and / or implementation requirement information input by the user; wherein the demand information is used to describe the functional objectives and expected results corresponding to the target business, as well as the scenario information in the historical real scenario corresponding to the target business; the technical implementation constraint information is used to describe the user's preference information for decision-making agent construction and training; the implementation requirement information is used to describe the user's basic demand information for the target code; Based on the demand information, the technical implementation constraint information and / or the implementation requirement information, a target prompt engineering text corresponding to the target business is generated.

7. The method according to claim 1, characterized in that The method further comprises: Creating corresponding independent class libraries for each network model in the network model library and each reinforcement learning algorithm in the reinforcement learning algorithm library, respectively, and defining class library functions used by the network model or the reinforcement learning algorithm in the independent class libraries, explaining the parameters, return values ​​and usage examples of the class library functions, and obtaining the description documents corresponding to the network model library and the reinforcement learning algorithm library; Based on the example codes of constructing different historical decision-making agents using the class library functions, the example codes corresponding to the network model library and the reinforcement learning algorithm library are sorted out.

8. An intelligent agent construction system based on a large language model, characterized in that: The system comprises: A large language model selection module, used to determine a target large language model; A first construction module is used to establish a first prompt engineering text corresponding to the network model based on the description document and example code corresponding to the network model library; The second building module is used to establish a second prompt engineering text corresponding to the reinforcement learning algorithm based on the description document and example code corresponding to the reinforcement learning algorithm library; A third construction module is used to establish a third prompt engineering text corresponding to the simulation training environment based on the interface documentation, class library functions and code examples of the simulation training environment; A fourth construction module, configured to construct a complete demonstration prompt engineering text for different business requirements based on the first prompt engineering text, the second prompt engineering text and the third prompt engineering text; A decision agent model construction scheme generation module is used to generate a target prompt engineering text corresponding to a target business based on user input information, and input the target prompt engineering text into the target large language model so that the target large language model performs demand analysis on the target prompt engineering text to obtain a demand label, and based on the demand label and the complete demonstration prompt engineering text, generates a target code corresponding to the target business, and the target code is used to construct a corresponding decision agent model and train the decision agent model; Among them, the first prompt engineering text is used to describe the relevant knowledge and usage demonstration of the network model, the second prompt engineering text is used to describe the relevant knowledge and usage demonstration of the reinforcement learning algorithm, and the third prompt engineering text is used to describe the relevant knowledge and usage demonstration of the simulation training environment. The complete demonstration prompt engineering text is constructed according to the functions and function implementation logic of business requirements.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the intelligent agent construction method based on a large language model as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the agent construction method based on a large language model as described in any one of claims 1 to 7.

Images