Instruction-based agent multi-style stepless adjustment method and system

Abstract

The invention provides an instruction-based agent multi-style stepless adjustment method and system, and the method comprises the steps: enabling a control strategy model to control an expression strategy of an agent in an environment according to a style parameter, and enabling the style parameter to represent the tendency of controlling the agent to execute different expression strategies; a natural language instruction input by a user is analyzed, and style parameters needing to be adjusted and the corresponding numerical value adjustment degree are determined; and inputting the style parameters needing to be adjusted and the corresponding numerical value adjustment degrees into the control strategy model, and enabling the control strategy model to control an expression strategy of the intelligent agent in the environment according to the adjusted style parameters. According to the method, the intention of the user can be understood and analyzed into the related style parameters, and the numerical value of each style parameter can be determined, so that the behavior style of the strategy can be finely adjusted.

Description

Technical Field

[0001] The present invention relates to the field of AI technology, and in particular to an instruction-based intelligent agent multi-style infinite adjustment method and system. Background Art

[0002] An agent is an agent that can perceive the environment and take actions to achieve specific goals. It can be software, hardware or a system with autonomy, adaptability and interaction. The agent perceives changes in the environment (such as through sensors or data input), makes judgments and decisions based on the knowledge and algorithms it has learned, and then performs actions to affect the environment or achieve predetermined goals. Agents are widely used in the field of artificial intelligence, and are commonly found in automated systems, robots, virtual assistants, and game characters. The core of agents is the ability to learn autonomously and continuously evolve to better complete tasks and adapt to complex environments. However, in scenarios with multiple style requirements, it is difficult for agents in existing technologies to make smooth style adjustments, and different models are designed for different style requirements to switch.

[0003] Patent document CN105390135A discloses a game voice control system, a voice acquisition module, through a trigger signal, starts the microphone in real time to collect the user's voice information, and encodes and converts it into a digital file; a voice recognition module is used to identify the voice information collected by the voice acquisition module, and convert the voice information into voice meaning data in a specified format; a voice processing module is used to receive the voice meaning data transmitted by the voice recognition module, compare and match it with the corresponding instructions, and judge the validity of the matched instructions. If the instructions are judged to be valid, it is triggered and sent to the control logic module of the game to perform the corresponding control operation. Similarly, the above patent document has a single model, and the main content is to convert instructions into limited specific control modules. It is difficult to control and adjust the style in reasoning, and it cannot have a variety of strategies at the same time. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide an instruction-based intelligent agent multi-style infinite adjustment method and system.

[0005] According to the present invention, a method for infinite adjustment of multiple styles of intelligent agents based on instructions includes: Initial step: let the control strategy model control the agent's performance strategy in the environment according to the style parameters, where the style parameters represent the tendency of controlling the agent to execute different performance strategies; Instruction acquisition step: parsing the natural language instructions input by the user to determine the style parameters that need to be adjusted and the corresponding numerical adjustment degree; Style adjustment step: input the style parameters to be adjusted and the corresponding numerical adjustment degree into the control strategy model, so that the control strategy model controls the performance strategy of the intelligent agent in the environment according to the adjusted style parameters.

[0006] Further, the performance strategy includes executing a meta-behavior or a combination of meta-behaviors, and the tendency includes controlling the reward scale after the agent executes different meta-behaviors or a combination of meta-behaviors; The range of the style parameter includes [-1, 1]; When the style parameter is -1, it means that executing this meta-behavior or meta-behavior combination will obtain the opposite reward, thus forming the opposite tendency; When the style parameter is 0, it means that there is no reward for performing the meta-behavior or meta-behavior combination, thus indicating that the performance of the behavior is neither encouraged nor supported; When the style parameter is 1, it indicates that the reward for the meta-behavior or meta-behavior combination is positively increased to the maximum extent.

[0007] Furthermore, the instruction acquisition step includes supplementing the natural language instructions input by the user into pre-constructed prompt words, inputting the supplemented prompt words into the large language model, parsing the large language model to obtain meta-behaviors related to the natural language instructions, evaluating the preference degree of each related meta-behavior based on the sentiment analysis of the natural language instructions, giving the numerical results of the corresponding style parameters and generating structured outputs.

[0008] Furthermore, the supplemented prompt words include: scenario description, parameter type and meaning description, instruction parsing example, natural language instruction and output format requirements, among which: a) Scenario description: describes the scenario in which the agent is located and the role played by the large language model; b) Parameter type and meaning description: explain the meaning of each style parameter, and explain the value type and range; c) Instruction parsing example: Give an example of parsing natural language instructions into style parameters for reference by the large model; The purpose of the examples described is to: 1) The large model is used to decompose the natural language instructions. The large language model uses the prompt words to find the meta-behaviors related to the natural language instructions. 2) Let the big model determine the numerical degree of each relevant meta-behavior based on the intent and tone of the natural language instruction; d) Natural language instructions: insert natural language instructions input by the user; e) Output format requirements: Describe the format in which the large language model needs to be returned.

[0009] According to the present invention, a multi-style stepless adjustment system for intelligent agents based on instructions is provided, comprising: Initial module: let the control strategy model control the performance strategy of the agent in the environment according to the style parameters, where the style parameters represent the tendency of controlling the agent to execute different performance strategies; Instruction acquisition module: parses the natural language instructions input by the user to determine the style parameters that need to be adjusted and the corresponding numerical adjustment degree; Style adjustment module: inputs the style parameters to be adjusted and the corresponding numerical adjustment degree into the control strategy model, so that the control strategy model controls the performance strategy of the intelligent agent in the environment according to the adjusted style parameters.

[0010] Further, the performance strategy includes executing a meta-behavior or a combination of meta-behaviors, and the tendency includes controlling the reward scale after the agent executes different meta-behaviors or a combination of meta-behaviors; The range of the style parameter includes [-1, 1]; When the style parameter is -1, it means that executing this meta-behavior or meta-behavior combination will obtain the opposite reward, thus forming the opposite tendency; When the style parameter is 0, it means that there is no reward for performing the meta-behavior or meta-behavior combination, thus indicating that the performance of the behavior is neither encouraged nor supported; When the style parameter is 1, it indicates that the reward for the meta-behavior or meta-behavior combination is positively increased to the maximum extent.

[0011] Furthermore, the instruction acquisition module includes supplementing the natural language instructions input by the user into pre-built prompt words, inputting the supplemented prompt words into the large language model, parsing the large language model to obtain meta-behaviors related to the natural language instructions, evaluating the preference degree of each related meta-behavior based on the sentiment analysis of the natural language instructions, giving the numerical results of the corresponding style parameters and generating structured outputs.

[0012] Furthermore, the supplemented prompt words include: scenario description, parameter type and meaning description, instruction parsing example, natural language instruction and output format requirements, among which: a) Scenario description: describes the scenario in which the agent is located and the role played by the large language model; b) Parameter type and meaning description: explain the meaning of each style parameter, and explain the value type and range; c) Instruction parsing example: Give an example of parsing natural language instructions into style parameters for reference by the large model; The purpose of the examples described is to: 1) The large model is used to decompose the natural language instructions. The large language model uses the prompt words to find the meta-behaviors related to the natural language instructions. 2) Let the big model determine the numerical degree of each relevant meta-behavior based on the intent and tone of the natural language instruction; d) Natural language instructions: insert natural language instructions input by the user; e) Output format requirements: Describe the format in which the large language model needs to be returned.

[0013] According to a computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of the instruction-based intelligent agent multi-style infinite adjustment method are implemented. An electronic device provided according to the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the instruction-based intelligent agent multi-style infinite adjustment method are implemented.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The behavior style of the strategy of the present invention can be controlled by style parameters. According to the application environment and multi-style strategies, targeted prompt words are designed, and a large language model is used in combination with prompt words to convert the user's natural language instructions into inputs of the multi-style strategies, thereby achieving the strategy desired by the user.

[0015] The present invention can understand the user's intention and parse it into relevant style parameters, and can determine the numerical value of each style parameter, so as to fine-tune the behavior style of the strategy. The user can adjust the corresponding behavior that does not conform to his or her own ideas to different degrees through natural language. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 It is a work flow chart of the present invention; Figure 2 Schematic diagram of the learning method of the control strategy model; Figure 3 Schematic diagram for meta-behavior and reward shaping; Figure 4 Schematic diagram of style generator; Figure 5 Diagram of training a network for a multi-style strategy. DETAILED DESCRIPTION

[0017] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0018] Reinforcement learning (RL): Reinforcement learning is a machine learning method that allows an agent to learn how to make optimal decisions by interacting with the environment. The goal of reinforcement learning is to find a strategy that allows the agent to obtain the maximum cumulative reward within a certain period of time.

[0019] Training environment: Or simply environment, in reinforcement learning, the environment is the world that the agent interacts with. The environment receives the agent's actions and returns new states and rewards based on these actions. The environment can be any system that the agent can interact with, such as a game, the physical world of a robot, or even a stock market model.

[0020] Agent: In reinforcement learning, an agent is an entity that can perceive the environment and perform actions in the environment. The goal of the agent is to maximize the total reward by learning how to choose actions.

[0021] State: A state is a description of the environment at a certain moment in time, which contains all the necessary information that the agent needs to make a decision. For example, in a board game, the state may include the type of chess piece at each position on the board.

[0022] Action: An action is a basic operation that an agent can perform in a given state. An agent interacts with the environment by performing actions. For example, if the agent is a character in a game, possible actions include moving left, moving right, jumping, etc.

[0023] Reward: Reward is the feedback from the environment to the agent's actions. When the agent performs an action, the environment returns a reward to tell the agent whether the action is good or bad. Rewards can be positive (indicating encouragement of the action) or negative (indicating a desire to inhibit the action). The goal of the agent is to maximize the total reward by learning how to choose actions.

[0024] Training epoch: The training environment does not run indefinitely, but starts and ends in units of training epochs. At the beginning of each training epoch, the environment will be initialized to a certain state; at the end of each training epoch, done will be used as a signal.

[0025] Reward shaping: Also known as reward setting, reward shaping is a reinforcement learning technique that guides the agent to learn faster by designing the reward function.

[0026] Meta-behavior: The main behavior of the agent in the training environment. Usually, most of the agent's behaviors in the environment can be composed of meta-behaviors. For example, the behavior of the agent jumping left can be completed by the combination of the two meta-behaviors of the agent moving left and jumping.

[0027] Large Language Model (LLM): A large language model is a large-scale neural network trained on massive amounts of data that can understand and generate human language. Such models typically have billions to trillions of parameters and can perform a variety of language tasks such as text generation, question answering, translation, and summarization.

[0028] Natural Language: Natural language refers to the language that humans use naturally in daily life, such as Chinese, English, French, etc., for communication between people. It is different from artificially designed formal languages, such as programming languages ​​or logical symbol systems, which are usually used in specific technical fields.

[0029] Prompt: refers to the input text or instructions given to the model when using a large language model, which is used to guide the model to generate specific outputs or perform specific tasks. Prompt design can significantly improve the performance of the model, allowing it to more accurately understand user intent and produce the desired results.

[0030] Speech recognition (Automatic Speech Recognition, ASR): is a technology that automatically converts human speech into text and is a key technology for realizing human-computer voice interaction. It processes audio signals into corresponding text through components such as acoustic models, language models, and decoders.

[0031] like Figure 1 As shown, the present invention provides an instruction-based agent multi-style infinite adjustment method, comprising: Initial step: Let the control strategy model control the agent's performance strategy in the environment according to the style parameters. The style parameters represent the tendency of the control agent to execute different performance strategies.

[0032] Instruction acquisition step: parse the natural language instructions input by the user to determine the style parameters that need to be adjusted and the corresponding numerical adjustment degree.

[0033] Style adjustment step: input the style parameters to be adjusted and the corresponding numerical adjustment degree into the control strategy model, so that the control strategy model controls the performance strategy of the intelligent agent in the environment according to the adjusted style parameters.

[0034] like Figure 2 As shown, in the initial step, the learning method of the control strategy model includes: Meta-behavior acquisition step: obtain the meta-behavior of the agent in the environment, and shape the reward for each meta-behavior. Each meta-behavior includes a corresponding style parameter, and the style parameter controls the reward scale after completing these meta-behaviors.

[0035] In the present invention, the behaviors of the agent to be trained in the environment are first summarized, and a series of meta-behaviors are designed. These meta-behaviors can be completed by the basic actions of the agent. Then, each meta-behavior is rewarded, that is, a reward value will be obtained after completing the meta-behavior. Each meta-behavior has a corresponding style parameter, which controls the reward scale after completing these meta-behaviors, so each style parameter can represent the tendency of the corresponding meta-behavior. The design criteria of meta-behaviors are: 1. Use as few meta-behaviors as possible to cover as many behaviors of the agent in the environment as possible; 2. The rewards of meta-behaviors are easy to design.

[0036] Style generation step: During the interaction between the agent and the environment, a set of style parameters is randomly generated according to the number of meta-behaviors. The style parameters and state combination are sent to the agent's model for inference to obtain actions. After the actions are fed back to the environment, the reward obtained by the agent at the current moment and the state at the next moment are generated.

[0037] The inclinations of different meta-behaviors of the agent can freely combine into a large number of strategic styles. In the process of interaction between the agent and the environment, the style generator will randomly generate a set of style parameters according to the number of meta-behaviors. This style parameter will be combined with the state and then sent to the agent's model for reasoning to obtain the action. After the action is fed back to the environment, the reward that the agent can obtain at the current moment and the state at the next moment are generated. In this process, the reward will be affected by multiple style parameters.

[0038] Reinforcement learning steps: Perform reinforcement learning on the style parameters, states, actions, and rewards in the process of interaction between the agent and the environment, fit multiple style parameters and meta-behavioral tendencies, and finally train a multi-style, highly controllable strategy model.

[0039] Different multi-style parameter settings can form multi-style strategies, so that a single reinforcement learning model can have multiple styles of strategies at the same time, which increases the generalization of the model and reduces the redundancy of the model. By adjusting the multi-style parameters, the model's strategy can be controlled, such as controlling the style of the current strategy, the timing of style switching, etc., which increases the controllability of the strategy and reduces the complexity of the business. With the help of data statistics, multi-style parameter configurations that can meet specific business needs can be automatically selected, reducing manpower investment.

[0040] The following is an example of a basketball game: We first need to design a series of meta-behaviors for the agent in this environment. The principles for designing meta-behaviors are: (1) The correlation between meta-behaviors should be as small as possible; (2) By combining meta-behaviors, we can cover as many behaviors as possible that the agent can perform in the environment; (3) Under the above conditions, the fewer the number of meta-behaviors, the better.

[0041] This is to make the subsequent process simpler and greatly increase the success rate of multi-style strategy training. Then we need to shape the reward for each meta-behavior, that is, how much reward will be obtained for completing such a meta-behavior. The process is as follows: Figure 3 As shown. a n Indicates completion of n The basic reward for each behavior. w n Indicates the style parameter corresponding to the meta-behavior. The basic reward and the style parameter are multiplied to get the final reward of the meta-behavior. Meta-behaviors can be combined to form a variety of behaviors. For example, "two-point shooting" and "area 1 activities" can be combined to form the behavior of "making two-point shooting in area 1". Through the adjustment of style parameters, the tendency of these behavior combinations also has different degrees, thus forming a variety of strategic styles.

[0042] During the reinforcement learning phase, the environment will interact with the training end to continuously generate data for the latest strategy. At the beginning of each training period of the environment, a style generator will generate style parameters used in that training period. The style generator can generate style parameters that correspond to the reward configuration one by one. The style parameters can be multiplied by the corresponding rewards to control the value of the reward obtained when the strategy performs the behavior during training. The range of style parameters can usually be set to [-1, 1]. When the style parameter is -1, it means that the strategy will receive the opposite reward for executing the behavior, thus forming an opposite tendency; when the style parameter is 0, it means that no reward will be obtained for executing the line, which means that the behavior is neither encouraged nor supported; when the style parameter is 1, it means that the reward for the behavior is positively increased to the greatest extent. Style parameters can be generated by random sampling or by setting specific rules, such as Figure 4 As shown. For example, you can randomly generate any value between -1 and 1, or generate it at intervals of 0.1, or only generate values ​​between 0 and 1, etc. Through the style generator, the reinforcement learning strategy can obtain data under different reward settings, thus providing a basis for the subsequent reinforcement learning algorithm to fit the optimal strategy under different style parameters.

[0043] The generated style parameters are encoded by a style encoder during training to obtain style features, which are then merged with the features of the original input state and sent to the subsequent network for learning. This process will eventually generate action probabilities and values. When the style parameters change, the features of the state do not change, and the changes in the strategy are only fitted by the subsequent network parameters, which is conducive to the learning of state and style parameters. This process is as follows Figure 5 shown.

[0044] In order to efficiently train models with multi-style policies, we use the Proximal Policy Optimization (PPO) algorithm as our main algorithm. PPO is a popular policy gradient algorithm that performs well in various tasks. The PPO algorithm limits the degree of policy change by limiting the action probability ratio between the current policy and the old policy. The action probability ratio between the current policy and the old policy is as follows:

[0045] in s t 、c t 、a t Represents the time step t The following states, style parameters and actions; and The current strategy and the old strategy are in state s t and style parameters c t Take action a t The probability of; the restricted ratio is ,in ε is a hyperparameter used to control the degree of restriction. This restriction can prevent the strategy from fluctuating greatly, thus avoiding instability in the training process.

[0046] The policy objective function is defined as:

[0047] in represents the expectation of a sample batch, A t is the time step calculated by the Generalized Advantage Estimator (GAE) t Advantage estimation; The value function objective is defined as:

[0048] Among them, the target return , is the value function of the current strategy, is the value function of the old strategy; Through continuous optimization and , let the training network be in different states s and style parameters c The optimal strategy is available under all situations. Different style parameters and corresponding rewards are fitted by the value function. The value function can guide the optimal behavior of the strategy under different styles, thus ensuring the convergence of the algorithm. At the same time, by setting different style parameters in the input to call different behavior styles, the controllability of the strategy is ensured.

[0049] In training, multi-style parameters can generally be randomly generated, thereby increasing the stability and generalization of the strategy. When in use, the corresponding style parameters are generated according to the user's instructions. In this way, different style parameters can be selected in a targeted manner. Since the style parameters can be changed at any time during the game, this method realizes real-time control of the strategy, greatly increasing the practicality and flexibility of AI Bot.

[0050] Specifically, the instruction acquisition step includes: supplementing the natural language instructions input by the user into the pre-built prompt words, inputting the supplemented prompt words into the large language model, parsing the large language model to obtain the meta-behaviors related to the natural language instructions, evaluating the preference degree of each related meta-behavior based on the sentiment analysis of the natural language instructions, giving the numerical results of the corresponding style parameters and generating structured output. The prompt words include: scene description, parameter type and meaning description, instruction parsing example, natural language instructions and output format requirements, among which the natural language instruction part is the content waiting to be supplemented. Among them: a) Scenario description: describes the scenario in which the agent is located and the role played by the large language model; b) Parameter type and meaning description: explain the meaning of each style parameter, and explain the value type and range; c) Instruction parsing example: Give an example of parsing natural language instructions into style parameters for reference by the large model; The purpose of the example is to: 1) The large model is used to decompose the natural language instructions. The large language model uses the prompt words to find the meta-behaviors related to the natural language instructions. 2) Let the big model determine the numerical degree of each relevant meta-behavior based on the intent and tone of the natural language instruction; d) Natural language instructions: insert natural language instructions input by the user; e) Output format requirements: Describe the format in which the large language model needs to be returned.

[0051] Taking into account the limitations of large language models in processing precise numerical values, this conversion method allows it to output a scale of 0 to 10 to represent the degree of floating-point style parameters, and then maps this scale to the actual adjustable range of the style parameters. For Boolean style parameters, 0 or 1 is output. Finally, the 0-10 degree value or Boolean type parameter output by the large language model is mapped to the actual parameter range. For example, the instruction "Full defense!" means that the user wants the AI ​​to fully defend. Therefore, meta-behaviors related to defense should have a high preference level, while meta-behaviors related to offense should have a low preference level.

[0052] The style adjustment step inputs the style parameter values ​​obtained after the large model converts the user instructions into the multi-style intelligent agent as the current style parameters of the intelligent agent to execute the behavior style specified by the user.

[0053] Considering that the adjusted behavior style may not meet the user's needs at one time, fine-tuning steps may also be included: In a complex interactive environment, the same instruction may produce different expected results due to differences in user understanding. For example, in football, for the instruction "shrink defense", some users think that retreating to their own half is enough, while other users think that they need to retreat to their own penalty area. To solve this problem, this solution introduces a refined instruction adjustment mechanism that allows users to fine-tune the behavioral tendencies of AI through subsequent instructions. When the instruction does not meet the user's expectations, the user can continue to issue instructions to fine-tune some meta-behaviors of AI. We achieve this function by adding the current style parameters to the prompt words and adjusting the new style parameters based on the current style parameters. The process of this method is as follows: a) Receive the user's adjustment instruction (such as "step back a little more"). b) The large language model identifies the relevant style parameters (such as formation, position) through the designed prompt words. c) Infer the user's intention and determine the direction of parameter adjustment (such as reducing the relevant parameters). d) Select the appropriate change range according to the user's intention and tone. Since large language models are not very effective in processing precise numerical values, when it comes to adjusting the amplitude of style parameters, we do not directly let it output specific change values, but discretize them into multiple change amplitudes, such as unchanged, slightly reduced, greatly reduced, slightly increased, and greatly increased. Each change amplitude level corresponds to a Gaussian distribution. The large language model selects the corresponding change amplitude based on the user's intention and tone, and then samples a change value from the Gaussian distribution corresponding to each change amplitude and adds it to the current style parameter. This method maps the adjustment amplitude of the user's instruction to a numerical value, introduces a moderate amount of randomness, and increases the flexibility of the system.

[0054] The present invention also provides an instruction-based intelligent agent multi-style infinite adjustment system, which can be implemented by executing the process steps of the instruction-based intelligent agent multi-style infinite adjustment method, that is, those skilled in the art can understand the instruction-based intelligent agent multi-style infinite adjustment method as a preferred implementation of the instruction-based intelligent agent multi-style infinite adjustment system. The system includes: Initial module: Let the control strategy model control the performance strategy of the agent in the environment according to the style parameters, where the style parameters represent the tendency of controlling the agent to execute different performance strategies.

[0055] Instruction acquisition module: parses the natural language instructions input by the user to determine the style parameters that need to be adjusted and the corresponding numerical adjustment degree.

[0056] Style adjustment module: inputs the style parameters to be adjusted and the corresponding numerical adjustment degree into the control strategy model, so that the control strategy model controls the performance strategy of the intelligent agent in the environment according to the adjusted style parameters.

[0057] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0058] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A multi-style stepless adjustment method for intelligent agents based on instructions, characterized in that: include: Initial step: let the control strategy model control the agent's performance strategy in the environment according to the style parameters, where the style parameters represent the tendency of controlling the agent to execute different performance strategies; Instruction acquisition step: parsing the natural language instructions input by the user to determine the style parameters that need to be adjusted and the corresponding numerical adjustment degree; Style adjustment step: input the style parameters to be adjusted and the corresponding numerical adjustment degree into the control strategy model, so that the control strategy model controls the performance strategy of the intelligent agent in the environment according to the adjusted style parameters.

2. The method for infinitely adjusting multiple styles of intelligent agents based on instructions according to claim 1, characterized in that: The performance strategy includes executing a meta-behavior or a combination of meta-behaviors, and the tendency includes controlling the reward scale after the agent executes different meta-behaviors or a combination of meta-behaviors; The range of the style parameter includes [-1, 1]; When the style parameter is -1, it means that executing this meta-behavior or meta-behavior combination will obtain the opposite reward, thus forming the opposite tendency; When the style parameter is 0, it means that there is no reward for performing the meta-behavior or meta-behavior combination, thus indicating that the performance of the behavior is neither encouraged nor supported; When the style parameter is 1, it indicates that the reward for the meta-behavior or meta-behavior combination is positively increased to the maximum extent.

3. The method for infinitely adjusting multiple styles of intelligent agents based on instructions according to claim 1, characterized in that: The instruction acquisition step includes supplementing the natural language instructions input by the user into pre-constructed prompt words, inputting the supplemented prompt words into the large language model, parsing the large language model to obtain meta-behaviors related to the natural language instructions, evaluating the preference degree of each related meta-behavior based on the sentiment analysis of the natural language instructions, giving the numerical results of the corresponding style parameters and generating structured outputs.

4. The method for infinitely adjusting multiple styles of intelligent agents based on instructions according to claim 3, characterized in that: The supplemented prompt words include: scenario description, parameter type and meaning, instruction parsing example, natural language instructions and output format requirements, among which: a) Scenario description: describes the scenario in which the agent is located and the role played by the large language model; b) Parameter type and meaning description: explain the meaning of each style parameter, and explain the value type and range; c) Instruction parsing example: Give an example of parsing natural language instructions into style parameters for reference by the large model; The format of the example is: 1) Disassemble the natural language instructions, and the large language model finds the meta-behavior related to the natural language instructions through the prompt words; 2) determine the numerical degree of each relevant meta-behavior based on the intent and tone of the natural language instruction; d) Natural language instructions: insert natural language instructions input by the user; e) Output format requirements: Describe the format in which the large language model needs to be returned.

5. A multi-style infinite adjustment system for intelligent agents based on instructions, characterized in that: include: Initial module: Instruct the control strategy model to control the performance strategy of the agent in the environment according to preset style parameters, where the style parameters represent the tendency of controlling the agent to execute different performance strategies; Instruction acquisition module: parses the natural language instructions input by the user to determine the style parameters that need to be adjusted and the corresponding numerical adjustment degree; Style adjustment module: inputs the style parameters to be adjusted and the corresponding numerical adjustment degree into the control strategy model, so that the control strategy model controls the performance strategy of the intelligent agent in the environment according to the adjusted style parameters.

6. The command-based agent multi-style infinite adjustment system according to claim 5, characterized in that: The performance strategy includes executing a meta-behavior or a combination of meta-behaviors, and the tendency includes controlling the reward scale after the agent executes different meta-behaviors or a combination of meta-behaviors; The range of the style parameter includes [-1, 1]; When the style parameter is -1, it means that executing this meta-behavior or meta-behavior combination will obtain the opposite reward, thus forming the opposite tendency; When the style parameter is 0, it means that there is no reward for performing the meta-behavior or meta-behavior combination, thus indicating that the performance of the behavior is neither encouraged nor supported; When the style parameter is 1, it indicates that the reward for the meta-behavior or meta-behavior combination is positively increased to the maximum extent.

7. The command-based agent multi-style infinite adjustment system according to claim 5, characterized in that: The instruction acquisition module includes supplementing the natural language instructions input by the user into pre-built prompt words, inputting the supplemented prompt words into the large language model, parsing the large language model to obtain meta-behaviors related to the natural language instructions, evaluating the preference degree of each related meta-behavior based on the sentiment analysis of the natural language instructions, giving the numerical results of the corresponding style parameters and generating structured outputs.

8. The command-based agent multi-style stepless adjustment system according to claim 7, characterized in that: The supplemented prompt words include: scenario description, parameter type and meaning, instruction parsing example, natural language instructions and output format requirements, among which: a) Scenario description: describes the scenario in which the agent is located and the role played by the large language model; b) Parameter type and meaning description: explain the meaning of each style parameter, and explain the value type and range; c) Instruction parsing example: Give an example of parsing natural language instructions into style parameters for reference by the large model; The purpose of the examples described is to: 1) The large model is used to decompose the natural language instructions. The large language model uses the prompt words to find the meta-behaviors related to the natural language instructions. 2) Let the big model determine the numerical degree of each relevant meta-behavior based on the intent and tone of the natural language instruction; d) Natural language instructions: insert natural language instructions input by the user; e) Output format requirements: Describe the format in which the large language model needs to be returned.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the instruction-based agent multi-style infinite adjustment method described in any one of claims 1 to 4 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the instruction-based agent multi-style infinite adjustment method described in any one of claims 1 to 4 are implemented.

Images