A Diverse Interaction Agent Construction System Based on Offline Learning

Abstract

The present invention discloses a diversified interactive agent construction system based on offline learning. The system includes: an offline loop system and an agent interaction system. The offline loop system includes a process splitting unit, a self-evolution unit, a storage processing unit, and a model calling unit. The agent interaction system includes an information sharing unit and a collaboration unit. The offline loop system uses an offline reinforcement algorithm to process the basic functions of an agent under the condition of a single agent's independence, including thinking function, perception function, and execution function. The agent interaction system uses an interactive adjustment learning algorithm to enhance the information in the offline loop system, and uses a distance contraction adjustment algorithm to improve the collaboration efficiency of the agent interaction system, and conducts information interaction with other single agents to effectively complete complex tasks. The present invention realizes efficient learning of agents in an offline state and improves the data processing efficiency of diversified agents in completing complex tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-agent interaction, and particularly to a diversified interaction agent construction system based on offline learning. Background Art

[0002] An agent is a core concept in the field of artificial intelligence. It refers to an entity that can perceive the environment and make decisions based on the perception results to achieve specific goals. An agent can be in the form of software, such as an automated trading system, or a physical entity, such as an autonomous navigation robot. An offline agent, also known as batch reinforcement learning, is a variant of reinforcement learning. It mainly studies how to use a pre-collected large-scale static data set to train a reinforcement learning agent without any form of online interaction and exploration. A multi-agent system is a computational system composed of multiple agents that interact, communicate, cooperate, or compete with each other in a shared environment to complete complex tasks that are difficult for a single agent to complete.

[0003] Single-agent and multi-agent each have their own advantages and are suitable for different vertical fields. The reinforcement learning principle of a single agent is based on Markov decision-making. Briefly speaking, it can be divided into a state set S, an action set A, and a reward R. The state and reward at the next moment are only related to the action at the previous moment and have nothing to do with the states earlier than that. Its model principle is to let the agent learn by trial and error. If a certain strategy can obtain a reward, the strategy for the agent to generate this behavior will be strengthened. Its purpose is to act in a single environment and obtain the maximum reward as much as possible. The application fields are also relatively extensive at present. For example, the training of continuous actions in a racing game: controlling actions such as the steering wheel, accelerator, and brake can be decided by algorithms such as DDPG, A3C, and PPO. The training of some discrete actions, such as the Go agent AlphaGo, can be decided by algorithms such as Q-Learning. The decision-making of a multi-agent is not only related to its own actions but also related to the actions of other agents in the system. There will be more than two agents in a multi-agent system, and they generally have a cooperative or competitive relationship. Such a model is called a Markov game, whose state transition conforms to Markov decision-making and the relationship conforms to game theory. In a multi-agent model, the goal of each agent is to find an optimal strategy to maximize its long-term cumulative reward in any state. Due to reasons such as its more complex model and more interference factors, there are currently few commercial products for multi-agent models.

[0004] The intelligent agent system disclosed in Chinese Invention Patent CN117789729A actively determines topics on the system side, identifies the interlocutors and conducts conversations with them, thereby making the conversations of the occupants in the vehicle cabin smoother and creating a pleasant space. The microphone installed in the vehicle cabin is used to collect the voice information of the occupants. The control unit enables the analysis unit to analyze the speech content included in the collected voice information, determines the listener, retrieves the latest events from the occupants' mobile terminals and determines topics, and outputs the voice information from the speaker. In this way, the system side actively determines the listener, determines topics based on the latest events and outputs them. This invention is mainly applied to the field of vehicle voice interaction, but the idea can also be borrowed in other intelligent agent fields. However, the idea of this invention is limited to intelligent agents in the offline state. In fact, it is a transplantation of basic artificial intelligence and cannot be widely applied to most complex scenarios.

[0005] The health intelligent all-in-one machine disclosed in Chinese Invention Patent CN109859843B includes a health detection module, a main controller, a power supply module, a face recognition module, a voice module, a data storage module and a display module; the health detection module is electrically connected to the main controller; the main controller is electrically connected to each module, analyzes and processes the health data of the user, and controls the signal transmission between each module; the power supply module is electrically connected to the main controller and each module; the face recognition module includes a camera, an identity recognition unit and an emotion recognition unit, and the camera is electrically connected to the identity recognition unit and the emotion recognition unit; the voice module includes a voice input module, a voiceprint recognition unit and a voice recognition unit; the voice input module is electrically connected to the voiceprint recognition unit, the voice recognition unit and the voice broadcast unit; the data storage module is used to store user data, and the display module is used to display the interface. This invention is a relatively simple human-computer interaction device system developed by Yuyue Medical Equipment Company for personal user health examinations. The main problem this invention solves is the human-computer friendliness, and there is no further optimization for the functional expansion in actual use and the data judgment accuracy.

[0006] In the existing invention patents or actual products, only the construction of intelligent agents for single products or single functions in the same industry and field has been described. In fact, there are many relatively complex problems in the actual application of intelligent agents that require an intelligent agent performance improvement system that comprehensively includes offline and online processing of diverse problems, such as learning evaluation, medical health assessment, agricultural product futures grading, etc. Therefore, the existing intelligent agent processing system is not ideal for dealing with complex problems. Summary of the Invention

[0007] The present invention provides a diversified interaction intelligent agent construction system based on offline learning to solve the technical problems that existing intelligent agents rely on online construction during learning, cannot perform diversified interactions with multiple preset functions, and have weak interaction functions between intelligent agents, low efficiency and low accuracy.

[0008] To solve the above technical problems, the present invention provides the following technical solutions:

[0009] The present invention provides a diversified interactive agent construction system based on offline learning, including:

[0010] An offline loop system and an agent interaction system, the offline loop system includes a process splitting unit, a self-evolution unit, a storage processing unit, and a model calling unit, and the agent interaction system includes an information sharing unit and a cooperation unit;

[0011] The offline loop system uses an offline reinforcement algorithm to process the basic functions of a single agent under independent conditions, and the basic functions of the agent include a thinking function, a perception function, and an execution function;

[0012] The agent interaction system uses an interaction adjustment learning algorithm to enhance the information in the storage processing unit of the offline loop system, and uses a distance contraction adjustment algorithm to improve the cooperation efficiency of the agent interaction system, and can perform information interaction with other single agents to effectively complete complex tasks.

[0013] Further, the offline reinforcement algorithm regards the basic functions of a single agent under independent conditions as an offline function trajectory;

[0014] Serializes and models the offline function trajectory, and marks the agent action feedback of each time series;

[0015] Through the offline MARL data-driven training paradigm, uses the agent action feedback to predict the offline function trajectory to obtain a regression offline function trajectory;

[0016] Through the offline function trajectory After serialization modeling, a regression offline function trajectory is obtained Realize the basic functions of the single agent under independent conditions.

[0017] Further, the interaction adjustment learning algorithm enhances the stability of the agent through unified training and distributed implementation of the basic functions of the agent;

[0018] The interaction adjustment learning algorithm includes the steps of:

[0019] S1 Normalize and reconstruct the regression offline function trajectory;

[0020] S2 Use the hidden state analysis method to establish a self-consistent model;

[0021] S3 Improve the cooperation efficiency of the agent interaction system through the distance contraction adjustment algorithm.

[0022] Further, when interacting with other single agents, the agent error is reduced by introducing a self-supervised pre-training transformation model;

[0023] The self-supervised pre-training transformation model fits the normalized regression offline functional trajectories G of multiple agents through the MAPPO algorithm, improving the accuracy of completing complex tasks.

[0024] The beneficial effects brought by the technical solution provided by the present invention at least include:

[0025] The offline loop system uses an offline reinforcement algorithm to process the basic functions of a single agent under independent conditions, and the basic functions of the agent include thinking function, perception function, and execution function. The efficient learning of the agent in the offline state is realized.

[0026] Using the offline reinforcement algorithm, the basic functions of a single agent under independent conditions are regarded as an offline functional trajectory, and the offline functional trajectory is serially modeled, and the agent action feedback of each time series is marked. The standardized description of a single agent is completed, which is convenient for data processing of diverse agents in subsequent improvement steps.

[0027] Using the interactive adjustment learning algorithm to enhance the information in the storage and processing unit in the offline loop system, enhancing the agent data in the offline state, and improving the data processing efficiency of diverse agents.

[0028] Using the distance contraction adjustment algorithm to improve the cooperation efficiency of the agent interaction system, interacting with other single agents, and improving the efficiency of completing complex tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic structural diagram of a system for constructing diverse interactive agents based on offline learning;

[0031] Figure 2 It is a flowchart of the interactive adjustment learning algorithm;

[0032] Figure 3 It is a schematic diagram of the distance contraction adjustment algorithm;

[0033] Figure 4 It is a schematic structural diagram of diverse agents for medical teaching. DETAILED DESCRIPTION OF THE INVENTION

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the accompanying drawings. Embodiment

[0035] This embodiment provides a system for constructing a diverse interactive agent based on offline learning.

[0036] Please refer to Figure 1 the schematic structural diagram of the system for constructing a diverse interactive agent based on offline learning provided by the embodiment of the present invention as shown.

[0037] This system includes an offline loop system and an agent interaction system. The offline loop system and the agent interaction system. The offline loop system includes a process splitting unit, a self-evolution unit, a storage and processing unit, and a model calling unit. The agent interaction system includes an information sharing unit and a cooperation unit;

[0038] The offline loop system uses an offline reinforcement algorithm to process the basic functions of a single agent under independent conditions. The basic functions of the agent include a thinking function, a perception function, and an execution function;

[0039] The agent interaction system uses an interactive adjustment learning algorithm to enhance the information in the storage and processing unit of the offline loop system, and uses a distance contraction adjustment algorithm to improve the cooperation efficiency of the agent interaction system. It can perform information interaction with other single agents and effectively complete complex tasks.

[0040] Specifically, the offline loop system is a diverse unit set for an agent to achieve preset functions without information interaction with other agents. The diverse unit set is composed of different types of unit sets. The unit set includes a process splitting unit, a self-evolution unit, a storage and processing unit, and a model calling unit;

[0041] The unit set can independently complete the preset function, and the diverse unit set can complete multiple preset functions;

[0042] The process splitting unit decomposes complex functions into a set of simple functions;

[0043] The self-evolution unit improves the implementation effect of the preset function of the unit set by evaluating the unit set to achieve the preset function;

[0044] The storage and processing unit uses the accumulated existing unit set to guide the implementation of the preset function;

[0045] The model calling unit can call an external model to enhance the function of the unit set. The external model is a database and a plugin library that assist in implementing the existing functions of the current unit set.

[0046] Specifically, the agent interaction system improves the performance of agents through the collaboration of different agents;

[0047] The information sharing unit can share the diverse unit set in the offline loop system with other agents;

[0048] The collaboration unit improves the ability of multiple agents to complete preset functions through the collaboration of multiple agents.

[0049] Specifically, the thinking function enables the agent to split complex tasks into simpler subtasks and continuously analyze the results to achieve the ability of improvement and thinking function;

[0050] The perception function realizes external input through text input, image capture, and speech recognition;

[0051] The execution function can call an external model to assist the agent in realizing the preset function.

[0052] Specifically, the offline reinforcement algorithm regards the basic function under the independent condition of a single agent as an offline function trajectory , expressed as:

[0053] ;

[0054] Among them, represents the observation state, represents the global state, represents the instruction, is the reinforcement value, T represents the total length of the time step, the i in is the agent number;

[0055] Serializes and models the offline function trajectory, marking the agent action feedback of each time series , expressed as:

[0056] ;

[0057] Among them, T represents the total length of the time step, e represents the natural constant, represents the agent action feedback of the previous time step;

[0058] Through the offline MARL data-driven training paradigm, use the agent action feedback to predict the offline function trajectory and obtain the regression offline function trajectory , expressed as:

[0059] ;

[0060] Among them, is the observation state of the agent at time step t, is the global state of the agent at time step t, and i is the agent number, is the state of the agent at time step t, is the reward of the agent at time step t;

[0061] Through the offline functional trajectory The regression offline functional trajectory is obtained through serialized modeling Realize the basic functions of the agent under the condition of a single independent agent.

[0062] Specifically, the interactive adjustment learning algorithm enhances the stability of the agent through unified training and distributed implementation of the basic functions of the agent;

[0063] Please refer to Figure 2 the flowchart of the interactive adjustment learning algorithm method shown.

[0064] The interactive adjustment learning algorithm includes the steps:

[0065] S1 Normalize and reconstruct the regression offline functional trajectory;

[0066] S2 Use the hidden state analysis method to establish a self-consistent model;

[0067] S3 Improve the cooperation efficiency of the agent interaction system through the distance contraction adjustment algorithm.

[0068] Specifically, the action feedback of the agent for each time series , is expressed as:

[0069] ;

[0070] where, represents the normalized action feedback of the previous time series, T represents the total length of the time step, the summation symbol, representing the sum of all terms from time step t = 1 to time step t = T;

[0071] The normalized policy C, is expressed as:

[0072] ;

[0073] where, represents the policy function, t represents the current time step t, represents the execution action selected at time step t, K represents the time series sampling degree, represents the action feedback of the past K time steps of the current time step t, represents the normalized feedback of the past K time steps of the current time step t, Represents the observed state of the past K time steps of the current time step t, Represents the executed actions of the past K time steps of the current time step t;

[0074] For the normalized regression offline functional trajectory G after normalization reconstruction, it is expressed as:

[0075] ;

[0076] Completes the offline functional trajectory The regression offline functional trajectory is obtained through serialized modeling The process is normalized to obtain the normalized regression offline functional trajectory, where Is the normalized observed value, Represents the executed action selected at time step t, and i is the agent number.

[0077] Specifically, the hidden state analysis method first samples other agents n to obtain the sampling trajectories of other agents , which is expressed as:

[0078] ;

[0079] Among them, v represents the sampling degree, Represents the sampling state of agent n at time step t.

[0080] Specifically, the probability distribution is extracted from the sampling trajectories of other agents by using the nearest neighbor algorithm. When extracting the probability distribution, the shrinkage adjustment algorithm is used. The shrinkage adjustment algorithm is to calculate the difference between the sampling trajectories of other agents and the normalized regression offline functional trajectory G to obtain the normalized regression offline functional shrinkage trajectory of the time series with the smallest difference , which is expressed as:

[0081] ;

[0082] Among them, Represents the vector of the normalized regression offline functional trajectory of the index, Represents all the vectors of the normalized regression offline functional trajectories taken, Represents the minimum value taken, Represents The estimated trajectory vector of, j represents the traversal index, and k represents the nearest neighbor index;

[0083] Among them, when When taking all the vectors of the normalized regression offline functional trajectories, the exponential moving average method is used to improve efficiency.

[0084] Please refer toFigure 3 Schematic diagram of the distance contraction adjustment algorithm method shown

[0085] Specifically, when interacting with other single agents, the error of the agent is reduced by introducing a self-training pre-transformation model

[0086] The self-training pre-transformation model fits the normalized regression offline functional trajectory G of multiple agents through the MAPPO algorithm, improving the accuracy of completing complex tasks

[0087] Second embodiment

[0088] This embodiment provides an offline loop system and an agent interaction system, which are applied in the field of agent medical teaching

[0089] Please refer to Figure 4 Schematic diagram of the diverse agent structure for medical teaching shown

[0090] In this embodiment, the diverse agents include a medical teaching information management agent, a collaborative interaction agent, a student assistant agent, and a medical teaching exploration agent

[0091] In the field of medical education, medical teaching information management is a core link, which covers the sharing, acquisition, and composition of medical knowledge between teachers and students

[0092] It should be noted that medical school teachers often have difficulty accurately grasping the detailed situation of each medical student's mastery of medical knowledge. Although medical students have a certain degree of self-analysis and evaluation of their learning situation, due to the influence of experience and knowledge reserve, it is difficult for medical students to accurately master the real situation of medical learning. The diverse interaction agent system of medical knowledge and rich teaching resources, as an educational platform, can significantly promote the management and dissemination of knowledge

[0093] The specific application forms of the medical teaching information management system in medical teaching include a medical knowledge agent, a medical analysis agent, a medical test paper agent, and a medical homework agent

[0094] The medical knowledge agent can automatically collect the questions of medical students in the medical field, obtain answers from the medical dataset, classify the questions and store them in the question set, and continuously update the dataset and question set; the medical analysis agent automatically analyzes relevant medical materials based on the status of medical students; the medical test paper agent selects questions from the dataset and automatically forms a medical test paper according to the goals set by medical teachers in advance, and forms a test paper for students' personalization; the medical homework agent automatically generates homework according to the learning progress of the target medical student, the goals preset by teachers, the learning progress of students, exam scores, etc., and gives reminders. By reasonably using the database, the medical teaching information management agent can effectively improve the learning ability and efficiency of medical students

[0095] It should be noted that the intelligent agent for medical teaching information management can be used offline as the core intelligent agent. Being offline does not mean disconnecting from the network, but rather temporarily not interacting with other intelligent agents.

[0096] I. Intelligent Agent for Medical Teaching Information Management

[0097]

[0098] The specific application forms of the collaborative interaction intelligent agent in medical teaching include the medical group intelligent agent and the medical evaluation intelligent agent.

[0099] The medical group intelligent agent can automatically group students based on multiple factors such as the learning progress of medical students, teacher grouping, the learning progress of medical students, exam scores, and the composition of exam scores, grouping students with similar learning progress and the targeted teachers together to achieve efficient teaching and learning. The medical evaluation intelligent agent can conduct all-round evaluations targeted at the learning situation of medical students, provide suggestions to teachers and students respectively based on the learning situation, and collect feedback on the later learning progress and effects of students to improve the abilities of students and teachers.

[0100] II. Collaborative Interaction Intelligent Agent

[0101]

[0102] The specific application forms of the student assistant intelligent agent in medical teaching include the medical self-test intelligent agent, the medical auxiliary reminder intelligent agent, and the medical teaching control intelligent agent.

[0103] The medical self-test intelligent agent can help medical students generate self-tests during self-study to help them quickly master the learning situation and progress. The medical auxiliary intelligent agent helps medical students formulate study plans and assists teachers in formulating learning progress plans and controlling students. The medical teaching control intelligent agent has functions such as controlling account permissions, managing learning materials, exam materials, and data verification.

[0104] III. Student Assistant Intelligent Agent

[0105]

[0106] The specific application forms of the medical teaching exploration intelligent agent in medical teaching include the medical courseware collection intelligent agent and the medical teaching intelligent agent.

[0107] The intelligent body for collecting medical courseware can make full use of the learning situations of existing medical students and the teaching situations of teachers to automatically integrate the publicly available teaching resources on the network, form a teaching resource library, and provide it to teachers and students for use after review. The intelligent body for medical teaching can compare and integrate the obtained teaching resource library with the existing data and teacher resource data to generate new artificial intelligence courseware to assist students in extra learning or for teachers as a reference.

[0108] IV. Intelligent body for medical teaching exploration

[0109]

[0110] For any intelligent body in the medical teaching information management system, the functions implemented are regarded as an offline function trajectory, serialized modeling is performed, the agent action feedback of each time series is marked, and through the offline MARL data-driven training paradigm, the offline function trajectory is predicted using the agent action feedback to obtain a regression offline function trajectory. The regression offline function trajectory is obtained through serialized modeling of the offline function trajectory to realize the basic functions of the intelligent body under the condition of a single intelligent body being independent.

[0111] In addition, it should be noted that the present invention can be provided as a method, device, or computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0112] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0113] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the functions in Figure 1 one process or multiple processes and / or blocksFigure 1 The functions specified in one or more boxes. These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes and / or boxes Figure 1 One process or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0114] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0115] Finally, it should be noted that the above is the preferred embodiment of the present invention. It should be pointed out that although the preferred embodiments of the present invention have been described, for those skilled in the art of this technology, once the basic creative concept of the present invention is known, without departing from the principle described in the present invention, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A system for constructing a diversified interactive agent based on offline learning, characterized in that Including: An offline loop system and an agent interaction system. The offline loop system includes a process splitting unit, a self-evolution unit, a storage and processing unit, and a model invocation unit. The agent interaction system includes an information sharing unit and a collaboration unit; The offline loop system uses an offline reinforcement algorithm to process the basic functions of a single agent under independent conditions. The basic functions of the agent include a thinking function, a perception function, and an execution function; The agent interaction system uses an interaction adjustment learning algorithm to enhance the information in the storage and processing unit of the offline loop system, and uses a distance contraction adjustment algorithm to improve the collaboration efficiency of the agent interaction system. It can perform information interaction with other single agents and effectively complete complex tasks; The diverse interaction agents include a medical teaching information management agent, a collaborative interaction agent, a student assistant agent, and a medical teaching exploration agent; The agent interaction system uses an interaction adjustment learning algorithm to enhance the information in the storage and processing unit of the offline loop system, and uses a distance contraction adjustment algorithm to improve the collaboration efficiency of the agent interaction system; where: The interaction adjustment learning algorithm enhances the stability of the agent through unified training and distributed implementation of the basic functions of the agent; The interaction adjustment learning algorithm includes the steps: S1 Normalize and reconstruct the regression offline function trajectory; S2 Use the hidden state analysis method to establish a self-consistent model; S3 Improve the collaboration efficiency of the agent interaction system through the distance contraction adjustment algorithm; Agent action feedback for each time series , expressed as: ； Among them, represents the normalized agent action feedback at the previous time step, the summation symbol, indicating the sum of all terms from time step t = 1 to time step t = T, where T represents the total length of the time steps; The normalization strategy C, expressed as: ； Among them, represents the policy function, where t directly represents the current time step t, represents the execution action selected at time step t, and K represents the time series sampling degree, represents the action feedback of the past K time steps of the current time step t, represents the normalized feedback of the past K time steps of the current time step t, represents the observation state of the past K time steps of the current time step t, represents the execution action of the past K time steps of the current time step t; For the normalized regression offline function trajectory G after normalization and reconstruction, expressed as: ； Completion of the offline function trajectory The regression offline function trajectory is obtained through serialized modeling The process is normalized to obtain the normalized regression offline function trajectory, where is the normalized observation value represents the execution action selected at time step t, and i is the agent number; The hidden state analysis method first samples other agents n to obtain the sampling trajectories of other agents , which is expressed as: ； where \(v\) represents the sampling degree, represents the sampling state of agent \(n\) at time step \(t\); The sampling trajectories of other agents are processed by the nearest neighbor algorithm to extract the probability distribution. When extracting the probability distribution, the shrinkage adjustment algorithm is used. The shrinkage adjustment algorithm calculates the sampling trajectories of other agents and compares them with the normalized regression offline functional trajectory G to obtain the normalized regression offline functional shrinkage trajectory of the time series with the smallest difference , which is expressed as: ； Among them, represents the normalized regression offline functional trajectory vector of the index, represents all the normalized regression offline functional trajectory vectors obtained, represents taking the minimum value, represents the estimated trajectory vector of, j represents the traversal index, and k represents the nearest neighbor index; Among them, when the exponential moving average method is used to improve efficiency when obtaining all normalized regression offline function trajectory vectors; It can perform information interaction with other single agents and effectively complete complex tasks; where: When performing information interaction with other single agents, the agent error is reduced by introducing a spontaneous pre-training conversion model; The spontaneous pre-training conversion model fits the normalized regression offline function trajectories G of multiple agents through the MAPPO algorithm to improve the accuracy of completing complex tasks.

2. The diversified interactive agent construction system based on offline learning according to claim 1, characterized in that The offline loop system includes a process splitting unit, a self-evolution unit, a storage and processing unit, and a model invocation unit; where, The offline loop system is a diverse unit set for an agent to achieve preset functions without information interaction with other agents. The diverse unit set is composed of different types of unit sets. The unit set includes a process splitting unit, a self-evolution unit, a storage and processing unit, and a model invocation unit; The unit set can independently complete preset functions, and the diverse unit set can complete multiple preset functions; The process splitting unit can decompose complex functions into a set of simple functions; The self-evolution unit improves the implementation effect of the preset functions of the unit set by evaluating the unit set to achieve the preset functions; The storage and processing unit uses the accumulated existing unit sets to guide the implementation of the preset functions; The model invocation unit can invoke external models to enhance the functions of the unit set. The external models are databases and plugin libraries that assist in implementing the existing functions of the current unit set.

3. The diversified interaction intelligent agent construction system based on offline learning according to claim 1, wherein The agent interaction system includes an information sharing unit and a collaboration unit; where: The agent interaction system improves the performance of agents through the collaboration of different agents; The information sharing unit can share the diverse unit set in the offline loop system with other agents; The collaboration unit improves the ability of multiple agents to complete preset functions through the collaboration of multiple agents.

4. The diverse interaction intelligent agent construction system based on offline learning according to claim 1, characterized in that The basic functions of the agent include thinking function, perception function, and execution function; among them: The thinking function enables the agent to split complex tasks into relatively simple subtasks and continuously analyze the results to achieve the ability of improvement and thinking function; The perception function realizes external input through text input, image capture, and voice recognition; The execution function can call external models to assist the agent in realizing preset functions.

5. The diversified interactive agent construction system based on offline learning as described in claim 1, characterized in that, The offline loop system uses an offline reinforcement algorithm to process the basic functions of the agent under the condition of a single agent's independence; among them: The offline reinforcement algorithm regards the basic functions under the independent condition of a single agent as an offline function trajectory , which is expressed as: ； Among them, represents the observation state, represents the global state, represents the instruction, is the reinforcement value, and T represents the total length of the time step, where i in it is the agent number; Serialize and model the offline function trajectory, and mark the agent action feedback for each time series , expressed as: ； where T represents the total length of the time step, and e represents the natural constant, denotes the agent action feedback from the previous time step; Through the offline MARL data-driven training paradigm, using the agent action feedback Predict the offline functional trajectory to obtain the regression offline functional trajectory , expressed as: ； Among them, is the observation state of the agent at time step t, is the global state of the agent at time step t, i is the agent number, is the state of the agent at time step t, is the reward of the agent at time step t; Through the offline function trajectory The regression offline function trajectory is obtained through serialized modeling Realize the basic functions of the single agent under independent conditions

Images