Multi-agent internal biochemical model training method and system

Abstract

The invention belongs to the technical field of artificial intelligence, and provides a multi-agent internal biochemical model training method and system, and the method comprises the steps: sampling a task, extracting the embedded representation of each Agent and the meta-feature vector of the execution task, generating the cooperation probability of each pair of Agents through combining the embedded representation of each Agent, the meta-feature vector of the execution task and a self-supervision graph neural network, and carrying out the training of an internal biochemical model of each Agent. Forming an Agent cooperation plan matrix; and allocating corresponding communication resources according to the Agent cooperation plan matrix, performing asynchronous message interaction on the interaction bus by each Agent based on the allocated communication resources to obtain output distribution of each Agent, and mapping the output distribution of each Agent into a global reward vector and a target of a final task. The end-to-end differentiable multi-agent collaborative optimization is realized, and the generalization ability and the resource utilization rate of the model are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a multi-agent endogenous model training method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Existing multi-agent frameworks (such as AutoGen and LangGraph) typically use APIs to call upon multiple role agents to complete complex tasks during the inference phase. The collaboration logic is explicitly controlled by external orchestration scripts, separating training from inference and making it difficult for the model to truly "learn" collaboration strategies. This external dependency results in sluggish system response to dynamic tasks and prevents the internalization of collaboration knowledge through end-to-end optimization, limiting the model's generalization capabilities.

[0004] In 2024, OpenAI o1 first incorporated Chain-of-Thought (CoT) capabilities into the pre-training process. In 2025, Grok 4 Heavy went a step further, directly enabling multiple agents to engage in debate and self-checking during training, a practice known in the industry as "multi-agent intramuralization." For example, Grok 4 Heavy leveraged reinforcement learning to conduct multi-agent debates at pre-training scale, significantly improving the model's performance on abstract reasoning tasks such as the ARC-AGI benchmark. This approach demonstrates that embedding collaborative capabilities within the model can significantly improve complex reasoning and cross-domain generalization. However, existing multi-agent intramuralization approaches (such as MAPoRL and MAS-GPT) still rely on a fixed number of agents and static roles, lacking task-oriented autonomous orchestration and dynamic evolution mechanisms, and remain unaddressed for key issues such as cross-agent gradient alignment and computational resource flexibility. While these approaches incorporate multi-agent interaction during training, the collaborative topology is often predefined and incapable of adapting to task complexity, leading to performance bottlenecks in long-context or multimodal tasks. Summary of the Invention

[0005] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a multi-agent endogenous model training method and system, which realizes end-to-end differentiable multi-agent collaborative optimization. Compared with the existing external framework, it significantly improves the generalization ability and resource utilization of the model, and supports cross-modal expansion and bootstrap evolution.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A first aspect of the present invention provides a multi-agent endogenous model training method, comprising the following steps: Perform training preparation and training initialization; Sampling tasks, extracting the embedded representation of each agent and the meta-feature vector of the task, and combining the embedded representation of each agent, the meta-feature vector of the task, and the self-supervised graph neural network to generate the collaboration probability of each pair of agents to form the agent collaboration plan matrix; According to the agent collaboration plan matrix, corresponding communication resources are allocated. Based on the allocated communication resources, each agent performs asynchronous message interaction on the interaction bus to obtain the output distribution of each agent, and then maps the output distribution of each agent into the global reward vector and the goal of the final task.

[0007] Furthermore, training preparation and training initialization are carried out, including initializing the main model and each agent in the hardware environment, initializing the meta-scheduler, and training configuration and determining the security policy.

[0008] Furthermore, the calculation formula for the collaboration probability of each pair of agents is: , , in, express arrive The probability of collaboration, is a normalization function that ensures the sum of probabilities is 1, Attention For the attention mechanism, calculate the similarity between task features and role embeddings, is the meta-feature vector of task T, for Embedded representation of the role, for Embedded representation of roles.

[0009] Furthermore, after generating the collaboration probability of each pair of agents, a differentiable function is used to map the collaboration probability of each pair of agents to [0,1].

[0010] Furthermore, the allocation of corresponding communication resources according to the Agent collaboration plan matrix includes: determining the total available communication resources, including the total bandwidth or the number of channels; allocating resources according to the collaboration probability, including allocating resources to each pair of Agents in proportion to their collaboration probability, and ultimately ensuring that the sum of the allocated resources does not exceed the total amount.

[0011] Furthermore, when each agent performs asynchronous message interaction on the interactive bus based on the allocated communication resources to obtain the output distribution of each agent, they communicate through a high-throughput message queue and a two-way memory cache mechanism.

[0012] Furthermore, at each step of interaction, a Transformer-based encoder is used to implement differentiable propagation, and the interactive message is represented as: , in, express Send to message vector; for The hidden state of for The hidden state of It is a multi-layer perceptron, used for nonlinear transformation; It is a vector concatenation operation.

[0013] Furthermore, contrastive learning is introduced and KL divergence is used to minimize the multi-agent output distribution to obtain the global reward vector, which is expressed as: , in, represents the global reward vector; Reward candidates r Seek maximization; Represents the sum of conditional probability logarithms, measuring the final answer In the reward r Likelihood under; β To balance hyperparameters; represents the joint output distribution of multiple agents, which is the product of the output distribution of each agent. ,in, is based on the reward vector r Boltzmann distribution, ,in, Agent Output The feature map vector of is the normalization constant, , Q represents the preset reference distribution (such as uniform distribution), Represents the output distribution P With the reference distribution Q KL divergence of , used for regularization.

[0014] Furthermore, the total loss function of training is: , , in, Indicates the total loss; represents the task-specific loss, represents the weight hyperparameter of the alignment loss, The weight hyperparameter representing the scheduling loss; represents the scheduler optimization loss, represents the alignment loss; represents the matrix trace operation; Represents the covariance matrix, which calculates the covariance between gradient vectors; arrive Indicates the parameters of each Agent The gradient vector of the global objective function of the multi-agent system Relative to each agent Parameters Find the partial derivative to get .

[0015] A second aspect of the present invention provides a multi-agent endogenous model training system, comprising: Training initialization module, used for training preparation and training initialization; The multi-agent intra-agent biochemical module is used for collaborative analysis of sampling tasks by agents, extracting the embedded representation of each agent and the meta-feature vector of the task, and combining the embedded representation of each agent, the meta-feature vector of the task, and the self-supervised graph neural network to generate the collaboration probability of each pair of agents to form the agent collaboration plan matrix; According to the agent collaboration plan matrix, corresponding communication resources are allocated. Based on the allocated communication resources, each agent performs asynchronous message interaction on the interaction bus to obtain the output distribution of each agent, and then maps the output distribution of each agent into the global reward vector and the goal of the final task.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention embeds the collaborative behavior of complex intelligent agents into the pre-training and fine-tuning process of the model. The model can autonomously learn and optimize its internal decision-making logic, dynamic role allocation strategy and gradient consistency optimization across agents, thereby significantly improving its ability to handle complex reasoning tasks, achieve cross-domain knowledge generalization and efficiently utilize computing resources, solving the problems of separation between training and reasoning and inability to internalize collaborative strategies; at the same time, static role design and non-differentiable interaction limit the model's self-organizing ability and gradient optimization efficiency.

[0017] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0019] Figure 1 This is a flow chart of a multi-agent endogenous model training method provided by an embodiment of the present invention; Figure 2 This is a block diagram of a multi-agent endogenous model training system provided by an embodiment of the present invention; Figure 3 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0022] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0023] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, autonomous driving, drones, robots, smart medical care, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0024] The solutions provided in the embodiments of this application may involve related technologies such as computer vision and machine learning of artificial intelligence, which are specifically illustrated by the following embodiments.

[0025] In the embodiments provided in this application, the multiple agents included in the agent system can be in the form of hardware devices or software programs.

[0026] When the agent is a hardware device, it may include a processor, memory, and other components. The memory is used to store data related to the agent's processing, such as program code related to the agent's processing. The processor can load and process the data stored in the memory, thereby implementing the method for determining the agent's processing provided in this application. In addition, the agent may also include a communication component for communicating with other agents in the multi-agent system to enable data transmission between agents. When the agent is a hardware device, the agent may be a drone, robot, vehicle, etc.

[0027] When the agent is a software program, the computer device running the agent may include a processor, a memory, etc. The memory is used to store data related to the agent processing, such as program code related to the agent. The processor can load and process the data stored in the memory, thereby implementing the method for determining the agent processing provided in this application. Multiple agents in a multi-agent system can run on the same computer device. The computer device can start a corresponding processing process for each agent to run the corresponding agent, and can realize data transmission between agents based on communication between processes. When the agent is a software program, the agent can be an AI game character, etc.

[0028] In a multi-agent system, each agent resides in the same environment and can collaborate with each other to perform assigned tasks within that environment. During task execution, the agents determine the environmental characteristics of their surroundings based on a pre-defined decision cycle. The policy network then calculates these characteristics to determine the processing required of the agents during each decision cycle. For example, if the agents are soccer robots, the playing field they occupy is the environment in which each agent resides. Winning the match is the assigned task for the robots, and the processing performed by the robots during each decision cycle may include executing different designated actions.

[0029] The technical problems that the current multi-agent endogenous model training method needs to solve include: 1. Existing methods rely on external scripts, which limits the model's adaptability to unknown tasks. How can we achieve "self-organization" of the intelligent agent during the training phase rather than hard-coding manual scripts? 2. In traditional multi-agent frameworks, interaction signals are often non-differentiable, leading to interrupted gradient propagation and inability to optimize end-to-end. How can we effectively transform the high-dimensional signals generated by multi-agent interactions into differentiable gradients for the main model and its sub-models? 3. Multi-agent systems are prone to resource waste and gradient conflicts, affecting training stability. How can we maintain parallel efficiency and stable convergence on large-scale clusters with limited computing power? This paper proposes an "Endogenized-Multi-Agent Training (EMAT)" framework that innovatively embeds multi-agent collaboration into the training loop. This framework achieves end-to-end optimization through self-supervised learning and differentiable operators, overcoming the limitations of static roles and external orchestration. This framework achieves end-to-end differentiable multi-agent collaborative optimization.

[0030] Figure 1 A flow chart of a multi-agent endogenous model training method is shown in Figure 1, this embodiment includes the following steps: Step 1: Perform training preparation and training initialization; In this embodiment, training preparation and initialization specifically include initializing the main model and each agent in the hardware environment, initializing the meta-scheduler, and training configuration and determining the security policy; Specifically, the hardware environment of this embodiment adopts a starting scale of 512×H200 GPUs and adopts a hybrid strategy of tensor parallelism + pipeline parallelism; Initializing the main model in the hardware environment includes loading 70B-level base weights, such as Llama 3; Initialize each Role-Specialized Agent, including the shared vocabulary, and use the LoRA-Adapter to adjust 2% to 3% of the parameters, supporting hot swapping. Role-specialized agents are intelligent agents designed or trained to assume specific roles and perform specific tasks in artificial intelligence (especially agent systems based on large language models). For convenience, we will simply refer to them as agents.

[0031] Initialize the Meta-Scheduler as a self-supervised graph neural network (GNN) to generate a differentiable collaborative plan matrix P; Among them, the Meta-Scheduler is a framework for dynamically adjusting resource allocation strategies. It is mainly used to balance system efficiency and fairness, especially when processing data-intensive computing tasks. It can achieve efficient resource scheduling. Meta-Scheduler achieves a trade-off between efficiency and fairness by dynamically adjusting the scheduling priorities and resource allocation ratios of different components. For example, in a multi-tasking environment, it can automatically adjust the execution order based on task urgency and resource usage, avoiding a single task occupying resources for a long time, which would lead to a decrease in overall efficiency.

[0032] The training configuration includes Optimizer (Lion + Gradient Centralization), learning rate (2e-5 for the main model, 5e-5 for Scheduler & Agents), batch size (256 tasks / batch), and debate rounds k=3.

[0033] Security mechanism: Inject content security filters and bias detectors into the Consensus Reducer; implement task-level security policy selection in the Scheduler.

[0034] A consensus reducer is a mechanism or component used in multi-agent or distributed systems. Its primary function is to aggregate and integrate the opinions, results, or inferences of multiple agents or nodes to form a unified decision or consensus. It is often used to help the system automatically and efficiently reach consensus when multiple agents need to collaborate, disagree, or have conflicts.

[0035] Step 2: Sample tasks and extract the embedded representation of each agent and the meta-feature vector of the task. Combine the embedded representation of each agent, the meta-feature vector of the task, and the self-supervised graph neural network to generate the collaboration probability of each pair of agents and form the agent collaboration plan matrix. The specific steps include: Step 201: sampling a task T from a training corpus or a synthetic task pool; The training corpus or pool of synthetic tasks is based on historical collaborative data or publicly available datasets; assuming there is a task distribution from which tasks can be sampled, each task can be represented as a dataset or a set of observations; for example, in multi-agent reinforcement learning (MARL), a task may be an instance of an environment that requires multiple agents to cooperate to solve.

[0036] Step 202: Based on the sampled tasks, extract the meta-feature vectors of each agent performing task T ; In this embodiment, the embedding model is used to extract the meta-feature vector of task T. , including the structural features, dynamic features and collaborative features of the task, the structural features, dynamic features and collaborative features of the task are standardized and spliced ​​to obtain the meta-feature vector ; Among them, the structural characteristics of tasks include task dependency graph diameter, critical path length, parallelism factor, subtask coupling degree, etc.; dynamic characteristics include state change entropy, resource demand volatility, environmental disturbance index, etc.; collaboration characteristics include historical collaboration success rate, communication delay distribution, conflict probability, etc. It should be noted that the structural features, dynamic features, and collaborative features of the task are all extracted using existing feature extraction networks. For example, structural features can be analyzed using graph neural networks, dynamic features can be analyzed using LSTM time series, and collaborative features can be obtained using collaborative history statistics. The features currently extracted here can be determined based on actual needs. Due to the differences in the dimensions of the features obtained, the structural features, dynamic features, and collaborative features of the task are standardized and then concatenated to obtain the meta-feature vector. .

[0037] For example, there is a set of text classification tasks, the purpose of which is to analyze the difficulty of each task. For each task T, we obtain "task description text" or "task example"; we use embedding models such as BERT to extract these vectors from the text and convert them into task difficulty representation vectors, which are recorded as After a certain amount of training / fine-tuning, the vector distribution of "complex tasks" in the vector space is made close, which facilitates subsequent clustering, difficulty assessment, transfer learning and other operations.

[0038] Step 203: Combine the embedded representation of each agent, the meta-feature vector of the task being executed, and the self-supervised graph neural network to generate the collaboration probability of each pair of agents and form an agent collaboration plan matrix; In this embodiment, Meta-Scheduler is based on the meta-features Generate agent topology and processes in real time and calculate collaboration plan matrix ( n is the number of Agents), where express Towards The probability of sending messages, taking the embedding representation of each agent and the meta-feature vector of the task performed as the input of the self-supervised GNN, training through self-supervised tasks, and combining the attention mechanism to generate the probability of collaboration between agents; The calling / collaboration probability of each pair of agents is expressed as: , , in, express arrive The probability of collaboration; is a normalization function that ensures the sum of probabilities is 1, For the attention mechanism, calculate the similarity between task features and role embedding; is the meta-feature vector of task T, for Embedded representation of the role, for The role embedding representation ensures that the probability distribution satisfies , achieving adaptive resource allocation.

[0039] In order to ensure the differentiability of the collaborative plan, this embodiment uses a differentiable function such as sigmoid to Mapped to [0,1], the entire generation process from input, to GNN embedding, and then to the collaboration matrix , each step is differentiable and can therefore be trained via backpropagation.

[0040] This step supports dynamic adjustment of the number of Agents , based on computing power budget and convergence rate, it perceives GPU / TPU queue congestion and realizes flexible resource allocation.

[0041] Step 3: Allocate corresponding communication resources according to the agent collaboration plan matrix. Each agent performs asynchronous message interaction on the interaction bus based on the allocated communication resources to obtain the agent output distribution. The output distribution of each agent is mapped into the global reward vector and the goal of the final task. The specific steps include: Step 301: Allocate corresponding communication resources according to the Agent collaboration plan matrix; In this embodiment, when allocating corresponding communication resources according to the agent collaboration plan matrix, each element represents the collaboration probability between agent pairs. First, the total available communication resources, such as the total bandwidth or number of channels, are determined. Then, resources are allocated based on the collaboration probability. For example, proportional allocation is used, where the resources allocated to each agent pair are proportional to their collaboration probability. Ultimately, the total amount of allocated resources is ensured not to exceed the total amount. A resource allocation matrix is ​​generated, showing the specific amount of resources allocated to each agent pair.

[0042] Step 302: Agent Based on the allocated communication resources, asynchronous message interaction is performed on the interactive bus to obtain Agent Agent output distribution; In this embodiment, each Agent Based on the allocated communication resources, asynchronous communication is conducted on the Interaction Bus, and a k-step cycle of asynchronous debate, criticism, and refinement is repeated to obtain the results of multiple rounds of debate. It should be noted that the above Agent Conduct asynchronous debate, critique, and refinement k The step-by-step loop iteration process can be implemented using the existing mechanism; Furthermore, when Role-Specialized Agents communicate asynchronously on the Interaction Bus, they do so through a high-throughput message queue and a bidirectional memory cache mechanism. The high-throughput message queue supports asynchronous concurrent communication (such as RabbitMQ / Kafka), ensuring low-latency delivery. The bidirectional memory cache mechanism includes both short-term and long-term memory. The short-term cache stores the intermediate states of the current debate (such as the argument tree and evidence pool), while the long-term memory archives historical decision logic for retrieval in subsequent tasks.

[0043] Furthermore, at each step of interaction, a Transformer-based encoder is used to achieve differentiable propagation and reduce information loss. The interactive message is represented as: , in, express Send to message vector; for The hidden state of for The hidden state of MLP is a multi-layer perceptron, which is used for nonlinear transformation; It is a vector concatenation operation; Step 303: Map the output distribution of each agent into a global reward vector R and the target of the final task Â, and perform cross-agent gradient alignment with the ground-truth or self-generated teacher signal; In this embodiment, contrastive learning is introduced and KL divergence is used to minimize the multi-agent output distribution to obtain a global reward vector, which is expressed as: , in, represents the global reward vector; Reward candidates r Seek maximization; Represents the sum of conditional probability logarithms, measuring the final answer In the reward r Likelihood under; β To balance hyperparameters; represents the joint output distribution of multiple agents, which is the product of the output distribution of each agent. ,in, is based on the reward vector r Boltzmann distribution, ,in, Agent Output The feature map vector of is the normalization constant, , Q represents the preset reference distribution (such as uniform distribution), Represents the output distribution P With the reference distribution Q KL divergence of , used for regularization; This formula ensures unbiased consensus estimation. This step provides fine-grained error signals, improving the effectiveness of multi-role self-checking and adversarial debate.

[0044] In this embodiment, during training, cross-agent gradient alignment and conflict elimination are performed, and the alignment loss based on collaborative covariance minimization is:

[0045] in, represents the alignment loss; represents the matrix trace operation; Represents the covariance matrix, which calculates the covariance between gradient vectors; arrive Indicates the parameters of each Agent The gradient vector of The global objective function of the multi-agent system is Relative to each agent Parameters The partial derivatives are obtained, where the output distribution of each agent is By parameters Parameterization, if the agent is a neural network model, Can be the network weights, for each agent , the gradient vector is calculated as follows: , The first term is the partial derivative of the Boltzmann distribution, and the second term is the contribution of KL regularization.

[0046] Finally, the parameters of the main model and each agent Through the joint loss, back propagation is performed and the total loss of training is expressed as: , in, Indicates the total loss; represents the task-specific loss (such as cross entropy), represents the weight hyperparameter of the alignment loss, The weight hyperparameter representing the scheduling loss; Represents the scheduler optimization loss.

[0047] The entire process of the present invention adopts continuous relaxation (such as Gumbel-Softmax) to ensure differentiability and support end-to-end optimization.

[0048] In the reasoning phase, agents with a collaboration probability greater than a set threshold are enabled based on actual task requirements; In this embodiment, the default activation probability threshold is greater than 0.4, which enables lightweight reasoning with a time complexity of O( / m)); If you need extreme performance, you can enable all agents; The framework supports scalable applications, including automatic code generation and review (the three types of agents, "generation-review-unit testing," form an endogenous closed loop, improving code quality by 40%, based on SWE-Bench); cross-modal reasoning (vision and voice agents are mounted using the LoRA-Vision / Audio Adapter to support video understanding and improve ARC-AGI performance); and bootstrapped model evolution (the meta-learning capability of the Meta-Scheduler periodically outputs the next-generation topology, implementing a "collaboration-evolution" dual loop, drawing on MAS-GPT to improve long-term generalization).

[0049] This paper proposes an "endogenized" training model that deeply integrates multi-agent autonomous orchestration and collaboration mechanisms during the model training phase. This model aims to embed complex agent collaborative behaviors, traditionally implemented through external scripts or manual rules during the reasoning phase, into the model's pre-training and fine-tuning processes. Through this endogenous design, the model can autonomously learn and optimize its internal decision-making logic, dynamic role allocation strategies, and gradient consistency optimization across agents, significantly improving its overall performance in handling complex reasoning tasks, achieving cross-domain knowledge generalization, and efficiently utilizing computing resources. This invention not only represents a major breakthrough in existing LLM training methods but also lays a solid foundation for the future development of general artificial intelligence (AGI), enabling it to adapt to and solve unknown and complex problems more autonomously and efficiently.

[0050] Figure 2 A block diagram of a multi-agent endogenous model training system is shown in Figure 2 , the embodiment includes: A training initialization module 201 is used for training preparation and training initialization; The multi-agent intra-agent biochemical training module 202 is used for collaborative analysis of sampling tasks by agents, and extracts the embedded representation of each agent and the meta-feature vector of the task execution. The embedded representation of each agent, the meta-feature vector of the task execution, and the self-supervised graph neural network are combined to generate the collaboration probability of each pair of agents to form the agent collaboration plan matrix; According to the agent collaboration plan matrix, corresponding communication resources are allocated. Based on the allocated communication resources, each agent performs asynchronous message interaction on the interaction bus to obtain the output distribution of each agent, and then maps the output distribution of each agent into the global reward vector and the goal of the final task.

[0051] It should be noted that the specific implementation method of a multi-agent-based endogenous model training system in an embodiment of the present invention is similar to the specific implementation method of a multi-agent-based endogenous model training method in an embodiment of the present invention. Please refer to the description of the method part for details. In order to reduce redundancy, it will not be repeated here.

[0052] Figure 3 : is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. When the agent is a hardware device, the computer device 300 can be the agent in the above embodiment. When the agent is a software program, the computer device 300 can be used to run the agent. The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 301 and one or more memories 302. Among them, at least one instruction is stored in the memory 302, and the at least one instruction is loaded and executed by the processor 301 to implement the methods provided in the above-mentioned various method embodiments. Of course, the computer device 302 may also have components such as a wired or wireless network interface, various input and output interfaces, etc., for input and output. The computer device 300 may also include other components for implementing device functions, which will not be described here.

[0053] In an exemplary embodiment, a computer-readable storage medium is further provided, such as a memory including instructions, which can be executed by a processor in a terminal to implement a multi-agent endogenous model training method in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic tape, a floppy disk, an optical data storage device, etc. Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc. In an exemplary embodiment, a computer program product is also provided, which includes at least one instruction, and the at least one instruction can be loaded and executed by a processor to implement a multi-agent endogenous model training method provided in the present application. It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the environmental characteristics involved in this application are all obtained with full authorization. The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A multi-agent endogenous model training method, characterized in that: include: Perform training preparation and training initialization; Sampling tasks, extracting the embedded representation of each agent and the meta-feature vector of the task, and combining the embedded representation of each agent, the meta-feature vector of the task, and the self-supervised graph neural network to generate the collaboration probability of each pair of agents to form the agent collaboration plan matrix; According to the agent collaboration plan matrix, corresponding communication resources are allocated. Based on the allocated communication resources, each agent performs asynchronous message interaction on the interaction bus to obtain the output distribution of each agent, and then maps the output distribution of each agent into the global reward vector and the goal of the final task.

2. A multi-agent endogenous model training method according to claim 1, characterized in that: Perform training preparation and training initialization, including initializing the main model and each agent in the hardware environment, initializing the meta-scheduler, and training configuration and determining the security policy.

3. The method for training a multi-agent endogenous model according to claim 1, wherein: The formula for calculating the collaboration probability of each pair of agents is: ， ， in, express arrive The probability of collaboration, is a normalization function that ensures the sum of probabilities is 1, Attention For the attention mechanism, calculate the similarity between task features and role embeddings, is the meta-feature vector of task T, for Embedded representation of the role, for Embedded representation of roles.

4. The method for training a multi-agent endogenous model according to claim 1, wherein: After generating the collaboration probability of each pair of agents, a differentiable function is used to map the collaboration probability of each pair of agents to [0,1].

5. The method for training a multi-agent endogenous model according to claim 1, wherein: The allocation of corresponding communication resources according to the agent collaboration plan matrix includes: determining the total available communication resources, including the total bandwidth or the number of channels; allocating resources according to the collaboration probability, including allocating resources to each pair of agents in proportion to their collaboration probability, and ultimately ensuring that the total amount of allocated resources does not exceed the total amount.

6. A multi-agent endogenous model training method according to claim 1, characterized in that: When each agent performs asynchronous message interaction on the interactive bus based on the allocated communication resources to obtain the output distribution of each agent, they communicate through a high-throughput message queue and a two-way memory cache mechanism.

7. A multi-agent endogenous model training method according to claim 6, characterized in that: At each step of interaction, a Transformer-based encoder is used to implement differentiable propagation, and the interactive message is represented as: ， in, express Send to message vector; for The hidden state of for The hidden state of It is a multi-layer perceptron, used for nonlinear transformation; It is a vector concatenation operation.

8. The method for training a multi-agent endogenous model according to claim 1, wherein: Contrastive learning is introduced, and KL divergence is used to minimize the multi-agent output distribution to obtain the global reward vector, which is expressed as: ， in, represents the global reward vector; Reward candidates r Seek maximization; Represents the sum of conditional probability logarithms, measuring the final answer In the reward r Likelihood under; β To balance hyperparameters; represents the joint output distribution of multiple agents, which is the product of the output distribution of each agent. ,in, is based on the reward vector r Boltzmann distribution, ,in, Agent Output The feature map vector of is the normalization constant, , Q represents the preset reference distribution (such as uniform distribution), Represents the output distribution P With the reference distribution Q KL divergence of , used for regularization.

9. A multi-agent endogenous model training method according to claim 6, characterized in that: The total loss function for training is: ， ， in, Indicates the total loss; represents the task-specific loss, represents the weight hyperparameter of the alignment loss, The weight hyperparameter representing the scheduling loss; represents the scheduler optimization loss, represents the alignment loss; represents the matrix trace operation; Represents the covariance matrix, which calculates the covariance between gradient vectors; arrive Indicates the parameters of each Agent The gradient vector of the global objective function of the multi-agent system Relative to each agent Parameters Find the partial derivative to get .

10. A multi-agent endogenous model training system, characterized by: include: Training initialization module, used for training preparation and training initialization; The multi-agent intra-agent biochemical module is used for collaborative analysis of sampling tasks by agents, extracting the embedded representation of each agent and the meta-feature vector of the task, and combining the embedded representation of each agent, the meta-feature vector of the task, and the self-supervised graph neural network to generate the collaboration probability of each pair of agents to form the agent collaboration plan matrix; According to the agent collaboration plan matrix, corresponding communication resources are allocated. Based on the allocated communication resources, each agent performs asynchronous message interaction on the interaction bus to obtain the output distribution of each agent, and then maps the output distribution of each agent into the global reward vector and the goal of the final task.

Images