Cooperative control system for knowledge transmission among multiple agents and operation method thereof

Abstract

The invention provides a cooperative control system for knowledge transmission among multiple agents and an operation method thereof, and the system comprises at least two agent mechanisms which are used for carrying out the task analysis, knowledge extraction and storage of a shared task and an individual task, and obtaining a shared concept vector and an individual task concept vector; the concept sharing module is used for sending the individual task concept vector of the first intelligent agent mechanism to the second intelligent agent mechanism; and the task allocation module is used for allocating the to-be-executed individual task to a third agent structure, so that the third agent executes the to-be-executed individual task according to the shared concept vector and the individual task concept vector corresponding to the to-be-executed individual task. According to the system, lightweight and efficient knowledge transmission is realized, the communication overhead and the training cost are greatly reduced, each agent does not need to learn all tasks from the beginning through knowledge sharing, and the overall learning efficiency and the resource utilization rate are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a collaborative control system and its operation method for knowledge transfer among multiple agents. Background Technology

[0002] In the fields of artificial intelligence and machine learning, reinforcement learning, as an important learning paradigm, has achieved remarkable results in many application scenarios such as games, robot control, and autonomous driving.

[0003] Traditional single-agent reinforcement learning methods suffer from problems such as low learning efficiency, weak generalization ability, and opaque knowledge representation. Especially when facing complex environments or diverse tasks, a single agent needs to learn each task independently from scratch, resulting in a time-consuming and resource-intensive learning process.

[0004] In related technologies, multi-agent knowledge sharing methods employ techniques such as parameter sharing or policy distillation. These methods require sharing the entire network model and involve complex training procedures to transfer knowledge. The process is complex and computationally intensive, resulting in low efficiency of knowledge sharing when multiple agents perform tasks, which in turn leads to low efficiency of multi-agent task collaboration. Summary of the Invention

[0005] This invention provides a collaborative control system and its operation method for knowledge transfer among multiple agents, which solves the problems of complex processes and large computational loads in existing technologies that use parameter sharing or strategy distillation for knowledge sharing among multiple agents, resulting in low efficiency of knowledge sharing when multiple agents perform tasks. The method of this invention improves the efficiency of knowledge transfer and task collaboration among multiple agents.

[0006] This invention provides a cooperative control system for knowledge transfer among multiple agents, comprising: At least two intelligent agent mechanisms are used, each of which performs the same shared task and different individual tasks. For each intelligent agent mechanism, the mechanism performs task parsing, knowledge extraction, and storage on the shared task and individual tasks respectively, to obtain a shared concept vector and an individual task concept vector. The shared concept vector and individual task concept vector include task information and task configuration information. Different intelligent agent mechanisms correspond to different individual tasks, and the mechanisms are obtained through joint training. A concept sharing module is used to send the individual task concept vector of the first intelligent agent mechanism to the second intelligent agent mechanism; wherein the second intelligent agent mechanism is any one of the at least two intelligent agent mechanisms, and the first intelligent agent mechanism is another intelligent agent structure among the at least two intelligent agent mechanisms other than the second intelligent agent mechanism; A task allocation module is used to allocate individual tasks to be executed to a third agent structure, so that the third agent can execute the individual tasks according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein, the third agent structure is at least one of the at least two agent structures; the individual tasks to be executed include at least one of the different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

[0007] According to the present invention, a cooperative control system for knowledge transfer among multiple agents is provided, wherein each agent mechanism includes: A concept extraction module is used to process an initial concept vector through a nonlinear activation operation to obtain a concept modulation value; The task solving module is used to process the input features corresponding to an individual task through a nonlinear activation operation to obtain activation values, and then perform a linear transformation operation between the activation values ​​and the concept modulation values ​​to obtain the individual task concept vector. A concept vector storage module is used to store the concept vectors of the individual tasks.

[0008] According to the present invention, a collaborative control system for knowledge transfer among multiple agents is provided, wherein the concept extraction module and the task solving module are two-stream network structures. The network structure corresponding to the concept extraction module includes multiple hidden layers, each of which is represented by the following formula: c i = Sigmoid(LayerNorm(V i ·c {i-1} + d i )); The network structure corresponding to the task solving module includes multiple hidden layers, each of which is represented by the following formula: h0 = cat(s; a); h i = ReLU(LayerNorm(W i ·h {i-1} + b i )); h i = hh i * c i ; Q(s, a | c i ) = W n ·h {n-1} + b n ; Where c0 is the input of the concept extraction module, ci To solve for the output concept modulation value of the network corresponding to the concept in the i-th layer, sigmoid is the activation function, and V i Let c be the linear weight parameters corresponding to the i-th layer of the concept extraction module. {i-1} Let d be the concept modulation value corresponding to the (i-1)th layer of the network. i hi represents the bias parameters corresponding to the i-th layer of the concept extraction module; h0 is the input to the task solving module; cat is the vector dimension expansion and concatenation operation; hi is the input to the task solving module ... i To find the output activation value of the network for the i-th layer task, ReLU is the activation function, and W is the activation value. i hh represents the linear weight parameters corresponding to the i-th layer of the network in the task solving module. {i-1} b is the temporary activation value corresponding to the (i-1)th layer of the network. i The bias parameters are the bias parameters corresponding to the i-th layer of the task solving module; Q(s, a | c) is the Bellman equation state-action value function corresponding to state s and action a when facing task c; i = 1, 2, …, n.

[0009] According to the present invention, a collaborative control system for knowledge transfer among multiple agents is provided, wherein the agents are jointly trained through the following steps: For each agent, during different training cycles, sample tasks and corresponding sample individual task concept vectors are randomly obtained from the sample task set. Within each training cycle, the state features corresponding to the sample task are used as the input to the initial task solving module, and the individual task concept vector of the sample is used as the input to the initial concept extraction module. The initial task solving module and the initial concept extraction module are trained alternately using the ε-greedy action strategy and the mean squared error loss function. Under the condition of meeting the training conditions, the trained agent structure is obtained; the sample shared concept vectors corresponding to each agent change synchronously.

[0010] According to the collaborative control system for knowledge transfer among multiple agents provided by the present invention, the concept extraction module is further configured to obtain the initial concept vector based on the task index.

[0011] According to a collaborative control system for knowledge transfer among multiple agents provided by the present invention, the concept sharing module is further used for: The fourth intelligent agent mechanism is loaded according to the task coordination instruction, and the individual task concept vector of the fourth intelligent agent mechanism is obtained; wherein, the task coordination instruction is determined based on the number and content of the individual tasks to be executed; The task allocation module is also used for: According to the task coordination instruction, the individual task concept vector of the fourth intelligent agent is sent to the fifth intelligent agent so that the fifth intelligent agent can execute the individual task to be executed; wherein the fourth intelligent agent and the fifth intelligent agent are each one of the at least two intelligent agents.

[0012] According to the present invention, a cooperative control system for knowledge transfer among multiple agents is provided, the system further comprising: An evaluation and testing module is used to fine-tune and update the learning parameters of at least two intelligent agents based on the task completion rate and knowledge utilization rate of each agent, so as to obtain the updated agent. The task completion rate is determined based on the ratio of the number of individual tasks completed to the number of individual tasks assigned, and the knowledge utilization rate is determined based on the ratio of the number of effective accesses to the concept vector storage module to the total number of accesses.

[0013] This invention also provides a method for operating a cooperative control system for knowledge transfer among multiple agents, comprising: The intelligent agent mechanism performs task parsing, knowledge extraction, and storage for the task to be executed, resulting in a shared concept vector and individual task concept vectors. The task to be executed includes shared tasks and individual tasks. The shared concept vector and individual task concept vector include task information and task configuration information. Different intelligent agents correspond to different individual tasks, and the intelligent agents are obtained through joint training. The concept sharing module sends the individual task concept vector of the first intelligent agent to the second intelligent agent; wherein the second intelligent agent is any one of the at least two intelligent agent agents, and the first intelligent agent is another intelligent agent structure among the at least two intelligent agent agents other than the second intelligent agent. The task allocation module assigns individual tasks to a third agent structure, so that the third agent can execute the individual tasks according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein, the third agent structure is at least one of the at least two agent structures; the individual tasks to be executed include at least one of the different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the operation method of the cooperative control system for knowledge transfer among multiple agents as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the operation method of a cooperative control system for knowledge transfer among multiple agents as described above.

[0016] The present invention provides a collaborative control system and its operation method for knowledge transfer among multiple agents. The system uses agent mechanisms to perform task parsing, knowledge extraction, and storage for shared tasks and individual tasks, respectively, to obtain shared concept vectors and individual task concept vectors. A concept sharing module sends the individual task concept vectors from the first agent mechanism to the second agent mechanism. A task allocation module allocates the individual tasks to be executed to a third agent mechanism, allowing the third agent to execute the individual tasks based on the shared concept vectors and the corresponding individual task concept vectors. The system of the present invention shares only concept representations rather than the entire model parameters, achieving lightweight and efficient knowledge transfer, greatly reducing communication overhead and training costs. Furthermore, through knowledge sharing, each agent does not need to learn all tasks from scratch, improving overall learning efficiency and resource utilization. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of the collaborative control system for knowledge transfer among multiple agents provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the intelligent agent mechanism provided by the present invention.

[0020] Figure 3 This is a flowchart illustrating the operation method of a collaborative control system for knowledge transfer among multiple agents provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0022] Figure label: 100: Intelligent agent mechanism; 110: Concept extraction module; 120: Task solving module; 130: Concept vector storage module; 200: Concept sharing module; 300: Task allocation module; 400: Evaluation and Testing Module. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] The following is combined with Figures 1-3 This invention describes a collaborative control system for knowledge transfer among multiple agents and its operation method.

[0025] Figure 1 This is a schematic diagram of the structure of the cooperative control system for knowledge transfer among multiple agents provided by the present invention, as shown below. Figure 1 As shown, the collaborative control system for knowledge transfer among multiple agents includes: at least two agent mechanisms 100, a concept sharing module 200, and a task allocation module 300.

[0026] At least two intelligent agent mechanisms 100 are used to execute the same shared task and different individual tasks. For each intelligent agent mechanism 100, the intelligent agent mechanism 100 is used to perform task parsing, knowledge extraction and storage for the shared task and individual tasks respectively, to obtain a shared concept vector and an individual task concept vector. The shared concept vector and the individual task concept vector include task information and task configuration information. Different intelligent agent mechanisms 100 correspond to different individual tasks, and the intelligent agent mechanisms 100 are obtained through joint training.

[0027] In this embodiment, an individual task includes task requirements in one or more scenarios.

[0028] For example, the multiple intelligent agent mechanism 100 includes multiple robotic arm intelligent agents, which are used to collaboratively complete complex assembly tasks. For this assembly task, the corresponding individual task 1 includes "accurately picking up Class A parts (such as screws) from the hopper 1 and installing them at assembly point P1", the corresponding individual task 2 includes "accurately picking up Class B parts (such as nuts) from the hopper 2 and installing them at assembly point P2", and the corresponding individual task 3 includes "applying glue to the specified surface S".

[0029] For example, the multiple intelligent agent mechanism 100 includes multiple game intelligent agents for performing tasks that integrate different game skills. For this task, the corresponding individual task 1 includes "learning and mastering game G1 (e.g., a complex platforming game)," the corresponding individual task 2 includes "learning and mastering game G2 (e.g., a strategy role-playing game)," and the corresponding individual task 3 includes "learning and mastering game G3 (e.g., a fast-paced flight shooting game)."

[0030] In this embodiment, when the intelligent agent mechanism 100 performs an individual task, it can map the state of performing the individual task into action value to realize interactive decision-making with the environment. It can also extract knowledge from the individual task, modulate it into a concept representation, and combine the action value and the concept modulation value to determine the corresponding individual task concept vector, which is used to represent the environmental interaction mode and dynamic characteristics required to perform the basic action corresponding to the current task.

[0031] For example, in scenarios where multiple robotic arm agents collaborate to complete complex assembly tasks, the individual task concept vector represents the characteristics of different types of parts, the location information of the material bins, and assembly requirements; in scenarios where multiple game agents perform tasks integrating different game skills, the individual task concept vector represents the level design, adversarial behavior patterns, special item effects, and resource management of different games.

[0032] In this embodiment, multiple individual tasks share the same set of concept vectors, ensuring that the agents have a common cognitive basis for the basic interaction logic of the tasks to be performed.

[0033] For example, in scenarios where multiple robotic arm agents collaborate to complete complex assembly tasks, the shared concept vectors of each agent include basic operational skills such as "reaching a designated three-dimensional spatial point (x,y,z)", "grabbing a standard part", "releasing the workpiece", and "rotating the wrist to a specific angle". In scenarios where multiple game agents perform tasks integrating different game skills, the shared concept vectors of the agents include general game meta-skills such as "understanding two-dimensional / three-dimensional spatial navigation", "identifying interactive objects", "understanding goal-oriented behaviors (such as scoring, avoiding danger)" and "basic input control mapping (such as directional key movement, button jump / shoot)".

[0034] In this embodiment, the intelligent agent mechanism 100 parses and extracts knowledge from the executable individual tasks to obtain the corresponding individual task concept vector, and can store the individual task concept vector locally for subsequent reading and retrieval.

[0035] In this embodiment, during the joint training phase of multiple agent structures, the shared concept vectors among the agents change synchronously, that is, the shared concept vectors among the agents remain consistent in each training cycle of the agents.

[0036] In this embodiment, during each training cycle, all agent mechanisms 100 are traversed. For each agent mechanism 100, a task is randomly selected from the task set, the concept vector corresponding to the task in the agent mechanism 100 is indexed, and experience is obtained by interacting with the environment through a two-stream network structure. The network parameters and concept embeddings are trained alternately in combination with the action selection strategy. Finally, the learning effect is evaluated and the concept representation is optimized to obtain the agent mechanisms 100 after joint training.

[0037] The concept sharing module 200 is used to send the individual task concept vector of the first intelligent agent mechanism 100 to the second intelligent agent mechanism 100; wherein the second intelligent agent mechanism 100 is any one of at least two intelligent agent mechanisms 100, and the first intelligent agent mechanism 100 is another intelligent agent structure other than the first intelligent agent mechanism 100 among the at least two intelligent agent mechanisms 100.

[0038] In this embodiment, through the concept sharing module 200, each agent can load concept vectors unique to other agents and copy the concept vectors of other agents to the corresponding positions of the current agent; after knowledge transfer, each agent can use the shared concepts to solve tasks that it has never encountered before without retraining the network parameters.

[0039] For example, agent 100A can execute individual task 1 independently, agent 100B can execute individual task 2 independently, and agent 100C can execute individual task 3 independently. Each agent task corresponds to the same shared concept vector 0, individual task 1 corresponds to individual task concept vector a, individual task 2 corresponds to individual task concept vector b, and individual task 3 corresponds to individual task concept vector c. For agent 100B, by loading the individual task concept vectors a and c corresponding to agent 100A and C (corresponding to the first agent 100) and sending them to agent 100B (corresponding to the second agent 100), agent 100B, including the shared concept vector 0 and the individual task concept vector ac, can execute the individual tasks corresponding to agent 100A and C, realizing the transfer and integration of conceptual knowledge among multiple agents.

[0040] The task allocation module 300 is used to allocate individual tasks to be executed to a third intelligent agent structure, so that the third intelligent agent can execute the individual tasks according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein, the third intelligent agent structure is at least one of at least two intelligent agent mechanisms 100; the individual tasks to be executed include at least one of different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

[0041] In this embodiment, the task allocation module 300 can distribute tasks according to the content type of the individual task to be executed and the execution object.

[0042] For example, without specifying the execution object, the task allocation module 300 can distribute the individual task to be executed to any intelligent agent 100. Since any intelligent agent 100 includes a shared concept vector and all individual task concept vectors, any intelligent agent 100 can execute the individual task to be executed.

[0043] When a specific execution target is specified, the task allocation module 300 can distribute the individual task to be executed to the intelligent agent mechanism 100 corresponding to the execution target to execute the individual task. The intelligent agent mechanism 100 that has not encountered the individual task before can use the shared concept vector of the task without retraining the network parameters corresponding to the individual task. This separates task knowledge (concept) from problem-solving ability (network parameters), realizes modular learning, and enhances the interpretability and maintainability of the system.

[0044] The collaborative control system for knowledge transfer among multiple agents provided in this invention uses an agent mechanism 100 to perform task parsing, knowledge extraction, and storage for shared tasks and individual tasks, respectively, to obtain shared concept vectors and individual task concept vectors. A concept sharing module 200 sends the individual task concept vectors from the first agent mechanism 100 to the second agent mechanism 100. A task allocation module 300 allocates the individual tasks to be executed to a third agent mechanism, allowing the third agent to execute the individual tasks based on the shared concept vectors and the corresponding individual task concept vectors. The system of this invention shares only concept representations rather than the entire model parameters, achieving lightweight and efficient knowledge transfer, greatly reducing communication overhead and training costs. Furthermore, through knowledge sharing, each agent does not need to learn all tasks from scratch, improving overall learning efficiency and resource utilization.

[0045] In some embodiments, each agent mechanism 100 includes: a concept extraction module 110, a task solving module 120, and a concept vector storage module 130.

[0046] The concept extraction module 110 is used to process the initial concept vector through a nonlinear activation operation to obtain the concept modulation value.

[0047] In this embodiment, the initial concept vector can be read directly from a concept embedding library or local storage.

[0048] In this embodiment, the concept extraction module 110 can be an encoder that encodes the input initial concept vector, i.e., performs a non-linear activation operation, to obtain the concept modulation value corresponding to the current task.

[0049] For example, the concept extraction module 110 and the task solving module 120 are two-stream network structures; the network structure corresponding to the concept extraction module 110 includes multiple hidden layers, each of which is represented by the following formula: c i = Sigmoid(LayerNorm(V i ·c {i-1} + d i )); The task solving module 120 is used to process the input features corresponding to the individual task through nonlinear activation operations to obtain activation values, and then perform a linear transformation operation between the activation values ​​and the concept modulation values ​​to obtain the individual task concept vector.

[0050] For example, the task solving module 120 and the concept extraction module 110 are both branches of a two-stream network structure; the network structure corresponding to the task solving module 120 includes multiple hidden layers, each of which is represented by the following formula: h0 = cat(s; a); hh i = ReLU(LayerNorm(W i ·h {i-1} + b i )); h i = hh i * c i ; Furthermore, the linear transformation operation between the activation value and the concept modulation value can be expressed by the following formula: Q(s, a | c i ) = W n ·h {n-1} + b n ; Where c0 is the input of the concept extraction module 110, c i To solve for the output concept modulation value of the network corresponding to the concept in the i-th layer, sigmoid is the activation function, and V i Let c be the linear weight parameters corresponding to the i-th layer of the concept extraction module 110. {i-1} Let d be the concept modulation value corresponding to the (i-1)th layer of the network. i h0 is the bias parameter corresponding to the i-th layer of the concept extraction module 110; h0 is the input of the task solving module 120; cat is the vector dimension expansion and concatenation operation; h i To find the output activation value of the network for the i-th layer task, ReLU is the activation function, and W is the activation value. i For the linear weight parameters corresponding to the i-th layer of the network in the task solving module 120, hh {i-1} b is the temporary activation value corresponding to the (i-1)th layer of the network.i The bias parameters are the bias parameters corresponding to the i-th layer of the network in the task solving module 120; Q(s, a|c) is the Bellman equation state-action value function corresponding to state s and action a when facing task c; i = 1, 2, …,n.

[0051] The concept vector storage module 130 is used to store the concept vectors of individual tasks.

[0052] In this embodiment, the concept vector storage module 130 can be a concept embedding library, used to store individual task concept vectors corresponding to different individual tasks in the same scene or different scenes.

[0053] The collaborative control system for knowledge transfer among multiple agents provided in this embodiment of the invention processes the initial concept vector through a concept extraction module 110 using a nonlinear activation operation to obtain concept modulation values; processes the input features corresponding to individual tasks through a task solving module 120 using a nonlinear activation operation to obtain activation values, and performs a linear transformation operation between the activation values ​​and the concept modulation values ​​to obtain individual task concept vectors; and stores the individual task concept vectors through a concept vector storage module 130. By separating task knowledge (concepts) from problem-solving capabilities (network parameters), modular learning is achieved, enhancing the interpretability and maintainability of the system. Each agent maintains its unique task-solving capabilities while utilizing shared concepts, achieving a balance between personalization and collaboration.

[0054] In some embodiments, each agent mechanism 100 performs joint training through the following steps: For each agent, sample tasks and corresponding sample individual task concept vectors are randomly obtained from the sample task set during different training cycles; During each training cycle, the state features corresponding to the sample tasks are used as the input of the initial task solving module 120, and the sample individual task concept vectors are used as the input of the initial concept extraction module 110. The initial task solving module 120 and the initial concept extraction module 110 are trained alternately using an ε-greedy action strategy and a mean squared error loss function. Under the condition that the training conditions are met, the trained agent structure is obtained; The sample shared concept vectors corresponding to each agent change synchronously.

[0055] In this embodiment, the dual-stream network structure can be a concept-aware dual-stream network (CA-TS Network). The learning process for multiple agents includes initializing each CA-TS network and concept embedding library, and receiving an assigned task set (including shared tasks and individual-specific tasks).

[0056] Specifically, in each training cycle of each of the 100 intelligent agents, a task is first randomly selected from the task set, the corresponding concept vector is retrieved, and experience is gained through interaction with the environment via the CA-TS network. An ε-greedy policy is used to select actions, and the network parameters and concept embeddings are trained alternately. The Q-value is updated using the Q-value function and the mean squared error loss function. Finally, the learning effect is evaluated and the concept representation is optimized. The Q-value function is expressed by the following formula: Q_target = r + γ·max_a'Q(s',a')·(1-done); Where Q_target is the target Q value, r is the immediate reward, γ is the discount factor, max_a'Q(s',a') is the maximum Q value of the next state, done is the termination flag, and (1-done) is the complement of the termination flag.

[0057] The mean squared error loss function (Loss) is expressed by the following formula: Loss = MSE(Q(s,a), Q_target).

[0058] In this embodiment, during the initial training phase, system initialization is performed first, including building a task environment library, initializing all agents, and allocating task sets; during the training coordination process, the training progress of each agent is monitored, performance indicators and learning efficiency data are collected, and training parameters are dynamically adjusted.

[0059] The collaborative control system for knowledge transfer among multiple agents provided in this embodiment of the invention involves randomly acquiring sample tasks and corresponding individual task concept vectors from a sample task set during different training cycles for each agent. During each training cycle, the initial task solving module 120 is input using the state features corresponding to the sample tasks, and the initial concept extraction module 110 is input using the individual task concept vectors. An ε-greedy action strategy and a mean squared error loss function are used to alternately train the initial task solving module 120 and the initial concept extraction module 110, resulting in a trained agent structure. This joint training of multiple agents further enhances the agent's knowledge transfer capability.

[0060] In some embodiments, the input features of the task solving module 120 include state-action vector pairs or state vectors.

[0061] In this embodiment, the two-stream network structure is a concept-aware two-stream network (CA-TS Network), and the input features are state vectors. The process is described as follows: For the state-to-action mapping pattern (Q(s) → a), the input layer receives the state s, performs a linear transformation on each layer, and finally combines the task concept modulation value transmitted by the concept extraction module 110 to calculate the action value Q(s), which is the individual task concept vector. For the state-action-to-value mapping pattern (Q(s,a)), the input layer receives the combination of state s and action a, and calculates the action value Q(s) by combining the task concept modulation value transmitted by the concept extraction module 110, which is the individual task concept vector.

[0062] The collaborative control system for knowledge transfer among multiple agents provided in this embodiment of the invention improves the flexibility of task solving and expands application scenarios by setting the input features of the task solving module 120 to include state-action vector pairs or state vectors.

[0063] In some embodiments, the concept extraction module 110 is further configured to obtain an initial concept vector based on the task index.

[0064] In this embodiment, in the multi-agent knowledge transfer process, each agent independently learns its own task set. After learning, it performs a concept sharing process: each agent in the system loads the individual task concept vectors of other agents, obtains the index of the unique tasks of other agents, and copies the concept vectors of other agents to the corresponding positions of the current agent. After knowledge transfer, each agent can use the shared concepts to solve tasks that it has never encountered before without retraining the network parameters.

[0065] The collaborative control system for knowledge transfer among multiple agents provided in this embodiment of the invention improves the efficiency of individual task concept vector retrieval by obtaining initial concept vectors based on task indexes through the concept extraction module 110, thereby improving the efficiency of knowledge transfer among multiple agents.

[0066] In some implementations, the concept sharing module 200 is further configured to: load the fourth intelligent agent mechanism 100 according to the task coordination instruction, and obtain the individual task concept vector of the fourth intelligent agent mechanism 100; wherein the task coordination instruction is determined based on the number and content of the individual tasks to be executed; the task allocation module 300 is further configured to: send the individual task concept vector of the fourth intelligent agent mechanism 100 to the fifth intelligent agent mechanism 100 according to the task coordination instruction, so that the fifth intelligent agent mechanism 100 can execute the individual tasks to be executed; wherein the fourth intelligent agent mechanism 100 and the fifth intelligent agent mechanism 100 are each one of at least two intelligent agent mechanisms 100.

[0067] In this embodiment, the task coordination instruction can be issued by a remote server or control center. For example, the task coordination instruction is "After robotic arms A, B and C each complete an individual task, robotic arm A is required to execute the individual task corresponding to robotic arm C again" or "robotic arm A executes the individual task corresponding to robotic arm B", etc.

[0068] In this embodiment, different individual tasks are executed by different intelligent agents 100. For example, intelligent agent 100A can execute individual task 1 independently, and intelligent agent 100B can execute individual task 2 independently. When intelligent agent 100A needs to execute individual task 2, it can load intelligent agent 100B through the concept sharing module 200200 and read the individual task concept vector b stored in intelligent agent 100B, which is the first agent task concept vector. Finally, the decision-making behavior of intelligent agent 100A is guided by the individual task concept vector b to execute individual task 2, thereby realizing the transfer and integration of conceptual knowledge among multiple agents.

[0069] In this embodiment, after receiving the task coordination instruction and the first volume task concept vector, the task allocation module 300 can distribute the first volume task concept vector according to the task type and execution object indicated by the instruction.

[0070] For example, if the task coordination instruction requires agent 100A to execute individual task 2 corresponding to agent 100B, the task allocation module 300 will send the individual task concept vector b corresponding to individual task 2, which is read by the concept sharing module 200, to agent 100A to guide agent 100A to execute individual task 2. Then, agent 100A can use task 2, which has not been encountered before, without retraining the network parameters corresponding to individual task 2.

[0071] The method described in this invention separates task knowledge (concepts) from problem-solving ability (network parameters), realizing modular learning. Different intelligent agent mechanisms 100 can execute the individual tasks of all intelligent agents, and can also selectively execute the individual tasks of some intelligent agent mechanisms 100, thereby improving the flexibility of knowledge transfer of intelligent agent mechanisms 100.

[0072] In some embodiments, the collaborative control system for knowledge transfer among multiple agents further includes an evaluation and monitoring module 400.

[0073] The evaluation and monitoring module 400400 is used to fine-tune and update the learning parameters of at least two intelligent agent mechanisms 100 based on the task completion rate and knowledge utilization rate of each intelligent agent mechanism 100, so as to obtain the updated intelligent agent mechanism 100; wherein, the task completion rate is determined based on the ratio of the number of individual tasks completed to the number of individual tasks assigned, and the knowledge utilization rate is determined based on the ratio of the number of effective accesses to the concept vector storage module 130 to the total number of accesses.

[0074] In this embodiment, the number of individual tasks completed can be determined by counting whether the task completion conditions are met when the intelligent agent mechanism 100 makes a decision according to the current task concept vector. If the conditions are met, the task is recorded as completed; otherwise, the task fails.

[0075] In this embodiment, during the process of triggering the concept sharing mechanism and verifying the knowledge transfer effect, the evaluation and monitoring module 400 evaluates the overall performance and knowledge utilization of the system, analyzes the learning curves and task completion rates of each agent, and finally optimizes the system parameters and knowledge transfer strategy.

[0076] The collaborative control system for knowledge transfer among multiple agents provided in this invention uses an evaluation and detection module to fine-tune and update the learning parameters of at least two agents based on the task completion rate and knowledge utilization rate of each agent, resulting in an updated agent mechanism. This system can optimize the knowledge transfer performance of each agent in real time and can be extended to more agents and more complex task scenarios, supporting continuous knowledge accumulation and iterative optimization.

[0077] The following two scenarios are described separately: (1) A scenario in which multiple robotic arm agents work together to complete complex assembly tasks.

[0078] (1.1) Learning content and knowledge planning: All robotic arms first learn a set of basic operational skills, such as "reaching a specified 3D spatial point (x, y, z)", "grabbing a standard part", "releasing a workpiece", and "rotating the wrist to a specific angle". These basic skills correspond to a shared set of concept vectors, stored in their respective concept embedding libraries. A supervised learning mechanism ensures the consistency and alignment of these basic concepts across different agents, ensuring that all agents have a common understanding of the basic skills. These concepts represent the environmental interaction patterns and dynamic characteristics required to perform these basic actions.

[0079] (1.2) Specific knowledge (task-specific skill concepts): Agent A learns to "precisely pick up Class A parts (such as screws) from hopper 1 and install them to assembly point P1". This task forms a unique concept vector C_A1, which encodes the characteristics of Class A parts, the location of hopper 1, and the assembly requirements of P1.

[0080] Agent B learns to "precisely pick up Class B parts (such as nuts) from bin 2 and install them to assembly point P2". This task generates a unique concept vector C_B1, which encodes the characteristics of Class B parts, the location of bin 2, and the assembly requirements of P2.

[0081] Agent C learns to "apply adhesive to a specified surface S". This task generates a unique concept vector C_C1 that encodes the use of the adhesive application tool, the properties of the adhesive, and the geometric features of surface S.

[0082] It should be noted that the task solving module of the CA-TS network for each agent learns how to generate action sequences based on the input state and the concept vector of the current task, while the concept extraction module is responsible for generating corresponding concept vectors for each learned task (whether shared or specific).

[0083] (1.3) Knowledge transfer mechanism: When a new, complex assembly task arises, such as "picking up type A parts and then type B parts in sequence, applying glue to specific parts of type A parts, and finally installing type B parts onto type A parts," this task is entirely new to any single intelligent agent. The specific implementation process is as follows: Suppose Agent A is assigned to perform part of this new task. After Agent A completes the installation of its own type A parts, it continues to process type B parts. At this point, the system triggers the concept sharing mechanism (i.e., the function of loading concepts from other agents is activated). Agent A loads the concept vector C_B1 about "picking up type B parts and installing them at assembly point P2" from Agent B's concept embedding library and adds it to its own concept embedding library. The task solving network parameters of Agent A (i.e., the parameters of its core decision network) remain unchanged. That is, Agent A has learned the general ability of "using concepts to perform operations". Therefore, when Agent A performs this task, it can directly use the concept vector C_B1 obtained from Agent B to input into its CA module to perform the task.

[0084] (1.4) Judgment of successful knowledge transfer: After receiving the concept vector C_B1, Agent A can directly use C_B1 to modulate the activation of its task solving module when faced with a task that requires operating on B-type parts (even if it is a new assembly point P_new, as long as the nature of the task is similar to C_B1).

[0085] The hallmark of successful concept transfer is that Agent A can effectively perform the task of manipulating type B parts, originally handled by Agent B. For example, it can successfully grasp a type B part and move it to the target location in the new context, with significantly higher success rate, accuracy, and efficiency than without the concept transfer (e.g., avoiding trial-and-error learning from scratch). This indicates that Agent A has successfully utilized Agent B's experience (in the form of concept vectors) without retraining its core network parameters. Evaluation metrics can include task completion time, grasp success rate, placement accuracy, etc., and be compared with a baseline (without concept transfer). (2) A scenario in which multiple game agents perform tasks that integrate different game skills.

[0086] Suppose there are 3 game agents in the system (Agent Alpha, Beta, Gamma), each specializing in different types of video games.

[0087] (2.1) Learning content and knowledge division: Shared knowledge (universal game meta-concepts): All agents first learn a set of universal game meta-skills, such as "understanding 2D / 3D spatial navigation," "identifying interactive objects," "understanding goal-oriented behaviors (such as scoring, avoiding danger)," and "basic input control mapping (such as directional key movement, button jump / shoot)." These meta-skills form a shared conceptual vector, ensuring that agents have a common cognitive foundation for the basic interaction logic of the game.

[0088] (2.2) Unique knowledge (specific game skill concepts): Agent Alpha learns and becomes proficient in "Game G1 (such as a complex platforming game)," including its unique level design, enemy behavior patterns, special item effects, etc. This knowledge is abstracted into a concept vector set C_G1.

[0089] Agent Beta learns and becomes proficient in "Game G2 (such as a strategy role-playing game)," including its character progression system, combat strategies, resource management, etc. This knowledge is abstracted into a concept vector set C_G2.

[0090] Agent Gamma learns and masters "game G3 (such as a fast-paced flight shooter)," including its bullet hell dodging mode, boss battle strategies, weapon upgrade system, etc. This knowledge is abstracted into a concept vector set C_G3.

[0091] It should be noted that each agent's CA-TS network interacts with the game environment they are good at. Their task-solving module learns how to act under the game rules to achieve the goal, while the concept extraction module generates unique concept representations for G1, G2, and G3 respectively.

[0092] (2.3) Knowledge transfer mechanism: Suppose Agent Alpha needs to learn to play "Game G2". In traditional single-agent learning, Agent Alpha would need to explore G2's rules and policies from scratch. However, in this system, through a concept-sharing mechanism, Agent Alpha can load a core concept vector set C_G2 about Game G2 from Agent Beta's concept embedding library. These concept vectors encode G2's key mechanisms, policies, and objectives. Agent Alpha's task-solving network (its general game capabilities) remains unchanged.

[0093] (2.4) Judgment of successful knowledge transfer: After acquiring C_G2, Agent Alpha's CA-TS network is able to use these new concept vectors to guide its behavioral decisions when it starts playing the game G2.

[0094] The hallmark of successful knowledge transfer is a significantly steeper learning curve for Agent Alpha in Game G2, enabling it to grasp the core gameplay more quickly and exhibit higher initial scores, longer average survival times, or faster task completion speeds compared to an agent learning from scratch (or itself without knowledge transfer). For example, Agent Alpha can quickly understand mechanisms like "turn-based combat" or "skill cooldowns" in G2, even though these mechanisms are absent in G1. This demonstrates that Agent Alpha effectively utilizes Agent Beta's experience with G2, achieving rapid knowledge transfer. Evaluation can be conducted by comparing the training time and number of samples required for Agent Alpha to reach a certain proficiency level (e.g., an average score exceeding a certain threshold) in G2, with and without knowledge transfer.

[0095] Figure 2 This is a schematic diagram of the intelligent agent mechanism provided by the present invention. Figure 2In the illustrated embodiment, the system architecture (agent mechanism) includes 5 agents (Agent1-5). Each agent includes a concept extraction-task execution dual-stream network (corresponding to a dual-stream network structure composed of a concept extraction module and a task solving module) and a concept embedding library (corresponding to a concept vector storage module), totaling 3800 maze environments. The first 3700 (numbered 0-3699) are shared mazes, and each agent also has 20 unique mazes (Agent 1: 3700-3719, Agent 2: 3720-3739, Agent 3: 3740-3759, Agent 4: 3760-3779, Agent 5: 3780-3799).

[0096] exist Figure 2 In the illustrated embodiment, the input to the concept embedding layer of the concept extraction module in the concept extraction-task execution dual-stream network is: a concept vector obtained from the embedding library, the corresponding activation value (c_j) calculated in the hidden layer through the activation function, i.e., the modulation vector, and sent to the task solving module; the input layer of the task solving module inputs the target including state s or state-action pair [s,a], the corresponding activation value (h_j) calculated in the hidden layer through the activation function, and multiplied with the modulation vector c_j sent by the concept extraction module, and the output layer outputs the Q-value function, i.e., the task concept vector.

[0097] During the multi-agent training phase, each agent learns shared mazes and its own unique mazes (e.g., Agent 2 learns mazes 0-3699 and 3720-3739). It learns the optimal path by alternately optimizing network parameters and concept embeddings. The concept vectors corresponding to the first 3700 mazes in each agent remain consistent throughout the update process. When the system triggers the concept sharing mechanism, Agent 2 obtains the concept embeddings of its unique mazes from the other four agents, making its concept library contain the concept representations of all 3800 mazes. In the application phase, Agent 2 can attempt to solve previously unseen mazes (such as 3700-3719 and 3740-3799). For example, when faced with maze 3745, Agent 2 uses the concept vectors obtained from Agent 3, enabling it to successfully navigate the previously unseen maze using its existing task-solving capabilities and newly acquired concepts without retraining the network parameters.

[0098] The following describes the operation method of the cooperative control system for knowledge transfer among multiple agents provided by the present invention. The operation method of the cooperative control system for knowledge transfer among multiple agents described below can be referred to in correspondence with the cooperative control system for knowledge transfer among multiple agents described above.

[0099] Figure 3This is a flowchart illustrating the operation method of a cooperative control system for knowledge transfer among multiple agents provided by the present invention. The operation method of this cooperative control system for knowledge transfer among multiple agents includes the following steps: Step 310: The intelligent agent mechanism performs task parsing, knowledge extraction and storage for the task to be executed, and obtains shared concept vectors and individual task concept vectors; wherein, the task to be executed includes shared tasks and individual tasks; the shared concept vectors and individual task concept vectors include task information and task configuration information; different intelligent agents correspond to different individual tasks, and the intelligent agents are obtained through joint training.

[0100] In this step, in this embodiment, the individual task includes task requirements in one or more scenarios.

[0101] For example, the multiple intelligent agent mechanism 100 includes multiple robotic arm intelligent agents, which are used to collaboratively complete complex assembly tasks. For this assembly task, the corresponding individual task 1 includes "accurately picking up Class A parts (such as screws) from the hopper 1 and installing them at assembly point P1", the corresponding individual task 2 includes "accurately picking up Class B parts (such as nuts) from the hopper 2 and installing them at assembly point P2", and the corresponding individual task 3 includes "applying glue to the specified surface S".

[0102] For example, the multiple intelligent agent mechanism 100 includes multiple game intelligent agents for performing tasks that integrate different game skills. For this task, the corresponding individual task 1 includes "learning and mastering game G1 (e.g., a complex platforming game)," the corresponding individual task 2 includes "learning and mastering game G2 (e.g., a strategy role-playing game)," and the corresponding individual task 3 includes "learning and mastering game G3 (e.g., a fast-paced flight shooting game)."

[0103] In this embodiment, when the intelligent agent mechanism 100 performs an individual task, it can map the state of performing the individual task into action value to realize interactive decision-making with the environment. It can also extract knowledge from the individual task, modulate it into a concept representation, and combine the action value and the concept modulation value to determine the corresponding individual task concept vector, which is used to represent the environmental interaction mode and dynamic characteristics required to perform the basic action corresponding to the current task.

[0104] For example, in scenarios where multiple robotic arm agents collaborate to complete complex assembly tasks, the individual task concept vector represents the characteristics of different types of parts, the location information of the material bins, and assembly requirements; in scenarios where multiple game agents perform tasks integrating different game skills, the individual task concept vector represents the level design, adversarial behavior patterns, special item effects, and resource management of different games.

[0105] In this embodiment, multiple individual tasks share the same set of concept vectors, ensuring that the agents have a common cognitive basis for the basic interaction logic of the tasks to be performed.

[0106] For example, in scenarios where multiple robotic arm agents collaborate to complete complex assembly tasks, the shared concept vectors of each agent include basic operational skills such as "reaching a designated three-dimensional spatial point (x,y,z)", "grabbing a standard part", "releasing the workpiece", and "rotating the wrist to a specific angle". In scenarios where multiple game agents perform tasks integrating different game skills, the shared concept vectors of the agents include general game meta-skills such as "understanding two-dimensional / three-dimensional spatial navigation", "identifying interactive objects", "understanding goal-oriented behaviors (such as scoring, avoiding danger)" and "basic input control mapping (such as directional key movement, button jump / shoot)".

[0107] In this embodiment, the intelligent agent parses and extracts knowledge from the executable individual tasks to obtain the corresponding individual task concept vectors, which can be stored locally for later reading and retrieval.

[0108] In this embodiment, during the joint training phase of multiple agent structures, the shared concept vectors among the agents change synchronously, that is, the shared concept vectors among the agents remain consistent in each training cycle of the agents.

[0109] In this embodiment, during each training cycle, all agent mechanisms are traversed. For each agent mechanism, a task is randomly selected from the task set, the concept vector corresponding to the task in the agent mechanism is indexed, and experience is acquired through interaction with the environment via a two-stream network structure. The network parameters and concept embeddings are trained alternately in combination with an action selection strategy. Finally, the learning effect is evaluated and the concept representation is optimized to obtain the agent mechanisms after joint training.

[0110] Step 320: Send the individual task concept vector of the first intelligent agent to the second intelligent agent through the concept sharing module; wherein the second intelligent agent is any one of at least two intelligent agent agents, and the first intelligent agent is any other intelligent agent structure among at least two intelligent agent agents besides the second intelligent agent.

[0111] In this embodiment, through the concept sharing module, each agent can load concept vectors unique to other agents and copy the concept vectors of other agents to the corresponding positions of the current agent. After knowledge transfer, each agent can use the shared concepts to solve tasks that it has never encountered before without retraining the network parameters.

[0112] For example, agent A can independently execute individual task 1, agent B can independently execute individual task 2, and agent C can independently execute individual task 3. Each agent task corresponds to the same shared concept vector 0, individual task 1 corresponds to individual task concept vector a, individual task 2 corresponds to individual task concept vector b, and individual task 3 corresponds to individual task concept vector c. For agent B, by loading the individual task concept vectors a and c corresponding to agents A and C (corresponding to the first agent), and sending them to agent B (corresponding to the second agent), agent B, including the shared concept vector 0 and the individual task concept vector ac, can execute the individual tasks corresponding to agents A and C, realizing the transfer and integration of conceptual knowledge among multiple agents.

[0113] Step 330: Assign the individual tasks to be executed to the third agent structure through the task allocation module, so that the third agent can execute the individual tasks to be executed according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein, the third agent structure is at least one of at least two agent structures; the individual tasks to be executed include at least one of different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

[0114] In this embodiment, the task allocation module can distribute tasks according to the content type of the individual task to be executed and the execution object.

[0115] For example, without specifying the execution target, the task allocation module can distribute the individual task to be executed to any intelligent agent. Since any intelligent agent includes a shared concept vector and all individual task concept vectors, any intelligent agent can execute the individual task to be executed.

[0116] When a specific execution target is specified, the task allocation module can distribute the individual task to be executed to the intelligent agent corresponding to the execution target, so that the individual task can be executed. The intelligent agent that has not encountered the individual task before can use the shared concept vector of the task without retraining the network parameters corresponding to the individual task. This separates task knowledge (concept) from problem-solving ability (network parameters), realizes modular learning, and enhances the interpretability and maintainability of the system.

[0117] The collaborative control system for knowledge transfer among multiple agents provided in this invention uses agent mechanisms to perform task parsing, knowledge extraction, and storage for shared tasks and individual tasks, respectively, to obtain shared concept vectors and individual task concept vectors. A concept sharing module sends the individual task concept vectors of the first agent mechanism to the second agent mechanism. A task allocation module allocates the individual tasks to be executed to a third agent mechanism, allowing the third agent to execute the individual tasks based on the shared concept vectors and the corresponding individual task concept vectors. The system of this invention shares only concept representations rather than the entire model parameters, achieving lightweight and efficient knowledge transfer, greatly reducing communication overhead and training costs. Furthermore, through knowledge sharing, each agent does not need to learn all tasks from scratch, improving overall learning efficiency and resource utilization.

[0118] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for operating a cooperative control system for knowledge transfer among multiple agents. This method includes: performing task parsing, knowledge extraction, and storage on the task to be executed by the agent mechanisms to obtain a shared concept vector and individual task concept vectors; wherein the task to be executed includes a shared task and individual tasks; the shared concept vector and individual task concept vectors include task information and task configuration information; different agent mechanisms correspond to different individual tasks, which are obtained through joint training; and the individual task concept vectors of the first agent mechanism are sent to the second agent mechanism through a concept sharing module. The system comprises: a second intelligent agent structure, which is any one of at least two intelligent agent structures; and a first intelligent agent structure, which is any other intelligent agent structure among the at least two intelligent agent structures besides the second intelligent agent structure; a task allocation module allocates individual tasks to be executed to a third intelligent agent structure, so that the third intelligent agent can execute the individual tasks according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein the third intelligent agent structure is at least one of at least two intelligent agent structures; the individual tasks to be executed include at least one of different individual tasks; and the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

[0119] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the operation method of the cooperative control system for knowledge transfer among multiple agents provided by the above methods. The method includes: performing task parsing, knowledge extraction, and storage on the task to be executed by the agent mechanisms to obtain a shared concept vector and an individual task concept vector; wherein the task to be executed includes a shared task and an individual task; the shared concept vector and the individual task concept vector include task information and task configuration information; different agent mechanisms correspond to different individual tasks, and the agent mechanisms obtain the results through joint training. The process involves: sending the individual task concept vector of the first intelligent agent mechanism to the second intelligent agent mechanism via the concept sharing module; wherein the second intelligent agent mechanism is any one of at least two intelligent agent mechanisms, and the first intelligent agent mechanism is any other intelligent agent structure among the at least two intelligent agent mechanisms besides the second intelligent agent mechanism; allocating the individual tasks to be executed to the third intelligent agent structure via the task allocation module, so that the third intelligent agent can execute the individual tasks to be executed according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein the third intelligent agent structure is at least one of at least two intelligent agent mechanisms; the individual tasks to be executed include at least one of different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

[0121] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cooperative control system for knowledge transfer among multiple agents, characterized in that, include: At least two intelligent agent mechanisms are used, each of which performs the same shared task and different individual tasks. For each intelligent agent mechanism, the mechanism performs task parsing, knowledge extraction, and storage on the shared task and individual tasks respectively, to obtain a shared concept vector and an individual task concept vector. The shared concept vector and individual task concept vector include task information and task configuration information. Different intelligent agent mechanisms correspond to different individual tasks, and the mechanisms are obtained through joint training. A concept sharing module is used to send the individual task concept vector of the first intelligent agent mechanism to the second intelligent agent mechanism; wherein the second intelligent agent mechanism is any one of the at least two intelligent agent mechanisms, and the first intelligent agent mechanism is another intelligent agent structure among the at least two intelligent agent mechanisms other than the second intelligent agent mechanism; A task allocation module is used to allocate individual tasks to be executed to a third agent structure, so that the third agent can execute the individual tasks according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein, the third agent structure is at least one of the at least two agent structures; the individual tasks to be executed include at least one of the different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

2. The cooperative control system for knowledge transfer among multiple agents according to claim 1, characterized in that, Each intelligent agent mechanism includes: A concept extraction module is used to process an initial concept vector through a nonlinear activation operation to obtain a concept modulation value; The task solving module is used to process the input features corresponding to an individual task through a nonlinear activation operation to obtain activation values, and then perform a linear transformation operation between the activation values ​​and the concept modulation values ​​to obtain the individual task concept vector. A concept vector storage module is used to store the concept vectors of the individual tasks.

3. The cooperative control system for knowledge transfer among multiple agents according to claim 2, characterized in that, The concept extraction module and the task solving module are a two-stream network structure; The network structure corresponding to the concept extraction module includes multiple hidden layers, each of which is represented by the following formula: c i = Sigmoid(LayerNorm(V i ·c {i-1} + d i )); The network structure corresponding to the task solving module includes multiple hidden layers, each of which is represented by the following formula: h0 = cat(s; a); h i = ReLU(LayerNorm(W i ·h {i-1} + b i )); h i = hh i * c i ; Q(s, a | c i ) = W n ·h {n-1} + b n ; Where c0 is the input of the concept extraction module, c i To solve for the output concept modulation value of the network corresponding to the concept in the i-th layer, sigmoid is the activation function, and V i Let c be the linear weight parameters corresponding to the i-th layer of the concept extraction module. {i-1} Let d be the concept modulation value corresponding to the (i-1)th layer of the network. i hi represents the bias parameters corresponding to the i-th layer of the concept extraction module; h0 is the input to the task solving module; cat is the vector dimension expansion and concatenation operation; hi is the input to the task solving module ... i To find the output activation value of the network for the i-th layer task, ReLU is the activation function, and W is the activation value. i hh represents the linear weight parameters corresponding to the i-th layer of the network in the task solving module. {i-1} b is the temporary activation value corresponding to the (i-1)th layer of the network. i The bias parameters are the bias parameters corresponding to the i-th layer of the task solving module; Q(s, a | c) is the Bellman equation state-action value function corresponding to state s and action a when facing task c; i = 1, 2, …, n.

4. The cooperative control system for knowledge transfer among multiple agents according to claim 2, characterized in that, The various intelligent agent mechanisms are jointly trained through the following steps: For each agent, during different training cycles, sample tasks and corresponding sample individual task concept vectors are randomly obtained from the sample task set. Within each training cycle, the state features corresponding to the sample task are used as the input to the initial task solving module, and the individual task concept vector of the sample is used as the input to the initial concept extraction module. The initial task solving module and the initial concept extraction module are trained alternately using the ε-greedy action strategy and the mean squared error loss function. Under the condition of meeting the training conditions, the trained agent structure is obtained; the sample shared concept vectors corresponding to each agent change synchronously.

5. The cooperative control system for knowledge transfer among multiple agents according to claim 2, characterized in that, The concept extraction module is also used to obtain the initial concept vector based on the task index.

6. The cooperative control system for knowledge transfer among multiple agents according to claim 1, characterized in that, The concept sharing module is also used for: The fourth intelligent agent mechanism is loaded according to the task coordination instruction, and the individual task concept vector of the fourth intelligent agent mechanism is obtained; wherein, the task coordination instruction is determined based on the number and content of the individual tasks to be executed; The task allocation module is also used for: According to the task coordination instruction, the individual task concept vector of the fourth intelligent agent is sent to the fifth intelligent agent so that the fifth intelligent agent can execute the individual task to be executed; wherein the fourth intelligent agent and the fifth intelligent agent are each one of the at least two intelligent agents.

7. The cooperative control system for knowledge transfer among multiple agents according to claim 1, characterized in that, The system also includes: An evaluation and testing module is used to fine-tune and update the learning parameters of at least two intelligent agents based on the task completion rate and knowledge utilization rate of each agent, so as to obtain the updated agent. The task completion rate is determined based on the ratio of the number of individual tasks completed to the number of individual tasks assigned, and the knowledge utilization rate is determined based on the ratio of the number of effective accesses to the concept vector storage module to the total number of accesses.

8. A method for operating a cooperative control system oriented towards knowledge transfer among multiple agents, characterized in that, include: The intelligent agent mechanism performs task parsing, knowledge extraction, and storage for the task to be executed, resulting in a shared concept vector and individual task concept vectors. The task to be executed includes shared tasks and individual tasks. The shared concept vector and individual task concept vector include task information and task configuration information. Different intelligent agents correspond to different individual tasks, and the intelligent agents are obtained through joint training. The concept sharing module sends the individual task concept vector of the first intelligent agent to the second intelligent agent; wherein the second intelligent agent is any one of the at least two intelligent agent agents, and the first intelligent agent is another intelligent agent structure among the at least two intelligent agent agents other than the second intelligent agent. The task allocation module assigns individual tasks to a third agent structure, so that the third agent can execute the individual tasks according to the shared concept vector and the individual task concept vector corresponding to the individual tasks to be executed; wherein, the third agent structure is at least one of the at least two agent structures; the individual tasks to be executed include at least one of the different individual tasks; the individual tasks to be executed are used to realize multi-robot collaboration or multi-game interaction.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the operation method of the cooperative control system for knowledge transfer among multiple agents as described in claim 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the operation method of the cooperative control system for knowledge transfer among multiple agents as described in claim 8.

Images