A federated transfer learning enhanced multi-agent inter-vehicle dynamic regulation method

Abstract

The application discloses a kind of federal transfer learning enhanced multi-agent workshop dynamic regulation method, establishes the distributed flexible flow workshop dynamic scheduling model based on multi-agent system;Using federal learning, the feature extraction network with representation ability is obtained;According to workshop actual task demand and equipment characteristics, combined with the working data after feature processing, based on the feature extraction network, a multi-agent deep reinforcement learning model based on cross sampling is established;The multi-agent deep reinforcement learning model is trained using deep Q learning algorithm;Using federal transfer learning technology, collect the data of other workshops or factories and local similar tasks, and train Q network according to the data;Then, through adaptive weight fusion technology, the effective transfer of knowledge is realized;Complete multi-agent workshop dynamic regulation.The application combines the advantages of autonomous decision-making and group intelligence of multi-agent, can more flexibly allocate production tasks, and dynamically adjust production control according to real-time situation.

Description

Technical Field

[0001] This invention belongs to the field of smart factory technology, and in particular relates to a multi-agent workshop dynamic control method enhanced by federated transfer learning. Background Technology

[0002] As the digitalization of manufacturing accelerates, massive amounts of production data are being collected, analyzed, and used to guide production decisions. From equipment operating status to employee operating habits, from material flow to product quality control, every link generates a huge amount of data. This provides unprecedented opportunities to improve production efficiency and quality, but it also raises serious concerns about data privacy.

[0003] Especially in multi-agent systems, multiple agents need to frequently exchange information to complete collaborative tasks. This information may include sensitive data such as technical parameters, production plans, and employee operating habits. If this data is obtained by an unauthorized third party, it may not only damage the company's core competitiveness but also jeopardize employee privacy and the company's trade secrets.

[0004] Currently, with the development of the Industrial Internet of Things (IIoT) and edge computing, data processing and decision-making are increasingly shifting from the cloud to the device level, bringing both new opportunities and challenges to data security. In summary, given these challenges and opportunities, there is an urgent need for an effective, reliable, and highly generalizable dynamic scheduling strategy for the workshop. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a multi-agent workshop dynamic control method enhanced by federated transfer learning. This technique combines the advantages of autonomous decision-making by multiple agents with swarm intelligence, enabling more flexible allocation of production tasks and dynamic adjustment of production control based on real-time context.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a multi-agent workshop dynamic control method enhanced by federated transfer learning, comprising the following steps:

[0007] S1: Establish a distributed flexible flow workshop dynamic scheduling model based on a multi-agent system;

[0008] S2: Collect the working data of each device in the workshop, use edge computing for preliminary data processing and feature extraction; use federated learning to enable agents distributed on different devices to perform distributed learning together, and obtain a feature extraction network with representational capabilities.

[0009] S3: Based on the actual task requirements and equipment characteristics of the workshop, and combined with the work data after feature processing, a multi-agent deep reinforcement learning model based on cross-sampling is established based on the feature extraction network. In the process of establishing the multi-agent deep reinforcement learning model, the state space S, action space A, and state transition function T of each agent are defined, and the reward function R is constructed by transforming the distributed flexible flow workshop dynamic scheduling model in step S1 into an optimization problem.

[0010] S4: The multi-agent deep reinforcement learning model is trained using the deep Q-learning algorithm, including: an experience replay mechanism, which randomly samples from the experience buffer and updates the Q-network; and a fixed target network policy update frequency, which is used to stabilize the training process.

[0011] S5: Using federated transfer learning technology, data on similar tasks from other workshops or factories are collected and the Q-network is trained based on this data; subsequently, through adaptive weight fusion technology, effective knowledge transfer is achieved; and dynamic control of the multi-agent workshop is completed.

[0012] Furthermore, in step S1, a distributed flexible workflow dynamic scheduling model based on a multi-agent system is established, including:

[0013] (1) Establish an objective function to minimize the maximum completion time of the factory;

[0014] (2) Establish decision variables;

[0015] (3) Constraint: At any given time, the same process can only be performed on the same machine;

[0016] At any given time, the same machine can only perform one process.

[0017] Each step in the process requires processing time;

[0018] The transport time of any workpiece between two machines shall not be less than the placement time of the workpiece in the machine's dwell area.

[0019] The time for each workpiece to be completed in the i-th process is no earlier than the sum of the time for the i-1-th process to be completed, the transportation time, and the processing time of that process considering efficiency loss;

[0020] The completion time of the h-th product shall not be less than the sum of the processing time of each component and the product assembly time;

[0021] The total processing time shall not be less than the sum of the completion times of all products;

[0022] The factory's maximum completion time shall not be less than the total processing time.

[0023] Furthermore, in step S2, federated learning is used to enable agents distributed across different devices to jointly perform distributed learning, resulting in a feature extraction network with representational capabilities, including the following steps:

[0024] S21: Initialize the server model components, including the feature extractor T, classifier Q, and distribution generator G with parameters w. t ,w q and w g And the client-side model components, including the discriminator D parameter w d ;

[0025] S22: The server model parameters are sent to the client, and the client uses the local discriminator in combination with local data to train the model.

[0026] S23: After completing local training, the client will send the parameter w t ,w q and w g Uploaded to the server, the server aggregates model parameters from all clients to obtain updated parameters;

[0027] S24: The server sends the aggregated updated parameters to the client. After multiple rounds of server-client interaction, the feature extraction network is fully trained.

[0028] Furthermore, in step S22: the server model parameters are sent to the client, and the client uses a local discriminator combined with local data for training, including the following steps:

[0029] First, update parameter w t ,w q To achieve minimization;

[0030] Then, update parameter w. g ,w d To achieve a minimum.

[0031] Furthermore, in step S23: after completing local training, the client will transfer the parameter w t ,w q and w g Uploaded to the server, the server aggregates model parameters from all clients to obtain new parameters. The parameter update formula is:

[0032]

[0033] in, ω represents the local parameters uploaded by client i in round t. i Let x be local data, label be the label, and ω be the ω value. t The parameters are sent to the server, where T is the maximum number of training rounds. This indicates the number of samples used for training client i. Represents the sample-level loss function. This represents the training set.

[0034] Furthermore, in step S4, the multi-agent deep reinforcement learning model is trained using a deep Q-learning algorithm, including the following steps:

[0035] S41: Initialize the experience replay buffer Randomly initialize Q network parameters Target network parameters Obtain the initial state S0;

[0036] S42: Calculate the Q-value using the Q-network and select the action;

[0037] S43: Perform average cross-sampling on the selected action and select the final action using the current state;

[0038] S44: Store the historical operation data from the previous step into the experience replay buffer. Randomly sample a small batch of state transitions from the experience replay buffer;

[0039] S45: Based on the optimization objective, the Q-value network is updated using the mini-batch gradient descent method;

[0040] Then, update the parameters of the target network.

[0041] Finally, repeat the above process until training is complete, thus obtaining a Q-network adapted to local client tasks.

[0042] Furthermore, in step S42, the Q-value is calculated using the Q-network, and an action is selected according to the following formula:

[0043]

[0044] Among them, A * Let be the candidate action space set, and s be the state vector. It is the first The first candidate action within the action space. It is the first The k-th candidate action within the action space. Yes, yes, the first The i-th candidate action in the action space, {n1,...,n} k} represents the action space A * The action candidate set when performing different production tasks, Q(·) represents the Q network.

[0045] Furthermore, in step S43, average cross-sampling is performed on the selected action, as shown in the formula:

[0046]

[0047] Where ξ(·) represents the sampling operator, For the first The first action space within the action space One candidate action, For the first The i-th candidate action within the action space. For the first The j-th candidate action within the action space. The cross-sampling action space is represented by the current state s obtained through the feature extraction network obtained in step S2;

[0048] Then, using the current state s, the final action is selected, using the following formula:

[0049]

[0050] Where q(a|s) represents the conditional probability of performing action a when the state is s. For the first The first candidate action within the action space, a k Let Q(·) represent the Q-network, which is the k-th action space.

[0051] Furthermore, in step S5, federated transfer learning is employed to adaptively set the client network aggregation weight coefficients by evaluating the quality of the client training data, thereby achieving knowledge transfer of the Q network in step S4, including the following steps:

[0052] S51: Initialize server model parameters The global model is then sent to the client; after multiple rounds of training on the client, the feature vectors of the source client model are... With model parameters Send back to the server;

[0053] S52: Server Model Parameters Source client model feature vector and model parameters Send to the target client and obtain the feature vector of the target client's unlabeled data.

[0054] S53: Calculate the distance between the feature vectors of multiple source clients and the target client, and then use the distance to calculate the adaptive aggregation weight coefficient;

[0055] S54: Upload the weight coefficients to the server, and repeat this process to train a more adaptive Q-network.

[0056] Furthermore, the distance between the feature vectors of multiple source clients and the target client is calculated using the following formula:

[0057]

[0058] Among them, κ<x,·> Represents the reproducing kernel Hilbert space function;

[0059] The adaptive aggregation weight coefficient is then calculated using the distance; the adaptive aggregation weight coefficient is then calculated as follows:

[0060] m represents the total number of clients.

[0061] The beneficial effects of adopting this technical solution are:

[0062] This invention achieves efficient, stable, and secure production process operation through multi-agent collaborative work. This technology combines the advantages of autonomous decision-making by multiple agents with swarm intelligence, enabling more flexible allocation of production tasks and dynamic adjustment of production strategies based on real-time context. Employing edge computing and federated learning techniques, it effectively combines the advantages of data privacy with centralized learning. Agents can perform data preprocessing locally and participate in distributed learning, ensuring data security while achieving cross-device knowledge sharing. Using a cross-sampling-based DQN model, coupled with experience replay and a fixed-target network strategy, significantly improves the stability and efficiency of deep reinforcement learning. Furthermore, the introduction of federated transfer learning technology gives this technology excellent scalability, allowing for easy integration of new agents to meet ever-changing production needs.

[0063] This invention provides greater flexibility and adaptability for dynamic scheduling in flexible assembly lines through distributed modeling based on a multi-agent system. Independent policy-action modeling for each agent enables rapid adaptation to environmental changes, ensuring production efficiency. An improved deep Q-learning method is proposed, employing a cross-sampling strategy to address the inefficient exploration problem of agents in large-scale action spaces. A federated knowledge transfer algorithm that automatically adjusts based on edge data quality is designed. Considering the domain differences and data quality variations among different training clients, agents can learn not only from their local environment but also from other workshops or factories, further enhancing the model's generalization ability and scheduling performance. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of a multi-agent workshop dynamic control method enhanced by federated transfer learning according to the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings.

[0066] In this embodiment, see Figure 1 As shown, this invention proposes a multi-agent workshop dynamic control method enhanced by federated transfer learning, comprising the following steps:

[0067] S1: Establish a distributed flexible flow workshop dynamic scheduling model based on a multi-agent system;

[0068] S2: Collect the working data of each device in the workshop, use edge computing for preliminary data processing and feature extraction; use federated learning to enable agents distributed on different devices to perform distributed learning together, and obtain a feature extraction network with representational capabilities.

[0069] S3: Based on the actual task requirements and equipment characteristics of the workshop, and combined with the work data after feature processing, a multi-agent deep reinforcement learning model based on cross-sampling is established based on the feature extraction network. In the process of establishing the multi-agent deep reinforcement learning model, the state space S, action space A, and state transition function T of each agent are defined, and the reward function R is constructed by transforming the distributed flexible flow workshop dynamic scheduling model in step S1 into an optimization problem.

[0070] S4: The multi-agent deep reinforcement learning model is trained using the deep Q-learning algorithm, including: an experience replay mechanism, which randomly samples from the experience buffer and updates the Q-network; and a fixed target network policy update frequency, which is used to stabilize the training process.

[0071] S5: Using federated transfer learning technology, data on similar tasks from other workshops or factories are collected and the Q-network is trained based on this data; subsequently, through adaptive weight fusion technology, effective knowledge transfer is achieved; and dynamic control of the multi-agent workshop is completed.

[0072] As an optimization of the above embodiments, in step S1, a distributed flexible assembly line dynamic scheduling model based on a multi-agent system is established, including:

[0073] (1) Establish the objective function to minimize the maximum completion time of the factory: minC max (π), C max (π) represents the maximum completion time of the factory, and π represents the scheduling sequence.

[0074] (2) Establish decision variables: x i,k,f x ∈{0,1} i,k,fThis represents the execution status of the i-th process on the k-th machine in the f-th factory;

[0075] (3) Constraint: At any given time, the same process can only be performed on the same machine. F represents the number of factories, m represents the number of machines in the workshop, and n represents the process number;

[0076] At any given time, a single machine can only perform one process:

[0077] Each step in the process requires processing time. D f,k,i Let be a continuous variable representing the time when workpiece i leaves machine k and workshop f;

[0078] The transport time of any workpiece between two machines shall not be less than the time the workpiece is placed in the machine's dwell area: t η ≥χ i,h,f , t η For the transfer time, χ i,h,f This represents the dwell time of workpiece j in the buffer zone between factory h and machine k;

[0079] The time for each workpiece to complete the processing of the i-th operation is no earlier than the sum of the time for completing the processing of the (i-1)-th operation, the transportation time, and the processing time of that operation considering efficiency losses. D f,k,i Let φ be a continuous variable representing the departure time of workpiece i-1 in machine k and workshop f. j,k,f O represents the efficiency depletion factor of equipment k in factory f when processing product j. i,j Let i be the processing time for the i-th operation of the j-th workpiece;

[0080] The completion time of the h-th product shall not be less than the sum of the processing time of each component and the product assembly time: B h The completion time of the h-th product; The i-th workpiece required to assemble the h-th product; The total number of parts required to assemble the h-th product;

[0081] The total processing time shall not be less than the sum of the completion times of all products: S represents the quantity of products;

[0082] The factory's maximum completion time shall not be less than the total processing time: C max (π)≥T, where T is the total processing time.

[0083] As an optimization of the above embodiment, in step S2, federated learning is used to enable agents distributed on different devices to jointly perform distributed learning, thereby obtaining a feature extraction network with representational capabilities, including the following steps:

[0084] S21: Initialize the server model components, including the feature extractor T, classifier Q, and distribution generator G with parameters w. t ,w q and w g And the client-side model components, including the discriminator D parameter w d ;

[0085] S22: The server model parameters are sent to the client, and the client uses a local discriminator combined with local data for training; including the following steps:

[0086] First, update parameter w t ,w q To achieve minimization, the formula is:

[0087]

[0088] Where q(label|data) location ) represents the conditional probability distribution for inferring data categories from local data, δ represents the regularization parameter, and label max Indicates the upper limit of the category, data location Indicates local data;

[0089] Then, update parameter w. g ,w d To achieve the minimum form; the formula is:

[0090]

[0091]

[0092]

[0093]

[0094] in, Let k represent the probability distribution. i Let represent the i-th category, and ρ1+ρ2+ρ3=1 represent the sample tradeoff coefficient.

[0095] S23: After completing local training, the client will send the parameter w t ,w q and w g Uploaded to the server, the server aggregates model parameters from all clients to obtain updated parameters;

[0096] The parameter update formula is:

[0097]

[0098] in, ω represents the local parameters uploaded by client i in round t. i Let x be local data, label be the label, and ω be the ω value. t The parameters are sent to the server, where T is the maximum number of training rounds. This indicates the number of samples used for training client i. Represents the sample-level loss function. This represents the training set.

[0099] S24: The server sends the aggregated updated parameters to the client. After multiple rounds of server-client interaction, the feature extraction network is fully trained.

[0100] As an optimization of the above embodiment, in step S4, the multi-agent deep reinforcement learning model is trained using the deep Q-learning algorithm, including the following steps:

[0101] S41: Initialize the experience replay buffer Randomly initialize Q network parameters Target network parameters Obtain the initial state S0;

[0102] S42: Calculate the Q-value using the Q-network and select the action;

[0103] Select the action according to the following formula:

[0104]

[0105] Among them, A * Let be the candidate action space set, and s be the state vector. It is the first The first candidate action within the action space. It is the first The k-th candidate action within the action space. Yes, yes, the first The i-th candidate action in the action space, {n1,...,n} k} represents the action space A * The action candidate set when performing different production tasks, Q(·) represents the Q network.

[0106] S43: Perform average cross-sampling on the selected action, using the following formula:

[0107]

[0108] Where ξ(·) represents the sampling operator, For the first The first action space within the action space One candidate action, For the first The i-th candidate action within the action space. For the first The j-th candidate action within the action space. The cross-sampling action space is represented by the current state s obtained through the feature extraction network obtained in step S2;

[0109] Then, using the current state s, the final action is selected, using the following formula:

[0110]

[0111] Where q(a|s) represents the conditional probability of performing action a when the state is s. For the first The first candidate action within the action space, a k Let Q(·) represent the Q-network, which is the k-th action space.

[0112] S44: Transfer the historical operation data from the previous step (s t ,a t ,R t ,s t+1 Stored in the experience replay buffer Randomly sample a small batch of state transitions from the experience replay buffer;

[0113] For each sample, there is a formula:

[0114]

[0115] S45: Based on the optimization objective, the Q-value network is updated using the mini-batch gradient descent method;

[0116] The optimization objective is:

[0117] Then, update the parameters of the target network.

[0118] Finally, repeat the above process until training is complete, thus obtaining a Q-network adapted to local client tasks.

[0119] As an optimization of the above embodiment, in step S5, federated transfer learning technology is used. By evaluating the quality of the client training data, the client network aggregation weight coefficients are adaptively set, thereby realizing the knowledge transfer of the Q network in step S4, including the following steps:

[0120] S51: Initialize server model parameters The global model is then sent to the client; after multiple rounds of training on the client, the feature vectors of the source client model are... With model parameters Send back to the server;

[0121] S52: Server Model Parameters Source client model feature vector and model parameters Send to the target client and obtain the feature vector of the target client's unlabeled data.

[0122] S53: Calculate the distance between the feature vectors of multiple source clients and the target client, and then use the distance to calculate the adaptive aggregation weight coefficient;

[0123] The distance between the feature vectors of multiple source clients and the target client is calculated using the following formula:

[0124]

[0125] Among them, κ<x,·> Represents the reproducing kernel Hilbert space function;

[0126] The adaptive aggregation weight coefficient is then calculated using the distance; the adaptive aggregation weight coefficient is then calculated as follows:

[0127] m represents the total number of clients.

[0128] S54: Upload the weight coefficients to the server, and repeat this process to train a more adaptive Q-network.

[0129] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A multi-agent workshop dynamic control method enhanced by federated transfer learning, characterized in that, Including the following steps: S1: Establish a distributed flexible flow workshop dynamic scheduling model based on a multi-agent system; S2: Collect the working data of each device in the workshop, use edge computing for preliminary data processing and feature extraction; use federated learning to enable agents distributed on different devices to perform distributed learning together, and obtain a feature extraction network with representation capabilities; S3: Based on the actual task requirements and equipment characteristics of the workshop, and combined with the work data after feature processing, a multi-agent deep reinforcement learning model based on cross-sampling is established based on the feature extraction network. In the process of establishing the multi-agent deep reinforcement learning model, the state space S, action space A, and state transition function T of each agent are defined, and the reward function R is constructed by transforming the distributed flexible flow workshop dynamic scheduling model in step S1 into an optimization problem. S4: The multi-agent deep reinforcement learning model is trained using the deep Q-learning algorithm, including: an experience replay mechanism, which randomly samples from the experience buffer and updates the Q-network; and a fixed target network policy update frequency, which is used to stabilize the training process. In step S4, the multi-agent deep reinforcement learning model is trained using the deep Q-learning algorithm, including the following steps: S41: Initialize the experience replay buffer Randomly initialize Q network parameters Target network parameters Obtain the initial state S0; S42: Calculate the Q-value using the Q-network, select the action according to the following formula: ； in, Let be the candidate action space set, and s be the state vector. It is the first The first candidate action within the action space. It is the first The k-th candidate action within the action space. Yes, yes, the first The i-th candidate action within the action space. Representing the action space A set of action candidates for performing different production tasks. Represents a Q-network; S43: Perform average cross-sampling on the selected action, and select the final action using the current state, using the following formula: ； in, Indicates the sampling operator, For the first The first action space within the action space One candidate action, For the first The i-th candidate action within the action space. For the first The first action space within the action space One candidate action, The cross-sampling action space is represented by the current state s obtained through the feature extraction network obtained in step S2; Then, using the current state s, the final action is selected, using the following formula: ； in, Let represent the conditional probability of performing action a when the state is s. For the first The first candidate action within the action space. For the k-th action space, Represents a Q-network; S44: Store the historical operation data from the previous step into the experience replay buffer. A small batch of state transitions is randomly sampled from the experience replay buffer; S45: Based on the optimization objective, the Q-value network is updated using the mini-batch gradient descent method; Then, update the parameters of the target network. ; Finally, repeat the above process until training is complete to obtain a Q-network adapted to local client tasks. S5: Using federated transfer learning technology, data on similar tasks from other workshops or factories are collected and the Q-network is trained based on this data; subsequently, through adaptive weight fusion technology, effective knowledge transfer is achieved; and dynamic control of the multi-agent workshop is completed.

2. The method for dynamic control of a multi-agent workshop enhanced by federated transfer learning according to claim 1, characterized in that, In step S1, a distributed flexible workflow dynamic scheduling model based on a multi-agent system is established, including: (1) Establish an objective function to minimize the maximum completion time of the factory; (2) Establish decision variables; (3) Constraint: At any given time, the same process can only be performed on the same machine; At any given time, the same machine can only perform one process. Each step in the process requires processing time; The transport time of any workpiece between two machines shall not be less than the placement time of the workpiece in the machine's dwell area. The time for each workpiece to be completed in the i-th process is no earlier than the sum of the time for the i-1-th process to be completed, the transportation time, and the processing time of that process considering efficiency loss; The completion time of the h-th product shall not be less than the sum of the processing time of each component and the product assembly time; The total processing time shall not be less than the sum of the completion times of all products; The factory's maximum completion time shall not be less than the total processing time.

3. The method for dynamic control of a multi-agent workshop enhanced by federated transfer learning according to claim 1, characterized in that, In step S2, federated learning is used to enable agents distributed across different devices to perform distributed learning, resulting in a feature extraction network with representational capabilities. This includes the following steps: S21: Initialize the server model components, including the feature extractor T, classifier Q, and distribution generator G with parameters w. t w q and w g And the client-side model components, including the discriminator D parameter w d ; S22: The server model parameters are sent to the client, and the client uses the local discriminator in combination with local data for training; S23: After completing local training, the client will send parameter w t w q and w g Uploaded to the server, the server aggregates model parameters from all clients to obtain updated parameters; S24: The server sends the aggregated updated parameters to the client. After multiple rounds of server-client interaction, the feature extraction network is fully trained.

4. The method for dynamic control of a multi-agent workshop enhanced by federated transfer learning according to claim 3, characterized in that, In step S22: The server model parameters are sent to the client, and the client uses the local discriminator in conjunction with local data for training, including the following steps: First, update parameter w t w q To achieve minimization; Then, update parameter w. g w d To achieve a minimum.

5. The multi-agent workshop dynamic control method with federated transfer learning enhancement according to claim 3, characterized in that, In step S23: After completing local training, the client will transfer the parameter w t w q and w g Uploaded to the server, the server aggregates model parameters from all clients to obtain new parameters. The parameter update formula is: ； in, This represents the local parameters uploaded by client i in round t, where x is the local data and label is the tag. The parameters are sent to the server, where T is the maximum number of training rounds. This indicates the number of samples used for training client i. Represents the sample-level loss function. This represents the training set.

6. The method for dynamic control of a multi-agent workshop enhanced by federated transfer learning according to claim 1, characterized in that, In step S5, federated transfer learning is employed. By evaluating the quality of the client training data, the aggregation weight coefficients of the client network are adaptively set, thereby realizing the knowledge transfer of the Q network in step S4. This includes the following steps: S51: Initialize server model parameters The global model is then sent to the client; after multiple rounds of training on the client, the feature vectors of the source client model are... With model parameters Send back to the server; S52: Server Model Parameters Source client model feature vector and model parameters Send to the target client and obtain the feature vector of the target client's unlabeled data. ; S53: Calculate the distance between the feature vectors of multiple source clients and the target client, and then use the distance to calculate the adaptive aggregation weight coefficient; S54: Upload the weight coefficients to the server, and repeat this process to train a more adaptive Q-network.

7. The multi-agent workshop dynamic control method with federated transfer learning enhancement according to claim 6, characterized in that, The distance between the feature vectors of multiple source clients and the target client is calculated using the following formula: ； in, Represents the reproducing kernel Hilbert space function; The adaptive aggregation weight coefficient is then calculated using the distance; the adaptive aggregation weight coefficient is then calculated as follows: , where m is the total number of clients.

Images