An intelligent control method based on knowledge distillation and multi-agent reinforcement learning

Abstract

The present invention discloses an intelligent control method based on knowledge distillation and multi-agent reinforcement learning, comprising the following steps: using an environmental monitoring agent to perform graph-structured coding modeling on environmental parameters acquired by sensor equipment; using a decision-making and coordination agent to analyze various environmental variables based on the output of the environmental monitoring agent using a causal relationship model, and exploring the causal relationship between environmental changes and equipment control; based on a multi-agent reinforcement learning method, improving the autonomous decision-making ability and collaborative control of the decision-making and coordination agent; based on distributed prototype knowledge distillation, improving the learning and reasoning ability of the decision-making and coordination agent on edge devices; this method has the characteristics of improving response speed in a dynamic environment, gradually improving overall control performance and accuracy, thereby achieving adaptive adjustment, optimizing equipment operation and reducing energy consumption.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control, and in particular to an intelligent control method based on knowledge distillation and multi-agent reinforcement learning. Background Art

[0002] In the modern equipment management system, the pursuit of both efficiency and safety has become the key, especially in unattended scenarios, where technological advancement and practicality are particularly important. Intelligent control systems stand out in this context, demonstrating their excellent performance in precise control, energy saving and environmental protection, and automated operation, and have successfully verified their wide application value.

[0003] The current intelligent control system can capture tiny changes in the environment in real time by integrating highly sensitive temperature and humidity sensing technology. These sensors are not only responsive, but also accurate in data, ensuring that the temperature and humidity in the warehouse are always kept within the range that is most suitable for storage standards. At the same time, the intelligent control logic is closely integrated with these sensing technologies, giving the system a highly automated decision-making capability. The system can intelligently adjust key environmental variables such as temperature and humidity based on real-time collected environmental data, and automatically control air conditioning systems, ventilation equipment and other regulating equipment.

[0004] Although current intelligent control systems excel in automation, precision control, and energy efficiency optimization, they also have some defects and limitations. First, current intelligent control systems are highly dependent on preset control rules and thresholds. This means that the system's response is entirely based on predefined logic and cannot flexibly handle new or unknown situations. Although it can make certain adaptive adjustments through feedback mechanisms, existing intelligent control systems are still unable to actively learn or optimize control strategies. For example, the system cannot analyze and summarize new environmental patterns or trends based on long-term data accumulation, and cannot actively adjust set parameters or optimize the operation mode of the equipment. As the environment changes, such as seasonal changes, or equipment aging, the performance of the system may gradually decline. In addition, as the number of sensors and devices increases, the real-time data processing and response speed of intelligent control systems become bottlenecks. In particular, when faced with large amounts of data, the system must make decisions quickly to ensure accurate and real-time control. However, the limitations of centralized data processing architectures often lead to slow or delayed responses. As the number of sensors and devices increases, the amount of real-time data that intelligent control systems need to process also increases significantly. If the system cannot process this data efficiently, it may result in slow or delayed responses. Especially in situations where real-time adjustments are required, such as rapid response to changes in temperature and humidity, data processing and decision-making delays may affect the accuracy of control and the overall performance of the system. Summary of the invention

[0005] To this end, it is necessary to provide an intelligent control method that improves the adaptability, efficiency and accuracy of equipment management and environmental control according to the actual industrial environment.

[0006] To achieve the above objectives, the inventor provides an intelligent control method based on knowledge distillation and multi-agent reinforcement learning, comprising the following steps:

[0007] S1, the environmental parameters obtained by the sensor equipment are modeled by graph structured encoding through the environmental monitoring agent;

[0008] S2, through the decision-making and coordination agent based on the output of the environmental monitoring agent, uses the causal relationship model to analyze various environmental variables and explore the causal relationship between environmental changes and equipment control;

[0009] S3, based on multi-agent reinforcement learning methods, improves the autonomous decision-making ability and collaborative control of decision-making and coordination agents;

[0010] S4, based on distributed prototype knowledge distillation, improves the learning and reasoning capabilities of decision-making and coordination agents on edge devices.

[0011] As a preferred embodiment of the present invention, S1 comprises the steps of:

[0012] S101, formalizing the environmental parameters into [Type, Date, Loc, Tem, Hum], where Type represents the type of the current environment, Date represents the time, Loc represents the longitude and latitude coordinates, Tem represents the temperature, and Hum represents the humidity;

[0013] S102: Initialize the features of the above environment parameters respectively, and use an embedding layer composed of 1×1 convolution for the environment type. Map it to a d-dimensional environment type feature vector , the expression is:

[0014] ;

[0015] For the time parameter, the time feature vector is generated by combining sine and cosine. , the expression is:

[0016] ;

[0017] in, Indicates different time units. Indicates the corresponding period length;

[0018] The longitude and latitude coordinates Loc are first discretized and embedded through a 1×1 convolution layer. Convert the feature into a d-dimensional coordinate vector , the expression is:

[0019] ;

[0020] in, and Represent the longitude and latitude coordinates respectively. and Respectively represent the number of longitude and latitude coordinates;

[0021] For temperature Tem and humidity Hum, directly use the embedding layer composed of 1×1 convolution Perform feature mapping to generate temperature feature vector and humidity feature vector ,expression:

[0022] ;

[0023] S103, use the graph generation network to perform graph structured modeling on the parameter vector in S102, the graph generation network The environment type vector As the center point of the graph, the correlation between the center point and the remaining parameter vectors is calculated through the variable attention mechanism DA to construct the environment parameter relationship graph S, which is expressed as:

[0024] ,

[0025] ;

[0026] in, , Represents different environmental parameter characteristics, W represents the weight matrix of network learning, and k represents the environmental type feature vector Feature dimension index, there are d indexes in total. express Relative to The mathematical expectation of , MLP stands for multi-layer perceptron;

[0027] S104, according to different types of environments, the environment monitoring agent Divide the environmental parameter relationship graph S into different environmental domains , through the feature groups after cross-domain clustering, we can identify the shared or similar features in different environments. The expression is:

[0028] ;

[0029] in, represents the mean clustering algorithm, represents the eigenvector of the Laplace matrix, I represents the identity matrix, D represents the degree matrix, Indicates different environment domains Constructed similarity matrix.

[0030] As a preferred embodiment of the present invention, S2 comprises the steps of:

[0031] S201, Computing Environment Monitoring Agents via Self-Attention Mechanism Serialization characteristics , the expression is:

[0032] ;

[0033] Where N represents the number of environment monitoring agents, tanh represents the hyperbolic tangent function, and E represents the learnable training parameters. represents the tth environment monitoring agent, represents the rest of the environment monitoring agents, Represents similarity calculation;

[0034] S202, Decision-making and Coordination Agent Environmental Monitoring Agent Serialized feature representation of As the basic object of causal reasoning, the complex changes of the environment are simulated by the environment change module based on the temporal self-attention mechanism. The change of one of the environment monitoring agents at time x depends on its change in the past y hours and the change trend of other environmental parameters, expressed as:

[0035] ;

[0036] in, Represents an environment monitoring agent After simulating complex changes, R represents the learnable parameters. represents the change in the past y hours, represents the changing trend of other environmental parameters, TA() represents the temporal self-attention mechanism;

[0037] S203, Decision-making and Coordination Agent Use two reasoning modules with the same structure but different weights to simulate the front and back environmental monitoring agents to analyze the results and generate corresponding equipment control actions. The expression is:

[0038] ;

[0039] ;

[0040] in, and It is the reasoning module, which consists of three graph convolutional layers and two linear layers. represents the environment monitoring agent before simulation, represents the simulated environment monitoring agent, Representing decision-making and coordination agents Generate device control actions for the pre-simulation environment. Representing decision-making and coordination agents Generate device control actions for the simulated environment.

[0041] As a preferred embodiment of the present invention, S2 further comprises the steps of:

[0042] S204, by reconstructing decision-making and coordination agents The loss function improves the stability of the agent under complex environmental changes. The expression is:

[0043] ;

[0044] Where M and N represent the number of decision-making and coordination agents and environment monitoring agents, respectively, and e represents the natural base number. Represents the vector squared norm operation.

[0045] As a preferred embodiment of the present invention, S3 comprises the steps of:

[0046] S301, define the joint observation space U of the environment monitoring agent, expressed as:

[0047] ;

[0048] S302, based on the joint observation space U, the decision-making and coordination agents explore different joint observation states and propose new learning goals , improve the inference ability and collaborative control of the overall state of the system, the expression is:

[0049] ;

[0050] in, represents the information entropy of the joint observation space U, represents the mutual correlation information between decision-making and coordination agents in the joint observation space, and M represents the number of decision-making and coordination agents.

[0051] As a preferred embodiment of the present invention, S3 further comprises the steps of:

[0052] S303, by setting the reward function in the reinforcement learning algorithm , adjust and optimize the behavior of decision-making and coordination agents, the expression is:

[0053] ;

[0054] in, Indicates The prediction of the control state by a decision-making and coordination agent, represents the environment variables currently observed locally, represents the joint observation of all environmental variables, Represents the log-probability of the edge device action control prediction for the next time step given the environment state and the joint observation.

[0055] As a preferred embodiment of the present invention, S4 comprises the steps of:

[0056] S401, decomposes the complex decision-making and coordination agent into several lightweight prototype agents, expressed as:

[0057] ;

[0058] in, Indicates different control tasks, represents the action space of decision-making and coordination agents, Represents the mathematical expectation of different control tasks for decision-making and coordination agents.

[0059] S402, by the function of knowledge distribution difference Measuring Edge Devices With the prototype agent The consistency of knowledge distribution between them is expressed as:

[0060] ;

[0061] in, Represents the knowledge representation mapping on edge devices, represents the normalization function, Represents the mathematical expectation of the edge device for the prototype agent;

[0062] As a preferred embodiment of the present invention, S4 further comprises the steps of:

[0063] S403, introduce knowledge refinement loss function , the local knowledge is refined and the expression is:

[0064] ;

[0065] in, Represents different edge devices The index of represents the prototype agent weight, Represent different weights, Represents the mathematical expectation of the current edge device for other edge devices.

[0066] Different from the prior art, the above technical solution has the following beneficial effects:

[0067] (1) Through the collaboration between multiple agents, this method enables the system to actively learn and optimize control strategies, no longer relying on preset thresholds and rules. Each agent is responsible for different devices or environmental variables. Through continuous interaction and learning, they gradually improve the overall control performance in a dynamic environment, thereby achieving adaptive adjustment, optimizing equipment operation and reducing energy consumption.

[0068] (2) This method extracts knowledge from complex agents and transfers it to lightweight agents, enabling lightweight agents to run efficiently on edge devices with limited resources. It also disperses data processing tasks through knowledge distillation, reduces the computing pressure of the central server, and effectively improves the real-time response capability of the system. In addition, edge devices can make decisions quickly based on local environmental data, reduce dependence on central servers, effectively improve response speed, and maintain control accuracy when the device and environment change. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A flowchart of the method described in the specific implementation mode; DETAILED DESCRIPTION

[0070] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0071] like Figure 1 As shown, this embodiment provides an intelligent control method based on knowledge distillation and multi-agent reinforcement learning, including the following steps:

[0072] S1, the environmental monitoring agent models the environmental parameters obtained by the sensor equipment using graph structured coding; that is, through graph structured coding, the system can not only capture the dependencies between the environmental parameters, but also integrate these relationships into the subsequent decision-making process. Then, the environmental monitoring agent further enhances the interactivity of these parameters based on the cross-domain feature clustering algorithm, identifies hidden patterns and potential laws, and supports more accurate control strategies.

[0073] S2, the decision-making and coordination agent analyzes various environmental variables based on the output of the environmental monitoring agent through the causal relationship model, and explores the causal relationship between environmental changes and equipment control; that is, the agent can calculate the impact of different environmental factors, such as temperature, humidity, air quality, etc., on equipment operation in real time, and predict future change trends. Through the causal relationship model, the inference agent can not only understand the control effect of the current equipment, but also select the optimal equipment control plan by simulating the potential consequences of different control strategies;

[0074] S3, based on the method of multi-agent reinforcement learning, improves the autonomous decision-making ability and collaborative control of decision-making and coordination agents; that is, by setting the reward function, the agent can not only independently learn its own optimal strategy, but also optimize the control strategy of the entire environment through interaction and collaboration with the environment monitoring agent;

[0075] S4, based on distributed prototype knowledge distillation, improves the learning and reasoning capabilities of decision-making and coordination agents on edge devices, ensuring that decision-making tasks can be completed efficiently and accurately in resource-constrained edge computing environments; that is, by implementing a lightweight prototype distillation model on each edge device and working in collaboration with other agents in the distributed system, the overall intelligence and adaptability of the system can be significantly improved.

[0076] In the specific implementation process of step S in the above embodiment, S1 includes the following steps:

[0077] S101, formalize the environmental parameters as [Type, Date, Loc, Tem, Hum], where Type represents the type of the current environment, such as laboratory, factory, etc.; Date represents time, which can be composed of year-month-day-hour-minute-second; Loc represents longitude and latitude coordinates, Tem represents temperature, and Hum represents humidity;

[0078] S102, perform feature initialization for the above environment parameters respectively; specifically, use an embedding layer composed of 1×1 convolution for the environment type Map it to a d-dimensional environment type feature vector , the expression is:

[0079] ;

[0080] For the time parameter, the time feature vector is generated by combining sine and cosine. , the expression is:

[0081] ;

[0082] in, Indicates different time units. Indicates the corresponding period length;

[0083] The longitude and latitude coordinates Loc are first discretized and embedded through a 1×1 convolution layer. Convert the feature into a d-dimensional coordinate vector , the expression is:

[0084] ;

[0085] in, and Represent the longitude and latitude coordinates respectively. and Respectively represent the number of longitude and latitude coordinates;

[0086] For temperature Tem and humidity Hum, directly use the embedding layer composed of 1×1 convolution Perform feature mapping to generate temperature feature vector and humidity feature vector ,expression:

[0087] ;

[0088] S103, use the graph generation network to perform graph structured modeling on the parameter vector in S102, the graph generation network The environment type vector As the center point of the graph, the correlation between the center point and the remaining parameter vectors is calculated through the variable attention mechanism DA to construct the environment parameter relationship graph S, which is expressed as:

[0089] ,

[0090] ;

[0091] in, , Represents different environmental parameter characteristics, W represents the weight matrix of network learning, and k represents the environmental type vector Feature dimension index, there are d indexes in total express Relative to The mathematical expectation of , MLP stands for multi-layer perceptron;

[0092] S104, according to different types of environments, the environment monitoring agent Divide the environmental parameter relationship graph S into different environmental domains , through the feature groups after cross-domain clustering, we can identify the shared or similar features in different environments and enhance their interactions in different environments. The expression is:

[0093] ;

[0094] in, represents the mean clustering algorithm, represents the eigenvector of the Laplace matrix, I represents the identity matrix, D represents the degree matrix, Indicates different environment domains Constructed similarity matrix.

[0095] In the specific implementation process of step S2 in the above embodiment, S2 includes the steps of:

[0096] S201, Decision-making and Coordination Agent Environmental Monitoring Agent The provided environmental data is analyzed to find out the relationship between different environmental variables, such as temperature and humidity, and device control based on the causal relationship model; first, the environmental monitoring agent is calculated through the self-attention mechanism Serialization characteristics , the expression is:

[0097] ;

[0098] Where N represents the number of environment monitoring agents, tanh represents the hyperbolic tangent function, and E is a learnable training parameter. represents the tth environment monitoring agent, represents the rest of the environment monitoring agents, Represents similarity calculation;

[0099] S202, Decision-making and Coordination Agent Environmental Monitoring Agent Serialized feature representation of As the basic object of causal reasoning, the complex changes of the environment are simulated by the environment change module based on the temporal self-attention mechanism. The change of one of the environment monitoring agents at time x depends on its change in the past y hours and the change trend of other environmental parameters, expressed as:

[0100] ;

[0101] in, Represents an environment monitoring agent The result after simulating complex changes, MLP represents multi-layer perceptron, R represents learnable parameters, Indicates its change in the past y hours, Indicates the changing trend of other environmental parameters, TA() indicates the temporal self-attention mechanismTA() indicates the temporal self-attention mechanism;

[0102] S203, Decision-making and Coordination Agent Use two reasoning modules with the same structure but different weights to simulate the front and back environmental monitoring agents to analyze the results and generate corresponding equipment control actions. The expression is:

[0103] ;

[0104] ;

[0105] in, and It is the reasoning module, which consists of three graph convolutional layers and two linear layers. represents the environment monitoring agent before simulation, represents the simulated environment monitoring agent, Representing decision-making and coordination agents Generate device control actions for the pre-simulation environment. Representing decision-making and coordination agents Generate device control actions for the simulated environment;

[0106] In some embodiments, step S2 further includes: S204, by reconstructing the decision-making and coordination agent The loss function improves the stability of the agent under complex environmental changes. The expression is:

[0107] ;

[0108] Where M and N are the number of decision-making and coordination agents and environment monitoring agents, respectively, and e represents the natural base number. Represents the vector squared norm operation.

[0109] In the specific implementation process of step S3 in the above embodiment, S3 includes the steps of:

[0110] In traditional agent theory algorithms, it is assumed that agents can obtain complete information about the global state of the system; however, in practical applications, agents can often only obtain local information and cannot directly obtain the global state. To solve this problem, in this embodiment S301, a joint observation space U of an environment monitoring agent is first defined, which is expressed as:

[0111] ;

[0112] S302, based on the joint observation space, the decision and coordination agent explores different joint observation states and proposes a new learning goal , improve the ability to infer the overall state of the system, so as to better achieve collaborative control, the expression is:

[0113] ;

[0114] in, represents the information entropy of the joint observation space U, represents the mutual correlation between decision-making and coordination agents in the joint observation space, and M represents the number of decision-making and coordination agents;

[0115] Based on global observation, the decision-making and coordination agent independently learns its own optimal strategy and gradually improves its autonomous decision-making ability in a dynamic environment. For example, the decision-making and coordination agents of air conditioners, humidifiers, ventilation systems and other equipment can independently learn how to operate effectively according to environmental conditions, such as temperature, humidity, and air quality, to ensure that the environment meets the set requirements. In this process, S303, by setting the reward function in the reinforcement learning algorithm , constantly adjust and optimize the decision-making and coordination behavior of intelligent agents, so that they can adaptively make accurate decisions in complex environments, as follows:

[0116] ;

[0117] in, Indicates The prediction of the control state by a decision-making and coordination agent, represents the environment variables currently observed locally, represents the joint observation of all environmental variables, Represents the log-probability of the edge device action control prediction for the next time step given the environment state and the joint observation.

[0118] In the specific implementation process of step S4 in the above embodiment, S4 includes the steps of:

[0119] S401, in order to achieve efficient decision-making and control on edge devices, the present invention decomposes complex decision-making and coordination agents into multiple lightweight prototype agents, each of which focuses on a specific task or local environment control, such as temperature regulation, humidity control, etc. These prototype agents can not only handle their own tasks independently, but also work in collaboration with other prototype models to achieve optimal control of the entire environment. This modular design allows each prototype agent to maintain a certain degree of lightness and run efficiently in a distributed system, as expressed by:

[0120] ;

[0121] in, represents different control tasks, U is the joint observation space, represents the action space of decision-making and coordination agents, Represents the mathematical expectation of different control tasks for decision-making and coordination agents;

[0122] S402, on the central server side, multiple prototype agents that have been trained are used as "teacher models"; these teacher models contain prior knowledge in their respective fields. For example, the air conditioning model may have learned the best temperature adjustment strategy, and the humidifier model may have learned the best humidity control strategy. First, a function of knowledge distribution difference is used To measure edge devices With the prototype agent The consistency of knowledge distribution between them ensures that each edge device can adjust the local model through the prior distribution to make it close to the knowledge of the teacher model. The expression is:

[0123] ;

[0124] in, Knowledge representation mapping on edge devices, is the normalization function, Represents the mathematical expectation of the edge device for the prototype agent;

[0125] In some embodiments, step S4 further includes step: S403, since the local knowledge of different edge devices may vary due to environmental differences, hardware capabilities, etc., a knowledge refinement loss function is further introduced , refine the local knowledge into a distribution similar to the teacher model to ensure the consistency of knowledge distribution on different edge devices. The expression is:

[0126] ;

[0127] in Represents different edge devices The index of represents the prototype agent weight, Represent different weights, Represents the mathematical expectation of the current edge device for other edge devices, It is the vector squared norm operation.

[0128] In order to prove the effectiveness of the above embodiments, different data sets are used for verification. Specifically, the ASHRAE data set collects and publishes energy consumption data and meteorological data of buildings in many countries, with the aim of predicting the future energy consumption of buildings through machine learning and statistical models. Such predictions help improve energy efficiency, optimize the heating, ventilation, and air conditioning (HVAC) systems of buildings, and power dispatching. ASHRAE contains multi-dimensional features, such as building information, energy consumption types, weather conditions, etc., which can be widely used in smart building energy consumption prediction, energy-saving optimization, and intelligent control system development. This data set has wide application value in intelligent control, energy management, and data science research.

[0129] The experimental results are as follows:

[0130]

[0131] The experimental results show that the above experimental results show the comparison results of several models in intelligent control tasks. This method performs best in various indicators, with a determination coefficient of 0.96, a control accuracy mean square error of only 0.03, and the shortest running time of only 0.21 seconds, indicating that it has high control accuracy and excellent operating efficiency. In contrast, the determination coefficient of random forest is 0.75, which is low, the control accuracy mean square error is 2.11, and the running time is 5 minutes and 13 seconds, which is not ideal. Bi-RNN, Bi-LSTM and Bi-GRU have improved control accuracy compared with random forest, but the running time is longer, which is 8 minutes and 15 seconds, 7 minutes and 48 seconds and 9 minutes and 21 seconds respectively, which is suitable for scenarios with low real-time requirements. In summary, this method not only performs well in control accuracy and energy consumption optimization through multi-agent collaboration and knowledge distillation, but also greatly improves the decision-making ability of the system on edge devices in terms of operating efficiency. These beneficial effects enable the present invention to run stably, quickly and efficiently in resource-constrained environments, and are far superior to traditional methods.

[0132] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.

Claims

1. An intelligent control method based on knowledge distillation and multi-agent reinforcement learning, characterized in that: The following steps are involved: S1, the environmental parameters obtained by the sensor equipment are modeled by graph structured encoding through the environmental monitoring agent; S2, through the decision-making and coordination agent based on the output of the environmental monitoring agent, uses the causal relationship model to analyze various environmental variables and explore the causal relationship between environmental changes and equipment control; S3, based on multi-agent reinforcement learning methods, improves the autonomous decision-making ability and collaborative control of decision-making and coordination agents; S4, based on distributed prototype knowledge distillation, improves the learning and reasoning capabilities of decision-making and coordination agents on edge devices; S1 includes the steps: S101, formalize the environmental parameters as [Type, Date, Loc, Tem, Hum], where Indicates the type of the current environment, Date indicates the time, Loc indicates the longitude and latitude coordinates, Tem indicates the temperature, and Hum indicates the humidity; S102: Initialize the features of the above environment parameters respectively, and use an embedding layer composed of 1×1 convolution for the environment type. Map it to a d-dimensional environment type feature vector , the expression is: ； For the time parameter, the time feature vector is generated by combining sine and cosine. , the expression is: ； in, Indicates different time units. Indicates the corresponding period length; The longitude and latitude coordinates Loc are first discretized and embedded through a 1×1 convolution layer. Convert the feature into a d-dimensional coordinate vector , the expression is: ； in, and Represent the longitude and latitude coordinates respectively. and Respectively represent the number of longitude and latitude coordinates; For temperature Tem and humidity Hum, directly use the embedding layer composed of 1×1 convolution Perform feature mapping to generate temperature feature vector and humidity feature vector ,expression: ； S103, use the graph generation network to perform graph structured modeling on the parameter vector in S102, the graph generation network The environment type vector As the center point of the graph, the correlation between the center point and the remaining parameter vectors is calculated through the variable attention mechanism DA to construct the environment parameter relationship graph S, which is expressed as: ， ； in, , Represents different environmental parameter characteristics, W represents the weight matrix of network learning, and k represents the environmental type feature vector Feature dimension index, there are d indexes in total. express Relative to The mathematical expectation of , MLP stands for multi-layer perceptron; S104, according to different types of environments, the environment monitoring agent Divide the environmental parameter relationship graph S into different environmental domains , through the feature groups after cross-domain clustering, we can identify the shared or similar features in different environments. The expression is: ； in, represents the mean clustering algorithm, represents the eigenvector of the Laplace matrix, I represents the identity matrix, D represents the degree matrix, Indicates different environment domains Constructed similarity matrix.

2. The intelligent control method based on knowledge distillation and multi-agent reinforcement learning according to claim 1 is characterized in that: S2 includes the steps: S201, Computing Environment Monitoring Agents via Self-Attention Mechanism Serialization characteristics , the expression is: ； Where N represents the number of environment monitoring agents, tanh represents the hyperbolic tangent function, and E represents the learnable training parameters. represents the tth environment monitoring agent, represents the rest of the environment monitoring agents, Represents similarity calculation; S202, Decision-making and Coordination Agent Environmental Monitoring Agent Serialized feature representation of As the basic object of causal reasoning, the complex changes of the environment are simulated by the environment change module based on the temporal self-attention mechanism. The change of one of the environment monitoring agents at time x depends on its change in the past y hours and the change trend of other environmental parameters, expressed as: ； in, Represents an environment monitoring agent After simulating complex changes, R represents the learnable parameters. represents the change in the past y hours, represents the changing trend of other environmental parameters, TA() represents the temporal self-attention mechanism; S203, Decision-making and Coordination Agent Use two reasoning modules with the same structure but different weights to simulate the front and back environmental monitoring agents to analyze the results and generate corresponding equipment control actions. The expression is: ； ； in, and It is the reasoning module, which consists of three graph convolutional layers and two linear layers. represents the environment monitoring agent before simulation, represents the simulated environment monitoring agent, Representing decision-making and coordination agents Generate device control actions for the pre-simulation environment. Representing decision-making and coordination agents Generate device control actions for the simulated environment.

3. The intelligent control method based on knowledge distillation and multi-agent reinforcement learning according to claim 2 is characterized in that: S2 also includes the steps of: S204, by reconstructing decision-making and coordination agents The loss function improves the stability of the agent under complex environmental changes. The expression is: ； Where M and N represent the number of decision-making and coordination agents and environment monitoring agents, respectively, and e represents the natural base number. Represents the vector squared norm operation.

4. The intelligent control method based on knowledge distillation and multi-agent reinforcement learning according to claim 2 is characterized in that: S3 includes the following steps: S301, define the joint observation space U of the environment monitoring agent, expressed as: ； S302, based on the joint observation space U, the decision-making and coordination agents explore different joint observation states and propose new learning goals , improve the inference ability and collaborative control of the overall state of the system, the expression is: ； in, represents the information entropy of the joint observation space U, represents the mutual correlation information between decision-making and coordination agents in the joint observation space, and M represents the number of decision-making and coordination agents.

5. The intelligent control method based on knowledge distillation and multi-agent reinforcement learning according to claim 4 is characterized in that: S3 also includes the steps: S303, by setting the reward function in the reinforcement learning algorithm , adjust and optimize the behavior of decision-making and coordination agents, the expression is: ； in, Indicates The prediction of the control state by a decision-making and coordination agent, represents the environment variables currently observed locally, represents the joint observation of all environmental variables, Represents the log-probability of the edge device action control prediction for the next time step given the environment state and the joint observation.

6. The intelligent control method based on knowledge distillation and multi-agent reinforcement learning according to claim 4 is characterized in that: S4 includes the steps: S401, decomposes the complex decision-making and coordination agent into several lightweight prototype agents, expressed as: ； in, Indicates different control tasks, represents the action space of decision-making and coordination agents, Represents the mathematical expectation of different control tasks for decision-making and coordination agents; S402, by the function of knowledge distribution difference Measuring the parameter characteristic distribution of edge devices With the prototype agent The consistency of knowledge distribution between them is expressed as: ； in, Represents the knowledge representation mapping on edge devices, represents the normalization function, Represents the mathematical expectation of the parameter feature distribution of edge devices for the prototype agent.

7. The intelligent control method based on knowledge distillation and multi-agent reinforcement learning according to claim 6 is characterized in that: S4 also includes the steps of: S403, introduce knowledge refinement loss function , the local knowledge is refined and the expression is: ； in, Represents the parameter feature distribution of different edge devices The index of represents the prototype agent weight, Represent different weights, Represents the mathematical expectation of the current edge device for other edge devices.

Images