Intelligent building air conditioner adaptive control method based on deep reinforcement learning

Abstract

The invention discloses an intelligent building air conditioner adaptive control method based on deep reinforcement learning, and belongs to the technical field of intelligent building control, and the method specifically comprises the steps: collecting the multivariate information of an intelligent building, and combining a three-dimensional model with the position of an air conditioner to construct a three-dimensional space node network; an air conditioner control action range is defined, a discrete action space is formed, and a multi-target reward function integrated with a region weight is constructed; training a deep reinforcement learning model by adopting a Duelling DQN algorithm in combination with experience playback and a target network mechanism, and deploying the deep reinforcement learning model to a building control system; during operation, state data are obtained through the node network, the optimal action is selected from the action space to control the air conditioner, after the air conditioner executes the action, new state data are collected, a reward value is calculated according to a reward function and fed back to the model, and an optimal action selection strategy is dynamically adjusted. The method can accurately adapt to the complex environment of the building, the comfort level and energy consumption are balanced, and the intelligent level and the energy utilization efficiency of intelligent building air conditioner control are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent building control, and specifically relates to an adaptive control method for intelligent building air conditioners based on deep reinforcement learning. Background Art

[0002] With the acceleration of the urbanization process, the number of intelligent buildings is increasing continuously. As one of the devices with the largest energy consumption in intelligent buildings, the efficient control of air conditioning systems is of great significance for reducing building energy consumption and achieving the goals of energy conservation and emission reduction. Traditional building air conditioning control methods mostly adopt preset control strategies, such as PID control based on temperature set values. However, the building environment is highly complex and uncertain, and factors such as personnel activities and external climate conditions in different seasons, different time periods, and different regions will have a significant impact on air conditioning loads. The preset control strategies cannot adapt to these changes in real time, resulting in serious energy waste and difficulty in ensuring indoor comfort. In recent years, artificial intelligence technology has developed rapidly, providing new ideas for building air conditioning control. Some machine learning-based methods have been tried and applied to air conditioning control, such as predicting air conditioning loads by learning historical data and then adjusting control parameters. However, most of these methods rely on a large amount of labeled data, and the generalization ability of the models is limited. In the face of complex and changing building environments, it is still difficult to achieve efficient adaptive control. As a technology that combines the powerful perception ability of deep learning and the autonomous decision-making ability of reinforcement learning, deep reinforcement learning enables agents to continuously learn optimal strategies in the interaction with the environment, and is expected to bring innovative solutions for intelligent building air conditioning control. However, there are still relatively few studies and practices on effectively applying deep reinforcement learning to intelligent building air conditioning adaptive control, and there are many problems to be solved urgently, such as how to design reasonable state spaces, action spaces, and reward functions to achieve efficient learning and precise control. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention proposes an adaptive control method for intelligent building air conditioners based on deep reinforcement learning, which collects multi-source information of intelligent buildings, combines a three-dimensional model with the air conditioner position to construct a three-dimensional space node network; accordingly defines the range of air conditioning control actions to form a discrete action space, and constructs a multi-objective reward function incorporating regional weights; adopts the Dueling DQN algorithm, combines the experience replay and target network mechanisms to train a deep reinforcement learning model, and deploys it to the building control system; during operation, obtains state data through the node network, selects the optimal action from the action space to control the air conditioner, after the air conditioner executes the action, collects new state data, calculates the reward value according to the reward function and feeds it back to the model to dynamically adjust the optimal action selection strategy; this method can accurately adapt to the complex building environment, balance comfort and energy consumption, and improve the intelligent level and energy utilization efficiency of intelligent building air conditioning control.

[0004] To achieve the above object, the present invention provides the following technical solutions: An intelligent building air conditioner adaptive control method based on deep reinforcement learning, comprising: S1: Collect multi-source information in the intelligent building area, combine the three-dimensional model of the intelligent building and the air conditioner position to construct a three-dimensional space node network; S2: Define the control action range of the air conditioning system according to the three-dimensional space node network to form a discrete action space; S3: Construct a multi-objective reward function, combine the state differences of different regions in the three-dimensional space node network, and set regional weights; S4: Train the pre-constructed deep reinforcement learning model according to the three-dimensional space node network, discrete action space and multi-objective reward function. By introducing an experience replay mechanism and a target network mechanism, obtain a trained deep reinforcement learning model and deploy it to the control system of the intelligent building; S5: Conduct real-time monitoring in the control system, obtain the current state representation data through the three-dimensional space node network, select the optimal action from the discrete action space according to the current state representation data and output it to the air conditioning system; S6: After the air conditioning system executes the optimal action, collect new state representation data through the three-dimensional space node network again, calculate the reward value after this action execution according to the multi-objective reward function and feedback it to the deep reinforcement learning model. The deep reinforcement learning model dynamically adjusts the control strategy according to the feedback result; the control strategy is the optimal action selection process.

[0005] Specifically, the specific steps of S1 include: S1.1: Collect multi-source information in the intelligent building area, import the three-dimensional model of the intelligent building, and obtain the air conditioner position. At the same time, perform preprocessing; S1.2: Divide the three-dimensional space of the intelligent building into N irregular sub-regions according to the building function type, and define each sub-region as a node to form a node set and assign the preprocessed multi-source information to the corresponding nodes to form a node state vector; the building function type includes but is not limited to offices, meeting rooms, corridors, where, represents the i-th sub-region or the i-th node; S1.3: Define the spatial connection relationship between nodes according to the nodes and their position coordinates in the three-dimensional space; the spatial connection relationship includes: If the sub-regions to which two nodes belong share an entity edge in the three-dimensional space, establish an adjacency relationship, and adjust the weight according to the physical attributes of the shared edge to obtain an adjacency matrix A and a weight matrix W; S1.4: Determine each air conditioner device through CFD simulation Range of affected nodes , where k represents the air-conditioning equipment index; If the node center coordinates are located within the range of affected nodes of the air-conditioning equipment , then establish a connection edge , and use the operating parameters of the air conditioner as the edge attributes of the connection edge to form a connection edge set ; ; S1.5: Integrate the node set V, the adjacency matrix A, and the connection edge set , and output a three-dimensional space node network , where represents the union.

[0006] Specifically, the specific steps of S2 include: S2.1: Obtain a three-dimensional space node network, and extract the attributes of each air-conditioning equipment from the connection edge set; the attributes of the air-conditioning equipment include type, energy efficiency ratio, maximum power, and coordinates; S2.2: Associate the air-conditioning equipment with the node status to obtain an air-conditioning equipment attribute table with status association; the associated content includes: For each , bind the current temperature and population density of the node ; S2.3: Based on the basic functions of the air-conditioning system, obtain a basic control action set, and combine the temperature state vectors and environmental state components of each node in the three-dimensional space node network to determine the feasible range of the basic control actions. At the same time, combine the number of people and activity intensity information in each area of the node network and the operating state parameters of the air-conditioning system itself to determine an extended control action set; S2.4: According to the control accuracy of the air-conditioning equipment, discretely divide the basic control action set and the extended control action set, and combine the discrete gears after dividing the basic control actions and the extended control actions to form a discrete action combination candidate set; S2.5: According to the physical characteristics and safety specifications of the air-conditioning system, determine the boundary conditions of the action space and set the constraint conditions between actions; The boundary conditions of the action space include total power limit and compressor protection; The constraint condition between actions is that the temperature difference between two adjacent moments is less than or equal to 1; S2.6: Combine all discrete action combinations that have been discretized and meet the boundary conditions and constraint conditions, and assign a unique ID to each action to form a discrete action space.

[0007] ​Specifically, the specific steps of S4 include: S4.1: Using the node information in the three-dimensional spatial node network G, the node state vectors of each node are weighted and aggregated according to the regional weights obtained in S3 to obtain the global state representation s of the intelligent building; S4.2: Obtain the discrete action space in S2; the discrete action space contains all feasible air-conditioning control action combinations; S4.3: Use the constructed multi-objective reward function to calculate the reward value according to the global state representation s of the intelligent building and the discrete action combinations in the executed discrete action space; S4.4: Load the pre-constructed Dueling DQN model architecture; the network architecture of the Dueling DQN includes a feature extraction layer, an advantage function stream, and a state value function stream; S4.5: Randomly initialize the main network parameters and target network parameters of the pre-constructed Dueling DQN model. At the same time, initialize the experience replay buffer; the experience replay buffer is used to store state-action-reward-next state quadruples; S4.6: Train the pre-constructed Dueling DQN model. When the training reaches the preset number of iterations or meets the convergence condition, the training ends, and the trained Dueling DQN model is deployed to the control system of the intelligent building.

[0008] Specifically, when calculating the reward value in S4.3, it is necessary to calculate the reward values of the temperature deviation, energy consumption function, and air quality function in the multi-objective reward function step by step, and implement it through weighted summation. The formulas for calculating the reward values of the temperature deviation, energy consumption function, and air quality function in the multi-objective reward function are: ; where a represents the executed action, represents the temperature deviation penalty function, and ; represents the node temperature, 0]represents the set temperature, represents the temperature change, represents the basic power consumption of the air conditioner, represents the power consumption of the kth air conditioner under the action a, and satisfies: ; represents the control action for the kth air conditioner, represents the current temperature of the area where the kth air conditioner is located, which is the node temperature extracted from the global state representation s, represents the outdoor temperature, scale represents the power scaling coefficient, represents the time-of-use electricity price factor, and t represents the current moment. represents the node of the CO2 concentration. represents the CO2 concentration. represents the CO2 deviation penalty function and satisfies: , represents the change in CO2 concentration. represents the maximum value function.

[0009] Specifically, the specific process of training the pre-constructed Dueling DQN model in S4.6 includes: S4.61: Initialize the environmental state of the intelligent building , and select an action in the discrete action space according to the current global state representation ; S4.62: The air conditioning system executes the action , and calculates the final reward value according to the multi-objective reward function. At the same time, a new state is obtained through the three-dimensional space node network , and the experience sample is stored in the experience replay buffer; S4.63: If the buffer is full, discard the old samples according to the FIFO principle; S4.64: Randomly sample M experience samples from the experience replay buffer and calculate the target value using the target network; S4.65: Calculate the mean square error loss value between the output of the current network and the target value; S4.66: According to the mean square error loss value, use the optimizer to perform one step of gradient descent and update the current network parameters; S4.67: Copy the current network parameters to the target network parameters every C steps and set the next state as the current state.

[0010] Specifically, the specific steps of S4.61 include: Determine the intelligent building environmental state factors and set initial values for each intelligent building environmental state factor; Set a small probability u and satisfy ; Generate a random number g and satisfy ; If , then randomly select an action from the discrete action space; If , then according to the current state representation, calculate the reward value of each action using the pre-constructed Dueling DQN model and select the action with the highest reward value.

[0011] Specifically, the specific steps of S4.62 include: Decode the action index output by the Dueling DQN model into an air conditioner control instruction and send it to the building automation system through the Modbus RTU protocol: The building automation system receives and executes the air conditioner control instruction. Meanwhile, it collects the actual parameters after the air conditioner is executed; Obtain new state data through the nodes of the three-dimensional space node network, and combine the actual parameters after the air conditioner is executed to dynamically adjust the sub-region weights; Generate a new global state using the weighted average method based on the new state data and the adjusted sub-region weights; Calculate the final reward value using multi-objective rewards based on the new state data, the actual parameters after the air conditioner is executed, and the decoded air conditioner control instruction; Form a quadruple with the new state data, the action index, the final reward value, and the new global state to obtain a transition sample, and store the transition sample in the experience replay buffer.

[0012] Specifically, the specific steps of S5 include: S5.1: Synchronously collect the real-time data of each node from the sensor network of the three-dimensional space node network and perform preprocessing to generate a set of real-time node state vectors; S5.2: Aggregate the real-time node state vectors weighted by the regional weights to generate a real-time global state representation; S5.3: Convert the real-time global state representation into a tensor format. Meanwhile, add a batch dimension to form an input format that can be processed by the Dueling DQN model; the tensor dimension is the same as the input layer dimension of the Dueling DQN model; S5.4: Calculate the Q values of all possible actions through the trained Dueling DQN model and select the action index with the largest Q value; S5.5: Map the action index with the largest Q value to actual control parameters; the actual control parameters include the temperature set value and the wind speed gear; the mapping execution process mainly refers to a predefined decoding table: , where represents the action index with the largest Q value, FS represents the wind speed gear, represents the Dueling DQN model, represents the decoding function, and ControlParams represents the actual control parameters; S5.6: According to the air conditioner system communication protocol, convert the actual control parameters into an instruction frame through CF = ProtocolEncode(ControlParams), where CF represents the instruction frame, Indicates the protocol encoding function; S5.7: Send the instruction frame to the corresponding air conditioner device through the network interface and record the execution log.

[0013] Specifically, the multi-source information includes the temperature state vector of the intelligent building area, the inter-area temperature gradient information, the environmental state components, the number of people and activity intensity information in each area, and the operating state parameters of the air conditioning system itself; the control actions include the adjustment of the cooling or heating mode, the multi-gear setting of the fan speed, and the adjustment of the fresh air valve opening.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The present invention proposes an intelligent building air conditioner adaptive control method based on deep reinforcement learning. Through multi-step collaborative design, it realizes efficient and intelligent air conditioner management, with significant beneficial effects. First, by constructing a three-dimensional space node network, a discrete action space, and a multi-objective reward function, combined with deep reinforcement learning model training, a complete intelligent control system is established. This system can comprehensively integrate the multi-source information of the intelligent building, covering environmental parameters, personnel activities, air conditioner device attributes, etc., accurately define the range of air conditioner control actions, form a scientific reward mechanism, and enable the deep reinforcement learning model to learn the optimal control strategy that takes into account both energy conservation and comfort.

[0015] The present invention proposes an intelligent building air conditioner adaptive control method based on deep reinforcement learning. In actual operation, the system continuously optimizes the control strategy through real-time monitoring and dynamic feedback mechanisms. The control system quickly selects the optimal action to control the operation of the air conditioner according to the real-time state data obtained from the three-dimensional space node network. After the action is executed, it can timely collect new data to calculate the reward value and feedback it to the model to further optimize the parameters. This closed-loop control mode not only improves the response speed and control accuracy of the air conditioning system but also can adapt to the dynamic changes of the building environment, effectively reducing energy consumption while ensuring the indoor environmental comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the intelligent building air conditioner adaptive control method based on deep reinforcement learning of the present invention; Figure 2 Flow chart of the principle of the intelligent building air conditioner adaptive control method based on deep reinforcement learning of the present invention; Figure 3 Flow chart of constructing the three-dimensional space node network of the intelligent building air conditioner adaptive control method based on deep reinforcement learning of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] Example 1 Please refer to Figure 1 and Figure 2, an embodiment provided by the present invention: an intelligent building air conditioner adaptive control method based on deep reinforcement learning, comprising the following steps: S1: Collect multi-source information of the intelligent building area, and combine the three-dimensional model of the intelligent building and the air conditioner location to construct a three-dimensional space node network; The multi-source information includes the temperature state vector of the intelligent building area, the inter-region temperature gradient information, the environmental state components, the number of people and activity intensity information in each region, and the operating state parameters of the air conditioning system itself.

[0018] It should be noted that the environments and requirements of different regions in the intelligent building vary greatly, and it is difficult to achieve precise control relying solely on single data. By constructing a node network by combining multi-source information, three-dimensional models, and air conditioner locations, the spatial relationship and state information between each region and the air conditioner can be clearly reflected.

[0019] For example, through the node network, it can be known which regions are closer to a certain air conditioner and which regions are more affected by a certain air conditioner, so as to perform air conditioner control more targeted.

[0020] S2: Define the control action range of the air conditioning system according to the three-dimensional space node network to form a discrete action space; The control actions include the adjustment of the cooling or heating mode, the multi-gear setting of the fan speed, and the adjustment of the fresh air valve opening.

[0021] It should be noted that the discrete action space provides a set of optional control actions for the model. During the decision-making process, the model can select appropriate actions from this set to adjust the operating state of the air conditioner to meet the needs of different regions. For example, the action space can include different cooling or heating modes, fan speed gears, and fresh air valve openings.

[0022] S3: Construct a multi-objective reward function, and combine the state differences of different regions in the three-dimensional space node network to set regional weights; It should be noted that the reward function is the guidance for the deep reinforcement learning model to learn. The goal of the model is to maximize the reward value. By reasonably setting the reward function and regional weights, the model can learn to take different control actions in different regions to balance multiple objectives such as comfort, energy consumption, and air quality. For example, for areas with a high density of people, a higher weight can be set to ensure the comfort of that area.

[0023] Further, the specific steps of S3 include: (1) Obtain indoor temperature data from each node in the three-dimensional space node network, determine the set comfortable temperature range. At the same time, collect the real-time energy consumption information of the air conditioning system and gather indoor air quality data. The setting of the comfortable temperature range is usually a temperature interval determined according to human comfort research and actual usage requirements. The real-time energy consumption information of the air conditioning system is obtained through an energy consumption monitoring device installed on the air conditioning equipment or power supply line. The indoor air quality data is obtained through an air quality sensor. (2) For each node, calculate the deviation between its indoor temperature and the set comfortable temperature range through difference calculation. (3) Construct an energy consumption function based on the real-time energy consumption information of the air conditioning system. The energy consumption function adopts a linear function and is obtained by multiplying the real-time energy consumption information of the air conditioning system by a positive constant. (4) Construct an air quality function based on the indoor air quality index. Among them, the construction of the air quality function needs to be determined according to the relationship between the air quality index and comfort. For example: when the air quality index b is within the ideal range inside, , when the air quality index b satisfies the condition: or when, , where represents a positive constant used to adjust the weight of air quality in the reward function, represents the air quality function, and represent the minimum and maximum values of the ideal value of the air quality index respectively. (5) Combine the temperature deviation, energy consumption function, and air quality function through weighted summation to construct a multi-objective reward function. (6) Analyze the state differences in different regions of the three-dimensional space node network, which may include differences in personnel density, usage function, temperature change trend, etc. in different regions. For example, areas with high personnel density may have higher requirements for temperature and air quality, or the temperature in certain regions is more likely to fluctuate greatly due to the influence of the external environment. (7) According to the regional state differences, set corresponding weights for each region. Among them, the weight selection method is: by establishing a comprehensive evaluation model, quantifying and weighted summing factors such as personnel density, usage function, and temperature change trend to obtain the comprehensive score of each region, and then performing normalization processing to obtain the regional weight.

[0024] S4: According to the three-dimensional space node network, discrete action space, and multi-objective reward function, train the pre-constructed deep reinforcement learning model. By introducing an experience replay mechanism and a target network mechanism, obtain the trained deep reinforcement learning model and deploy it to the control system of the intelligent building. It can select the optimal action from the discrete action space according to the state information provided by the three-dimensional space node network to maximize the multi-objective reward function. The experience replay mechanism and the target network mechanism can improve the learning efficiency and stability of the model, enabling the model to better learn the optimal control strategy.

[0025] S5: Conduct real-time monitoring in the control system, obtain the current state representation data through the three-dimensional space node network, select the optimal action from the discrete action space according to the current state representation data, and output it to the air conditioning system; S6: After the air conditioning system executes the optimal action, collect new state representation data through the three-dimensional space node network again, calculate the reward value after this action execution according to the multi-objective reward function, and feedback it to the deep reinforcement learning model. The deep reinforcement learning model dynamically adjusts the control strategy according to the feedback result; the control strategy is the optimal action selection process.

[0026] Furthermore, the specific steps of S6 include: (1) The sensors in the three-dimensional space node network collect the environmental data of each node at a certain sampling frequency and accuracy. After these data are preprocessed, such as noise removal, data normalization, etc., they are combined into a new state representation vector; (2) Input the new state representation vector into the multi-objective reward function. The function calculates a comprehensive reward value according to the weights of different objectives and the differences between the state variables and the target values. For example, if the goal is to keep the indoor temperature within a comfortable range while reducing energy consumption, then the reward function will calculate the reward according to the proximity of the actual temperature to the comfortable temperature and the change in energy consumption caused by this action; (3) The reward value is transmitted as a feedback signal to the Dueling DQN model. The Dueling DQN model stores the current state, action, reward, and the next state as an experience sample in the experience replay buffer. Then, the Dueling DQN model randomly extracts a batch of experience samples from the experience replay buffer for learning, and updates the model parameters by optimizing the objective function, such as minimizing the loss function, so as to adjust the control strategy.

[0027] Embodiment 2 Please refer to Figure 3 In this embodiment, the specific steps of S1 include: S1.1: Collect the multivariate information of the intelligent building area, import the three-dimensional model of the intelligent building, and obtain the air conditioning location. At the same time, perform preprocessing; S1.2: Divide the three-dimensional space of the intelligent building into N irregular sub-regions according to the building function types, and define each sub-region as a node to form a node set , and assign the preprocessed multi-source information to the corresponding nodes to form node state vectors; the building function types include, but are not limited to, offices, meeting rooms, corridors, where, represents the i-th sub-region or the i-th node; Furthermore, the process of building function sub-region division and node definition includes: Based on the design drawings and actual usage functions of the intelligent building, identify different functional areas, such as offices, meeting rooms, corridors, elevator lobbies, machine rooms; Further divide each functional area into irregular sub-regions, considering the spatial continuity, functional consistency, and coverage of the air-conditioning system of the region during division. For example, a large office can be divided into multiple sub-regions according to factors such as internal partitions and personnel distribution density; Define each irregular sub-region as a node, and each node has a unique identifier and spatial location information.

[0028] S1.3: Define the spatial connection relationship between nodes according to the nodes and their position coordinates in the three-dimensional space; the spatial connection relationship includes: If the sub-regions to which two nodes belong share an entity edge in the three-dimensional space, establish an adjacency relationship , and adjust the weight according to the physical properties of the shared entity edge to obtain the adjacency matrix A and the weight matrix W, where, represents the adjacency relationship between the i-th node and the j-th node, 1 means connected, if , it means that the i-th node and the j-th node are not adjacent; Among them, the weight of the shared entity edge between the i-th node and the j-th node satisfies: , represents the attenuation coefficient, and , represents the thermal resistance of the partition wall, and is obtained through the three-dimensional model library of the intelligent building.

[0029] For example, in the three-dimensional space, if two nodes have a common boundary, such as the common wall between adjacent offices, establish a connection between them to construct a preliminary spatial topological structure between nodes.

[0030] S1.4: Determine the range of nodes affected by each air-conditioning device through CFD simulation, where k represents the air-conditioning device index; If the center coordinate of the node is located within the range of nodes affected by the air-conditioning device , establish a connection edge , and use the operating parameters of the air conditioner as the edge attributes of the connecting edges to form a set of connecting edges ; It should be noted that the full name of CFD (Computational Fluid Dynamics) simulation is computational fluid dynamics simulation. It mainly conducts experiments virtually on a computer to simulate the actual fluid flow situation. The basic principle is to numerically solve the differential equations that control fluid flow, obtain the discrete distribution of the fluid flow field in a continuous region, and thus approximately simulate the fluid flow situation. Moreover, the specific simulation process of CFD is the existing technical content in this field and is not the creative solution of this application, so it will not be elaborated here.

[0031] S1.5: Integrate the node set V, the adjacency matrix A, and the set of connecting edges , and output a three-dimensional space node network , where represents the union.

[0032] It should be emphasized that in the present invention, G represents the constructed three-dimensional space node network, which is a weighted directed graph or an undirected graph. V represents the node set, corresponding to the sub-regions divided by the building, such as offices, meeting rooms, corridors, etc. A represents the set composed of the physical space adjacency relationships between nodes, such as the node adjacency relationships of shared edges, represents the set of connecting edges between the air conditioning equipment and the affected nodes, W represents the strength of the connection between nodes, represents the set of edges after merging the set of connecting edges and the physical space adjacency relationships between nodes. After the two are merged, it includes all types of edges in the graph, where the graph refers to the three-dimensional space node network.

[0033] The specific steps of S2 include: S2.1: Obtain the three-dimensional space node network, and extract the attributes of each air conditioning equipment from the set of connecting edges based on the three-dimensional space node network; the attributes of the air conditioning equipment include type, energy efficiency ratio, maximum power, and coordinates; The attributes of the air conditioning equipment Furthermore, the specific steps of B1 include: (1) Extract the set of connecting edges from the already constructed three-dimensional space node network, and each edge is associated with an air conditioning equipment; (2) Parse each connecting edge, extract the attributes of the corresponding air conditioning equipment, and obtain the air conditioning equipment attribute table; S2.2: Associate the air conditioning equipment with the node status to obtain an air conditioning equipment attribute table with status association; the association content includes: For each , bind the current temperature and personnel density of the node ; Furthermore, the specific steps of B2 include: (1) Traverse each air conditioning device, and based on its coordinate position, associate it with the set of nodes affected in the three-dimensional space node network; (2) For each associated node, extract the state parameters and bind them to the corresponding air conditioning device record to obtain an air conditioning device attribute table with state associations; the state parameters include, but are not limited to, the current temperature and the personnel density.

[0034] S2.3: Based on the basic functions of the air conditioning system, obtain the basic control action set. Combine the temperature state vectors and environmental state components of each node in the three-dimensional space node network to determine the feasible range of the basic control actions. At the same time, combine the personnel quantity and activity intensity information of each area in the node network and the operating state parameters of the air conditioning system itself to determine the extended control action set; Furthermore, the specific steps of B3 include: (1) Study the basic functions of the air conditioning system, including core function modules such as refrigeration, heating, ventilation and air change, air speed regulation, and temperature setting, and clarify the control methods corresponding to each function; (2) Define basic control actions for the basic functions of the air conditioning system. For example, the refrigeration function corresponds to turning on the refrigeration mode, turning off the refrigeration mode, raising the refrigeration temperature, and lowering the refrigeration temperature; the air speed regulation function corresponds to actions such as increasing the fan speed and decreasing the fan speed to form a basic control action set; (3) Extract the temperature state vectors of each node from the three-dimensional space node network, compare with the set comfortable temperature range, and calculate the deviation between the current temperature and the comfortable temperature; If the temperature of a certain node is higher than the upper limit of the comfortable temperature and the deviation is greater than the preset error value, it can be determined that the feasible range of the action of lowering the refrigeration temperature is to lower the temperature; If the temperature is close to the comfortable temperature, the feasible range of this action is smaller; (4) Combine environmental state components such as humidity and air pressure. When the humidity is high, the feasible range of the dehumidification action can be increased in the refrigeration mode. When the air pressure is low, the intensity and frequency range of the ventilation and air change actions can be adjusted; (5) Based on the analysis results of the temperature state vector and environmental state components, determine the specific feasible parameter ranges for each action in the basic control action set; (6) According to the personnel quantity and activity intensity information of each area in the node network; If the number of people is large and the activity intensity is high, consider adding extended control actions such as strengthening ventilation and air change and increasing the fan speed; If the number of people is small, energy-saving actions such as reducing the air conditioning power and reducing the fresh air volume can be added; (7) Analyze the operating state parameters of the air conditioning system itself, such as the compressor operating duration and energy consumption data; If the compressor runs continuously for too long, extended control actions such as pausing the compressor operation and switching to a standby air conditioner can be added; When the energy consumption exceeds the set threshold, energy-saving actions such as reducing the cooling / heating intensity of the air conditioner are added; (8) Summarize the actions obtained from the analysis of the number of people and activity intensity information and the operating state parameters of the air conditioning system itself to form an extended control action set.

[0035] S2.4: According to the control accuracy of the air conditioning equipment, discretely divide the basic control action set and the extended control action set, and combine the discrete gears after dividing the basic control actions and the extended control actions to form a discrete action combination candidate set; Furthermore, the specific steps of B4 include: (1) Extract all continuous control parameters from the basic control action set and the extended control action set, such as temperature set value, fan speed, valve opening, and classify the parameters by type, such as temperature type, wind speed type, power type; (2) According to the temperature control accuracy of the air conditioning equipment, usually 0.5°C or 1°C, divide the temperature adjustment range into discrete gears. For example, the cooling temperature range of 16 - 30°C can be divided into 15 gears (16.0, 16.5,..., 30.0); (3) Discretize the fan speed by percentage or fixed gears, such as low speed, medium speed, high speed. For example, the fan speed range of 0% - 100% can be divided into 5 gears, such as 20%, 40%, 60%, 80%, 100%; (4) Map the discretized parameter gears back to the corresponding control actions. For example, the action of lowering the cooling temperature corresponds to the discrete gear of the temperature parameter; (5) Establish an action-parameter mapping table to clarify the range of parameter gears that each action can use; (6) Arrange and combine the basic control actions and the extended control actions to generate action sequences. For example, a combination of adjusting the temperature and the wind speed simultaneously; (7) For each action combination, determine its parameter configuration. For example, the cooling mode + temperature 24°C + wind speed 60% constitutes a complete action combination; (8) Based on physical constraints and control logic, filter out invalid action combinations. For example, a combination of turning on the cooling and heating modes simultaneously is invalid; (9) Apply the equipment capacity constraint to exclude combinations that exceed the equipment performance range. For example, a combination with a power exceeding the rated power of the equipment, and finally obtain a discrete action combination candidate set.

[0036] S2.5: Determine the boundary conditions of the action space according to the physical characteristics and safety specifications of the air conditioning system, and set the constraint conditions between actions; The boundary conditions of the action space include total power limit and compressor protection; The constraint condition between actions is that the temperature difference between two adjacent moments is less than or equal to 1; S2.6: Combine all the discretized actions that meet the boundary conditions and constraint conditions, and assign a unique ID to each action to form a discrete action space.

[0037] Furthermore, the specific steps of B6 include: (1) Screen from all the combined discretized actions obtained through discretization according to the boundary conditions and constraint conditions; (2) For each combined discretized action, check whether its various parameters are within the range specified by the boundary conditions and whether they meet the constraint conditions. If not, exclude this combined action; (3) Assign a unique ID to each combined discretized action retained after screening. In the present invention, the sequential numbering method is adopted, starting from 1 and increasing sequentially to ensure that each combined action has a unique identifier; (4) Establish a mapping relationship between the combined action and the ID for quickly finding the corresponding combined action according to the ID later; (5) Integrate the combined discretized actions with unique IDs into a data structure to form a discrete action space, and this data structure can be a list, a dictionary or a database table.

[0038] The specific steps of S4 include: S4.1: Utilize the node information in the three-dimensional space node network G to weighted aggregate the node state vectors of each node according to the regional weights obtained in S3 to obtain the global state representation s of the intelligent building; Furthermore, the specific steps of S4.1 include: (1) Obtain the three-dimensional space node network G and the regional weights obtained in S3; (2) Establish a mapping function , and map each node to the corresponding sub-region type index, where m represents the number of sub-region types, f represents the mapping function, represents the mapping relationship; (3) Obtain the global state representation of the intelligent building by calculating the ratio of the first type of numerator variable and the second type of denominator variable; the first type of numerator variable is the weight of the sub-region type to which the node belongs and the node The cumulative sum result of the node state vectors; the second type of denominator variable is the node The cumulative sum result of the weights of the sub-region types to which it belongs.

[0039] S4.2: Obtain the discrete action space in S2; the discrete action space contains all feasible air-conditioning control action combinations; S4.3: Use the constructed multi-objective reward function to calculate the reward value according to the global state representation s of the intelligent building and the discrete action combinations in the executed discrete action space; Furthermore, the specific steps of S4.3 include: (1) Obtain the constructed multi-objective reward function; the multi-objective reward function includes temperature deviation, energy consumption function, and air quality function; (2) Calculate the temperature comfort reward , energy consumption efficiency reward and air quality reward respectively according to the multi-objective reward function, and satisfy the formula: ; where a represents the executed action, represents the temperature deviation penalty function, and , represents the temperature of the node , represents the set temperature, represents the temperature change, represents the basic power consumption of the air conditioner, which refers to the minimum energy consumption of the air conditioner in the startup state and is also the power consumed even when not cooling, represents the power consumption of the k-th air conditioner under the action a, and satisfies: , represents the control action for the k-th air conditioner, represents the current temperature of the area where the k-th air conditioner is located, which is the node temperature extracted from the global state representation s, represents the outdoor temperature, scale represents the power scaling factor,[[]] represents the time-of-use electricity price factor. For example, during peak hours , during off-peak hours , t represents the current time, represents the node 's CO2 concentration, represents the CO2 concentration, represents the CO2 deviation penalty function, and satisfies: , represents the CO2 concentration change, represents the maximum value function; (3) The temperature comfort reward and the energy consumption efficiency reward and the air quality reward are integrated to obtain the reward value .

[0040] S4.4: Load the pre - constructed Dueling DQN model architecture; the network architecture of the Dueling DQN includes a feature extraction layer, an advantage function stream, and a state value function stream; S4.5: Randomly initialize the main network parameters and target network parameters of the pre - constructed Dueling DQN model. Meanwhile, initialize the experience replay buffer; the experience replay buffer is used to store state - action - reward - next - state quadruples; Furthermore, the random initialization of the main network parameters adopts the following method: (1) Weight parameters: Initialize using a Gaussian distribution or a uniform distribution; (2) Bias parameters: Initialize to 0 or a small constant.

[0041] Among them, the target network parameters are initialized synchronously with the main network parameters, satisfying: target network parameters main network parameters.

[0042] S4.6: Train the pre - constructed Dueling DQN model. When the training reaches the preset number of iterations or meets the convergence condition, the training ends, and the trained Dueling DQN model is deployed to the control system of the intelligent building.

[0043] The specific process of training the pre - constructed Dueling DQN model in S4.6 includes: S4.61: Initialize the environmental state of the intelligent building , and select an action from the discrete action space according to the current global state representation ; Exemplarily, an example code for selecting an action from the discrete action space using - greedy strategy is: if rand()<ε then a1 ← random action else a1 ← argmax_a Q(s1,a;θ) # the optimal action predicted by the current network Q end if S4.62: The air - conditioning system executes the action , and calculates the final reward value according to the multi - objective reward function. Meanwhile, obtain the new state through the three - dimensional space node network , and store the experience samples into the experience replay buffer; S4.63: If the buffer is full, discard the old samples according to the FIFO principle; Furthermore, the process of discarding old samples according to the FIFO principle includes: (1) Initialize the experience replay buffer and set the maximum capacity capacity of the experience replay buffer; (2) When a new experience sample is obtained, check whether the buffer length is equal to capacity; If the buffer length is equal to the maximum capacity capacity of the experience replay buffer, remove the first element in the list, that is, the earliest sample, and then add the new experience sample; If the buffer length is less than the maximum capacity capacity of the experience replay buffer, directly add the new experience sample.

[0044] Exemplarily, the implementation process includes: if len(replay_buffer) == capacity: del replay_buffer[0] replay_buffer.append(new_experience) S4.64: Randomly sample M experience samples from the experience replay buffer and calculate the target value using the target network; Exemplarily, the example code for calculating the target value is: if s' is the terminal state then y ← r else y ← r + γ * max_a'Q'(s',a';θ') # γ is the discount factor end if S4.65: Calculate the mean squared error loss value between the output of the current network and the target value. Among them, the mean squared error calculation formula is the prior art content in the field and is not the creative solution of this application, so it will not be elaborated here; S4.66: According to the mean squared error loss value, use the optimizer to perform one step of gradient descent to update the current network parameters. Among them, the gradient descent calculation formula is the prior art content in the field and is not the creative solution of this application, so it will not be elaborated here; S4.67: Copy the current network parameters to the target network parameters every C steps, and set the next state as the current state. In the present invention, C is set to 1000.

[0045] The specific steps of S4.61 include: Determine the intelligent building environment state factors and set initial values for each intelligent building environment state factor; Set a small probability u, and satisfy ; Generate a random number g, and satisfy ; If , randomly select an action from the discrete action space; If , calculate the reward value of each action according to the current state representation using the pre - constructed Dueling DQN model, and select the action with the highest reward value.

[0046] The specific steps of S4.62 include: A1: Decode the action index output by the Dueling DQN model into an air - conditioner control instruction and send it to the building automation system through the ModbusRTU protocol: Furthermore, the specific steps of A1 include: (1) Pre - define a mapping table between the action index and the air - conditioner control parameters, for example: action_mapping = { 0: {'temperature': 24, 'fan_speed':'medium','mode': 'cooling'}, 1: {'temperature': 25, 'fan_speed':'medium','mode': 'cooling'}, } (2) Convert the action index output by the Dueling DQN model into a control parameter dictionary. The conversion method is: control_params = action_mapping[B], where B represents the action index output by the Dueling DQN model, control_params represents the control parameters in the control parameter dictionary, action_mapping represents the mapping relationship, which is the control parameter dictionary in the present invention; (3) Verify whether the control parameters are within the allowable range of the device, for example; if not (16 <= control_params['temperature'] <= 30): # Correct the temperature value or trigger an exception control_params['temperature'] = min(max(control_params['temperature'], 16), 30) (4)Establish the correspondence between control parameters and Modbus register addresses; (5)Convert the control parameters into Modbus register values and construct a Modbus RTU frame; (6)Set the baud rate, data bits, stop bits, and parity bits, and send the Modbus RTU frame to complete the sending of the air conditioner control instruction.

[0047] A2: The building automation system receives and executes the air conditioner control instruction. Meanwhile, it collects the actual parameters after the air conditioner execution; A3: Obtain the new state data through the nodes of the three-dimensional space node network, and combine it with the actual parameters after the air conditioner execution to dynamically adjust the sub-region weights; A4: Based on the new state data and the adjusted sub-region weights, generate a new global state using the weighted average method. Among them, the weighted average method is the prior art content in this field and is not the creative solution of this application, so it will not be elaborated here; A5: Based on the new state data, the actual parameters after the air conditioner execution, and the decoded air conditioner control instruction, calculate the final reward value using a multi-objective reward function; A6: Combine the new state data, action index, final reward value, and new global state into a quadruple to obtain a transition sample, and store the transition sample in the experience replay buffer.

[0048] The specific steps of S5 include: S5.1: Synchronously collect the real-time data of each node from the sensor network of the three-dimensional space node network and perform preprocessing to generate a set of real-time node state vectors; S5.2: Weightedly aggregate the real-time node state vectors according to the regional weights to generate a real-time global state representation; S5.3: Convert the real-time global state representation into a tensor format. Meanwhile, add a batch dimension to form an input format that can be processed by the DuelingDQN model; the tensor dimension is the same as the input layer dimension of the Dueling DQN model; It should be noted that the batch dimension is a data dimension used in deep learning to represent processing multiple samples at once. In deep learning, usually more than one data sample is not processed at a time, but multiple samples are combined into a batch for processing, and the batch dimension is the dimension used to represent the number of samples in this batch. By combining multiple samples into a batch for calculation, the parallel computing power of the hardware can be fully utilized, reducing the calculation time and the number of memory accesses, thereby improving the efficiency of training and inference. At the same time, compared with calculating gradients for a single sample, batch gradients can better represent the overall distribution of the data, reducing the noise and fluctuations of the gradients.

[0049] S5.4: Calculate the Q-values of all possible actions through the trained Dueling DQN model, and select the action index with the largest Q-value; S5.5: Map the action index with the largest Q-value to actual control parameters; the actual control parameters include the temperature set value and the wind speed gear; the mapping execution process mainly refers to a predefined decoding table: , where, represents the action index with the largest Q-value, FS represents the wind speed gear, represents the Dueling DQN model, represents the decoding function, and ControlParams represents the actual control parameters; S5.6: According to the air-conditioning system communication protocol, convert the actual control parameters into an instruction frame through CF = ProtocolEncode(ControlParams), where CF represents the instruction frame, represents the protocol encoding function; S5.7: Send the instruction frame to the corresponding air-conditioning device through the network interface and record the execution log.

[0050] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make changes, modifications, substitutions, and variations to the above embodiments without departing from the purpose and scope of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. An intelligent building air-conditioning adaptive control method based on deep reinforcement learning, characterized in that Including: S1: Collect multi-source information of the intelligent building area, combine the 3D model of the intelligent building and the air conditioner location to construct a 3D space node network; S2: Define the control action range of the air conditioning system according to the 3D space node network to form a discrete action space; S3: Construct a multi-objective reward function, combine the state differences of different regions in the 3D space node network, and set regional weights; S4: Train the pre-constructed deep reinforcement learning model according to the 3D space node network, discrete action space and multi-objective reward function. By introducing an experience replay mechanism and a target network mechanism, obtain the trained deep reinforcement learning model and deploy it to the control system of the intelligent building; S5: Conduct real-time monitoring in the control system, obtain the current state representation data through the 3D space node network, select the optimal action from the discrete action space according to the current state representation data and output it to the air conditioning system; S6: After the air conditioning system executes the optimal action, collect new state representation data through the 3D space node network again, calculate the reward value after this action execution according to the multi-objective reward function and feedback it to the deep reinforcement learning model. The deep reinforcement learning model dynamically adjusts the control strategy according to the feedback result; The control strategy is the optimal action selection process.

2. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 1, wherein, The specific steps of S1 include: S1.1: Collect multi-source information of the intelligent building area, import the 3D model of the intelligent building, and obtain the air conditioner location. At the same time, perform preprocessing; S1.2: Divide the three-dimensional space of the intelligent building into N irregular sub-regions according to the building function types, define each sub-region as a node, and form a node set , and assign the preprocessed multi-source information to the corresponding nodes to form node state vectors; the building function types include but are not limited to offices, meeting rooms, corridors, where represents the i-th sub-region or the i-th node. S1.3: Define the spatial connection relationship between nodes according to the nodes and their position coordinates in the 3D space; The spatial connection relationship includes: If the sub-regions to which two nodes belong share an entity edge in the 3D space, establish an adjacency relationship, and adjust the weight according to the physical attributes of the shared edge to obtain the adjacency matrix A and the weight matrix W; S1.4: Determine the range of nodes affected by each air-conditioning device through CFD simulation where k represents the index of the air-conditioning device ; If the node center coordinates are located within the range of the nodes affected by the air conditioning equipment then a connection edge is established, and the operating parameters of the air conditioner are used as the edge attributes of the connection edge to form a set of connection edges ; ​ S1.5: Integrate the node set V, the adjacency matrix A, and the set of connected edges , and output a three-dimensional spatial node network , where represents the union set.

3. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 2, characterized in that, The specific steps of S2 include: S2.1: Obtain a three-dimensional space node network, and extract the attributes of each air conditioner device from the connection edge set ; the attributes of the air conditioner device include type, energy efficiency ratio, maximum power, and coordinates; The attributes of the air conditioner device S2.2: Associate the air conditioning equipment with the node state to obtain an air conditioning equipment attribute table with state association; The associated content includes: For each , bind the current temperature and personnel density of the node ; S2.3: Based on the basic functions of the air conditioning system, obtain the basic control action set, combine the temperature state vectors and environmental state components of each node in the 3D space node network to determine the feasible range of the basic control actions. At the same time, combine the number of people and activity intensity information in each region of the node network and the operation state parameters of the air conditioning system itself to determine the extended control action set; S2.4: According to the control accuracy of the air conditioning equipment, discretely divide the basic control action set and the extended control action set, and combine the discrete gears after dividing the basic control actions and the extended control actions to form a discrete action combination candidate set; S2.5: Determine the boundary conditions of the action space according to the physical characteristics and safety specifications of the air conditioning system, and set the constraint conditions between actions; The boundary conditions of the action space include total power limit and compressor protection; The constraint condition between actions is that the temperature difference between two adjacent moments is less than or equal to 1; S2.6: Combine all discrete action combinations that have been discretized and meet the boundary conditions and constraint conditions, and assign a unique ID to each action to form a discrete action space.

4. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 3, characterized in that, The specific steps of S4 include: S4.1: Using the node information in the three-dimensional spatial node network G, the node state vectors of each node are weighted and aggregated according to the regional weights obtained in S3 to obtain the global state representation s of the intelligent building; S4.2: Obtain the discrete action space in S2; the discrete action space contains all feasible air-conditioning control action combinations; S4.3: Use the constructed multi-objective reward function to calculate the reward value according to the global state representation s of the intelligent building and the discrete action combinations in the executed discrete action space; S4.4: Load the pre-constructed Dueling DQN model architecture; the network architecture of the Dueling DQN includes a feature extraction layer, an advantage function stream, and a state value function stream; S4.5: Randomly initialize the main network parameters and target network parameters of the pre-constructed Dueling DQN model. At the same time, initialize the experience replay buffer; the experience replay buffer is used to store state-action-reward-next state quadruples; S4.6: Train the pre-constructed Dueling DQN model. When the training reaches the preset number of iterations or meets the convergence condition, the training ends, and the trained Dueling DQN model is deployed to the control system of the intelligent building.

5. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 4, characterized in that, When calculating the reward value in S4.3, it is necessary to calculate the reward values of the temperature deviation, energy consumption function, and air quality function in the multi-objective reward function step by step, and implement it through weighted summation. The formulas for calculating the reward values of the temperature deviation, energy consumption function, and air quality function in the multi-objective reward function are: ； Among them, a represents the executed action, represents the temperature deviation penalty function, and , represents the temperature of node . represents the set temperature, represents the temperature change, represents the basic power consumption of the air conditioner, represents the power consumption of the k-th air conditioner under action a, and satisfies: , represents the control action for the k-th air conditioner, represents the current temperature of the area where the k-th air conditioner is located, which is the node temperature extracted from the global state representation s, represents the outdoor temperature, scale represents the power scaling factor, represents the time-of-use electricity price factor, t represents the current time, represents node 's CO2 concentration, represents the CO2 concentration, represents the CO2 deviation penalty function, and satisfies: , represents the CO2 concentration change, represents the maximum value function.

6. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 5, characterized in that, The specific process of training the pre-constructed Dueling DQN model in S4.6 includes: S4.61: Initialize the environmental state of the intelligent building , select an action from the discrete action space according to the current global state representation ; S4.62: The air conditioning system performs an action , calculates the final reward value according to the multi-objective reward function, and meanwhile, obtains a new state through the three-dimensional space node network , and the experience sample is stored in the experience replay buffer; S4.63: If the buffer is full, discard the old samples according to the FIFO principle; S4.64: Randomly sample M experience samples from the experience replay buffer and calculate the target value using the target network; S4.65: Calculate the mean square error loss value between the output of the current network and the target value; S4.66: According to the mean square error loss value, use the optimizer to perform one step of gradient descent to update the current network parameters; S4.67: Copy the current network parameters to the target network parameters every C steps, and set the next state as the current state.

7. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 6, characterized in that, The specific steps of S4.61 include: Determine the intelligent building environment state factors and set initial values for each intelligent building environment state factor; Set a relatively small probability u, and satisfy ; Generate a random number g that satisfies ; If , then randomly select an action from the discrete action space; If , then according to the current state representation, use the pre-built Dueling DQN model to calculate the reward value for each action and select the action with the highest reward value.

8. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 7, wherein The specific steps of S4.62 include: Decode the action index output by the Dueling DQN model into an air conditioner control command and send it to the building automation system through the Modbus RTU protocol: The building automation system receives and executes the air-conditioning control instructions. At the same time, it collects the actual parameters after the air-conditioning is executed; Through the nodes of the three-dimensional spatial node network, obtain the new state data, and combine the actual parameters after the air-conditioning is executed to dynamically adjust the sub-region weights; Based on the new state data and the adjusted sub-region weights, use the weighted average method to generate a new global state; Based on the new state data, the actual parameters after the air-conditioning is executed, and the decoded air-conditioning control instructions, use the multi-objective reward to calculate the final reward value; Form a quadruple with the new state data, action index, final reward value, and new global state to obtain a transfer sample, and store the transfer sample in the experience replay buffer.

9. The intelligent building air-conditioning adaptive control method based on deep reinforcement learning according to claim 8, wherein, The specific steps of S5 include: S5.1: Synchronously collect the real-time data of each node from the sensor network of the three-dimensional space node network, and perform preprocessing to generate a set of real-time node state vectors; S5.2: Weightedly aggregate the real-time node state vectors according to the regional weights to generate a real-time global state representation; S5.3: Convert the real-time global state representation into a tensor format. At the same time, add a batch dimension to form an input format that can be processed by the Dueling DQN model; the tensor dimension is the same as the input layer dimension of the Dueling DQN model; S5.4: Calculate the Q-values of all possible actions through the trained Dueling DQN model, and select the action index with the largest Q-value; S5.5: Map the action index with the largest Q value to the actual control parameters; the actual control parameters include the temperature set value and the wind speed gear; the mapping execution process mainly refers to a predefined decoding table: , where represents the action index with the largest Q value, FS represents the wind speed gear, represents the Dueling DQN model, represents the decoding function, and ControlParams represents the actual control parameters; S5.6: According to the communication protocol of the air conditioning system, the actual control parameters are converted into an instruction frame through CF = ProtocolEncode(ControlParams), where CF represents the instruction frame, indicating the protocol encoding function; S5.7: Send the instruction frame to the corresponding air-conditioning equipment through the network interface and record the execution log.

10. The intelligent building air conditioner adaptive control method based on deep reinforcement learning according to claim 9, characterized in that, The multivariate information includes the temperature state vector of the intelligent building area, the inter-regional temperature gradient information, the environmental state components, the number of people and activity intensity information in each area, and the operating state parameters of the air-conditioning system itself; the control actions include the adjustment of the cooling or heating mode, the multi-gear setting of the fan speed, and the adjustment of the fresh air valve opening.

Images