Reinforcement learning-based precise air conditioner energy-saving control optimization method

Abstract

The application discloses a precision air conditioner energy-saving control optimization method based on reinforcement learning, and comprises the following steps: collecting environmental parameters such as temperature, humidity, heat load, electric energy consumption and running state of an air conditioner system, generating a vector reflecting the current environmental condition, scoring multiple control actions through an improved Banditron perception machine model, selecting an optimal control action through a Thompson sampling mechanism, executing the selected control action, collecting environmental feedback information after execution, forming a feedback parameter set, dynamically adjusting the parameter value of the exploration behavior through data analysis of continuous control cycles, and finally inputting the current environmental state and the control action into a decision model to generate a parameter set for setting the air conditioner running. The application realizes intelligent optimization and energy-saving regulation and control of the air conditioner control process.

Description

Technical Field

[0001] This invention relates to the field of intelligent control and energy-saving optimization technology, and in particular to a precision air conditioning energy-saving control optimization method based on reinforcement learning. Background Technology

[0002] With the continuous advancement of building intelligence and green energy conservation concepts, precision air conditioning systems are increasingly widely used as key infrastructure in data centers, laboratories, and high-precision environmental control scenarios. The proportion of energy consumption of air conditioning systems continues to rise. How to reduce operating energy consumption while ensuring stable environmental parameters has become a core issue that current intelligent air conditioning control technology urgently needs to address.

[0003] In existing technologies, traditional air conditioning energy-saving control methods are mostly based on static rule settings or simple feedback control mechanisms, which make it difficult to achieve adaptive optimization of environmental perception and control strategies. Furthermore, existing technologies have significant shortcomings in the following aspects:

[0004] 1. Limited perception dimensions: Existing control strategies mainly rely on some environmental parameters, such as room temperature or humidity, and lack the ability to comprehensively perceive multi-dimensional environmental factors such as air conditioning operation status, power consumption, and heat load.

[0005] 2. Rigid control strategies: Traditional methods often use preset rules or fixed thresholds, lacking the ability to dynamically adjust based on feedback from the actual environment.

[0006] 3. Lack of intelligent decision-making mechanism: Most existing methods do not introduce reinforcement learning or probability sampling mechanisms, lack policy optimization based on historical data and uncertainty handling capabilities, and are prone to getting trapped in local optimal control strategies.

[0007] 4. Insufficient feedback utilization: The existing control system does not provide sufficient feedback analysis of the control results after environmental changes, and has not established an effective correlation mechanism between feedback parameters and control actions.

[0008] 5. Single path for generating control parameters: In existing methods, most control parameters are manually set or obtained by simple mapping, lacking a joint modeling method based on deep perception structure.

[0009] Therefore, how to provide a precision air conditioning energy-saving control optimization method based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0010] One objective of this invention is to propose a precision air conditioning energy-saving control optimization method based on reinforcement learning. This invention employs steps such as multi-dimensional environmental parameter acquisition, improved Banditron perceptron model scoring, Thompson sampling mechanism decision-making, feedback-driven parameter adjustment, and linear perceptron network control generation to construct a complete optimization path from state perception to control output. It has the advantages of refined environmental perception, intelligent control strategy, and sustainable energy-saving effect.

[0011] The precision air conditioning energy-saving control optimization method based on reinforcement learning according to embodiments of the present invention includes the following steps:

[0012] S1. Collect ambient temperature parameters, ambient humidity parameters, heat load parameters, power consumption parameters and operating status parameters of the air conditioning equipment, and generate an environmental state vector;

[0013] S2. Input the environmental state vector into the improved Banditron perceptron model, score multiple preset control actions, and output the set of control actions and the corresponding confidence score vector.

[0014] S3. Input the confidence score vector and the set of control actions into the Thompson sampling mechanism, initialize the posterior probability distribution of the control actions, set the exploration temperature parameter, sample the set of control actions, and output the target control action.

[0015] S4. Execute the target control action, and collect the ambient temperature parameters, ambient humidity parameters and power consumption parameters generated after the target control action is executed to form a set of feedback parameters.

[0016] S5. Analyze and process the distribution changes of the feedback parameter set and confidence score vector in the continuous control cycle, and adjust the exploration temperature parameter in the Thompson sampling mechanism.

[0017] S6. Construct a joint input vector consisting of the environmental state vector and the target control action, and input the joint input vector into the embedded linear perception decision network to generate a set of control parameters.

[0018] Optionally, S1 specifically includes:

[0019] S11, The ambient temperature parameters include the air inlet temperature of the air conditioning equipment, the air outlet temperature, the temperature of key indoor areas, and the outdoor reference temperature;

[0020] S12, The environmental humidity parameters include the air inlet humidity, air outlet humidity and indoor relative humidity of the air conditioning equipment;

[0021] S13. The heat load parameters include personnel activity load, IT equipment operation heat load and lighting system heat load, and the heat load is expressed as the heat dissipation power per unit time.

[0022] S14. The power consumption parameters include compressor power consumption, fan power consumption, condenser power consumption, and auxiliary control unit power consumption, in kilowatt-hours.

[0023] S15. The operating status parameters of the air conditioning equipment include compressor start / stop status, fan speed, condenser heat exchange efficiency, air supply valve opening degree and current control mode.

[0024] Optionally, S2 specifically includes:

[0025] S21. Input the environmental state vector into the improved Banditron perceptron model;

[0026] S22. In the improved Banditron perceptron model, a corresponding scoring weight structure is assigned to each of the multiple preset control actions, and the scoring value of each preset control action is calculated based on the linear mapping relationship between the environmental state vector and the scoring weight vector.

[0027] S23. Construct a set of control actions based on the score values ​​of multiple preset control actions, and generate a confidence score vector representing the relative magnitude of the score values. The confidence score vector is used to characterize the response magnitude of each preset control action under the current environmental state vector.

[0028] S24. Output the set of control actions corresponding to all rating values ​​and the confidence rating vector.

[0029] Optionally, the improved Banditron perceptron model specifically includes:

[0030] Construct a scoring weight matrix, where each row corresponds to a scoring weight vector for a preset control action, used to receive environmental state vectors and perform scoring calculations;

[0031] The environmental state vector at the current time step is linearly multiplied by each scoring weight vector in the scoring weight matrix to calculate the score value of each preset control action.

[0032] Sort all scores and select the preset control action with the highest score as the predicted action at the current time step;

[0033] Based on the preset exploration probability, a predicted action is selected, and a random action is selected from other preset control actions whose score value is not the maximum according to the exploration probability as the action to be executed at the current time.

[0034] Compare the current action with the predicted action. If they match, record a feedback value of 1; otherwise, record a feedback value of 0.

[0035] The state confidence factor is calculated based on the proportional relationship between the prediction error range corresponding to the current environmental state vector and the reference disturbance amplitude. The prediction error range is based on the difference between the environmental state vector at the current time step and the prediction score value at the previous time step. The reference disturbance amplitude is set as the reference disturbance amplitude in conjunction with a preset environmental state reference fluctuation amplitude.

[0036]

[0037] Where, Δ s tate is the prediction error at the current time step, Δ r ef is the preset reference disturbance amplitude, and β0 and β1 are adjustment coefficients;

[0038] When the feedback value is 1, the score weight vector corresponding to the action executed at the current time step is updated in a positive direction:

[0039]

[0040] When the feedback value is 0, the scoring weight vector is updated negatively:

[0041]

[0042] Where α is the negative feedback attenuation factor, x t This is the environment state vector at the current time step. The action to be executed at the current time step t is 'a'. t The rating weight vector.

[0043] Optionally, S3 specifically includes:

[0044] S31. Assign the confidence score vector to the set of control actions, and construct a posterior probability distribution for each control action in the set of control actions. The posterior probability distribution is input to the Thompson sampling mechanism with the confidence score value as the prior probability parameter.

[0045] S32. In the Thompson sampling mechanism, an independent sampling is performed based on the posterior probability distribution corresponding to each control action in the set of control actions to generate a set of sampled values.

[0046] S33. Determine the control action with the largest sample value in the sample value set, and take the corresponding control action as the candidate target control action under the current time step.

[0047] S34. Set the initial value of the exploration temperature parameter. The initial value of the exploration temperature parameter is calculated based on the mean and variance of the score distribution of the control action set, and satisfies the following constraint: Initial value of exploration temperature parameter = linear combination of mean and variance.

[0048] S35. Compare the explored temperature parameter at the current time step with the set temperature threshold. If the explored temperature parameter is less than the set temperature threshold, directly output the candidate target control action. If the explored temperature parameter is greater than the set temperature threshold, take the candidate target control action as the center, perform disturbance sampling in the control action set based on the current explored temperature parameter, determine and output the target control action.

[0049] Optionally, S4 specifically includes:

[0050] S41. Apply the target control action to the air conditioning equipment, wherein the target control action includes setting operations for compressor start / stop status, fan speed, air supply valve opening and current control mode;

[0051] S42. After the target control action is completed, collect the ambient temperature parameters, ambient humidity parameters and power consumption parameters within the corresponding control cycle.

[0052] S43. Normalize the air inlet temperature, air outlet temperature, indoor key area temperature and outdoor reference temperature of the air conditioning equipment in the ambient temperature parameters to generate the temperature feedback vector under the current control cycle.

[0053] S44. Normalize the air inlet humidity, air outlet humidity and indoor relative humidity of the air conditioning equipment in the environmental humidity parameters to generate a humidity feedback vector under the current control cycle.

[0054] S45. Weighted summation of compressor power consumption, fan power consumption, condenser power consumption and auxiliary control unit power consumption in the power consumption parameters to generate the power consumption value under the current control cycle.

[0055] S46. Connect the temperature feedback vector, humidity feedback vector, and power consumption value in series to construct a set of feedback parameters.

[0056] Optionally, S5 specifically includes:

[0057] S51. After each control cycle ends, record the set of feedback parameters and the confidence score vector for the current control cycle.

[0058] S52. Construct a feedback parameter matrix based on the set of feedback parameters recorded in multiple consecutive control cycles, wherein each row of the feedback parameter matrix corresponds to a control cycle;

[0059] S53. Calculate the moving average and standard deviation of the data in each dimension of the feedback parameter matrix, and extract them to form the feedback statistical fluctuation vector.

[0060] S54. Based on the confidence score vectors in two adjacent control cycles, calculate the relative rate of change of the score values ​​corresponding to each control action, and construct the confidence change vector.

[0061] S55. Linearly combine the feedback statistical fluctuation vector and the confidence change vector to generate the temperature adjustment factor.

[0062] S56. Based on the initial value of the exploration temperature parameter, and combined with the current temperature adjustment factor, the parameter is dynamically updated. The update method satisfies the following linear relationship: Current exploration temperature parameter = Initial value of exploration temperature parameter × Temperature adjustment factor.

[0063] Optionally, S6 specifically includes:

[0064] S61. The environmental state vector generated in the current control cycle and the target control action are dimensionally standardized and then spliced ​​together to form a joint input vector. The splicing method is a one-dimensional ordered series arrangement with the environmental state vector first and the target control action last.

[0065] S63. Input the joint input vector into the embedded linear sensing decision network, wherein the embedded linear sensing decision network includes, in sequence, an input layer, a hidden linear sensing layer, a normalization processing layer and an output layer.

[0066] S64. In the input layer, the joint input vector is received and the input dimension is uniformly mapped.

[0067] S65. In the hidden linear perceptron layer, a fully connected structure is used to perform a linear transformation on the input to generate intermediate mapping results;

[0068] S66. In the normalization processing layer, the intermediate mapping results are normalized to adjust the data distribution.

[0069] S67. In the output layer, the normalized data is subjected to perception mapping to generate a set of control parameters. The set of control parameters is used to guide the operation settings of the air conditioning equipment in subsequent control cycles.

[0070] The beneficial effects of this invention are:

[0071] (1) This invention introduces an improved Banditron perceptron model to score the multidimensional environmental state and combines it with the Thompson sampling mechanism to realize dynamic decision-making on air conditioning control actions, which significantly improves the system's adaptability to complex air conditioning operating environments.

[0072] (2) By introducing the joint analysis of confidence score vector and feedback parameter set into the control strategy, this invention explores temperature parameters by dynamically adjusting the temperature adjustment factor, and realizes the iterative optimization and adaptive adjustment of the control strategy in multiple control cycles, which effectively enhances the self-learning ability and control accuracy of the air conditioning energy-saving control system in various operating scenarios.

[0073] (3) In the stage of generating control parameters, the present invention adopts an embedded linear perception decision network to jointly model the environmental state and control action and embed perception mapping processing, which improves the accuracy and stability of the control parameter output and has high robustness and engineering feasibility. Attached Figure Description

[0074] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0075] Figure 1 This is a flowchart of the precision air conditioning energy-saving control optimization method based on reinforcement learning proposed in this invention. Detailed Implementation

[0076] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0077] refer to Figure 1 The optimization method for precision air conditioning energy-saving control based on reinforcement learning includes the following steps:

[0078] S1. Collect ambient temperature parameters, ambient humidity parameters, heat load parameters, power consumption parameters and operating status parameters of the air conditioning equipment, and generate an environmental state vector;

[0079] S2. Input the environmental state vector into the improved Banditron perceptron model, score multiple preset control actions, and output the set of control actions and the corresponding confidence score vector.

[0080] S3. Input the confidence score vector and the set of control actions into the Thompson sampling mechanism, initialize the posterior probability distribution of the control actions, set the exploration temperature parameter, sample the set of control actions, and output the target control action.

[0081] S4. Execute the target control action, and collect the ambient temperature parameters, ambient humidity parameters and power consumption parameters generated after the target control action is executed to form a set of feedback parameters.

[0082] S5. Analyze and process the distribution changes of the feedback parameter set and confidence score vector in the continuous control cycle, and adjust the exploration temperature parameter in the Thompson sampling mechanism.

[0083] S6. Construct a joint input vector consisting of the environmental state vector and the target control action, and input the joint input vector into the embedded linear perception decision network to generate a set of control parameters.

[0084] In this embodiment, S1 specifically includes:

[0085] S11, The ambient temperature parameters include the air inlet temperature of the air conditioning equipment, the air outlet temperature, the temperature of key indoor areas, and the outdoor reference temperature;

[0086] S12, The environmental humidity parameters include the air inlet humidity, air outlet humidity and indoor relative humidity of the air conditioning equipment;

[0087] S13. The heat load parameters include personnel activity load, IT equipment operation heat load and lighting system heat load, and the heat load is expressed as the heat dissipation power per unit time.

[0088] S14. The power consumption parameters include compressor power consumption, fan power consumption, condenser power consumption, and auxiliary control unit power consumption, in kilowatt-hours.

[0089] S15. The operating status parameters of the air conditioning equipment include compressor start / stop status, fan speed, condenser heat exchange efficiency, air supply valve opening degree and current control mode.

[0090] This implementation method collects detailed data on ambient temperature, humidity, heat load, power consumption, and air conditioning equipment operating status, including multi-dimensional data such as inlet and outlet air temperature and humidity, indoor and outdoor temperature and humidity environment, equipment power consumption and operating status. This constructs an environmental state vector that comprehensively reflects the operating conditions of the air conditioning system, significantly improving the system's perception accuracy of the operating environment. This provides a high-quality data foundation for subsequent perception model scoring and control strategy optimization, thereby enhancing the scientific nature and responsiveness of energy-saving control.

[0091] In this embodiment, S2 specifically includes:

[0092] S21. Input the environmental state vector into the improved Banditron perceptron model;

[0093] S22. In the improved Banditron perceptron model, a corresponding scoring weight structure is assigned to each of the multiple preset control actions, and the scoring value of each preset control action is calculated based on the linear mapping relationship between the environmental state vector and the scoring weight vector.

[0094] S23. Construct a set of control actions based on the score values ​​of multiple preset control actions, and generate a confidence score vector representing the relative magnitude of the score values. The confidence score vector is used to characterize the response magnitude of each preset control action under the current environmental state vector.

[0095] S24. Output the set of control actions corresponding to all rating values ​​and the confidence rating vector.

[0096] This implementation method inputs the environmental state vector into an improved Banditron perceptron model, uses linear mapping to calculate the score values ​​of multiple preset control actions, and then constructs a set of control actions and generates a confidence score vector. This enables quantitative evaluation and priority ranking of air conditioning control strategies under different environmental conditions. The setting of the score weight structure allows the model to respond flexibly to specific input states and output more targeted combinations of control actions, significantly improving the generalization ability and response accuracy of the decision model in real complex environments.

[0097] In this embodiment, the improved Banditron perceptron model specifically includes:

[0098] Construct a scoring weight matrix, where each row corresponds to a scoring weight vector for a preset control action, used to receive environmental state vectors and perform scoring calculations;

[0099] The environmental state vector at the current time step is linearly multiplied by each scoring weight vector in the scoring weight matrix to calculate the score value of each preset control action.

[0100] Sort all scores and select the preset control action with the highest score as the predicted action at the current time step;

[0101] Based on the preset exploration probability, a predicted action is selected, and a random action is selected from other preset control actions whose score value is not the maximum according to the exploration probability as the action to be executed at the current time.

[0102] Compare the current action with the predicted action. If they match, record a feedback value of 1; otherwise, record a feedback value of 0.

[0103] The state confidence factor λ is calculated based on the proportional relationship between the prediction error range corresponding to the current environmental state vector and the reference disturbance amplitude. t The prediction error variation range is based on the difference between the environmental state vector at the current time step and the predicted score value at the previous time step. The reference perturbation amplitude is set as the reference perturbation amplitude in conjunction with a preset environmental state reference fluctuation amplitude.

[0104]

[0105] Where, Δ s tate is the prediction error at the current time step, Δ r ef is the preset reference disturbance amplitude, and β0 and β1 are adjustment coefficients;

[0106] This formula constructs a dynamic adjustment mechanism for the state confidence factor based on the ratio between the prediction error of the current time step environmental state vector and the preset reference disturbance amplitude. The principle is as follows: by comparing the current prediction error with a historically set disturbance range, the stability of the system state is assessed, and a state confidence factor is generated accordingly. When the current prediction error is close to the reference disturbance amplitude, the confidence factor decreases, indicating that the system state fluctuates significantly. When the prediction error is much smaller than the reference disturbance, the confidence factor increases, indicating that the system state is more stable. This state confidence factor can be used to dynamically adjust the response level of the control strategy in subsequent steps, achieving adaptive control to complex environmental changes.

[0107] When the feedback value is 1, the score weight vector corresponding to the action executed at the current time step is updated in a positive direction:

[0108]

[0109] When the feedback value is 0, the scoring weight vector is updated negatively:

[0110]

[0111] Where α is the negative feedback attenuation factor, x t This is the environment state vector at the current time step. The action to be executed at the current time step t is 'a'. t The rating weight vector.

[0112] This formula describes a mechanism for updating weights based on feedback values, used to dynamically adjust the scoring weight vector in the control model. When the executed control action receives positive feedback, positive reinforcement is achieved by adding the product of the environmental state vector and the state confidence factor to the current scoring weight. When negative feedback is received, the product is scaled by a decay factor and subtracted from the current scoring weight, achieving negative penalty. This mechanism reflects the essential logic of reinforcement learning training for control strategies, possessing the ability to adaptively update and accumulate experience, and continuously adjusting the scoring direction.

[0113] This implementation constructs a scoring weight matrix and performs linear inner product operations on the environmental state vector and the scoring weight vector one by one to score multiple preset control actions. The predicted action and the executed action are determined by ranking the scores. After dynamically selecting the executed action using an exploration probability mechanism, feedback values ​​are obtained by judging whether the executed action is consistent with the predicted action. Furthermore, a state confidence factor is calculated based on the proportional relationship between the prediction error range of the current environmental state and the set reference disturbance amplitude. Finally, the scoring weight vector is updated positively or negatively based on the feedback value to dynamically optimize the scoring mechanism of the control action. This implementation introduces an exploration mechanism and a weight adjustment strategy driven by the state confidence factor, which effectively improves the model's learning ability and action selection accuracy in dynamic control scenarios, thereby enhancing the energy-saving control effect and adaptability of the precision air conditioning system in complex environments.

[0114] In this embodiment, S3 specifically includes:

[0115] S31. Assign the confidence score vector to the set of control actions, and construct a posterior probability distribution for each control action in the set of control actions. The posterior probability distribution is input to the Thompson sampling mechanism with the confidence score value as the prior probability parameter.

[0116] S32. In the Thompson sampling mechanism, an independent sampling is performed based on the posterior probability distribution corresponding to each control action in the set of control actions to generate a set of sampled values.

[0117] S33. Determine the control action with the largest sample value in the sample value set, and take the corresponding control action as the candidate target control action under the current time step.

[0118] S34. Set the initial value of the exploration temperature parameter. The initial value of the exploration temperature parameter is calculated based on the mean and variance of the score distribution of the control action set, and satisfies the following constraint: Initial value of exploration temperature parameter = linear combination of mean and variance.

[0119] S35. Compare the explored temperature parameter at the current time step with the set temperature threshold. If the explored temperature parameter is less than the set temperature threshold, directly output the candidate target control action. If the explored temperature parameter is greater than the set temperature threshold, take the candidate target control action as the center, perform disturbance sampling in the control action set based on the current explored temperature parameter, determine and output the target control action.

[0120] This implementation constructs the posterior probability distribution of each control action by inputting the confidence score vector as prior information into the Thompson sampling mechanism. It obtains candidate target control actions based on independent sampling, and introduces a disturbance sampling mechanism in conjunction with the dynamic setting and judgment strategy of exploring temperature parameters. This enhances the ability to balance the exploration and utilization of control action selection, effectively improves the robustness of the air conditioning control strategy in uncertain environments and the optimization level of energy-saving control, and enhances the adaptive performance of the system under changing operating conditions.

[0121] In this embodiment, S4 specifically includes:

[0122] S41. Apply the target control action to the air conditioning equipment, wherein the target control action includes setting operations for compressor start / stop status, fan speed, air supply valve opening and current control mode;

[0123] S42. After the target control action is completed, collect the ambient temperature parameters, ambient humidity parameters and power consumption parameters within the corresponding control cycle.

[0124] S43. Normalize the air inlet temperature, air outlet temperature, indoor key area temperature and outdoor reference temperature of the air conditioning equipment in the ambient temperature parameters to generate the temperature feedback vector under the current control cycle.

[0125] S44. Normalize the air inlet humidity, air outlet humidity and indoor relative humidity of the air conditioning equipment in the environmental humidity parameters to generate a humidity feedback vector under the current control cycle.

[0126] S45. Weighted summation of compressor power consumption, fan power consumption, condenser power consumption and auxiliary control unit power consumption in the power consumption parameters to generate the power consumption value under the current control cycle.

[0127] S46. Connect the temperature feedback vector, humidity feedback vector, and power consumption value in series to construct a set of feedback parameters.

[0128] This implementation method applies the target control action to the compressor start / stop state, fan speed, air supply valve opening, and current control mode. After execution, it collects multi-dimensional feedback parameters and generates a structured feedback parameter set by normalization and weighted fusion. This achieves high-precision quantitative modeling of the air conditioning operating state. The feedback parameter set reflects the environmental changes after the control action, significantly enhancing the energy-saving response capability and environmental control efficiency of the air conditioning system in different operating cycles.

[0129] In this embodiment, S5 specifically includes:

[0130] S51. After each control cycle ends, record the set of feedback parameters and the confidence score vector for the current control cycle.

[0131] S52. Construct a feedback parameter matrix based on the set of feedback parameters recorded in multiple consecutive control cycles, wherein each row of the feedback parameter matrix corresponds to a control cycle;

[0132] S53. Calculate the moving average and standard deviation of the data in each dimension of the feedback parameter matrix, and extract them to form the feedback statistical fluctuation vector.

[0133] S54. Based on the confidence score vectors in two adjacent control cycles, calculate the relative rate of change of the score values ​​corresponding to each control action, and construct the confidence change vector.

[0134] S55. Linearly combine the feedback statistical fluctuation vector and the confidence change vector to generate the temperature adjustment factor.

[0135] S56. Based on the initial value of the exploration temperature parameter, and combined with the current temperature adjustment factor, the parameter is dynamically updated. The update method satisfies the following linear relationship: Current exploration temperature parameter = Initial value of exploration temperature parameter × Temperature adjustment factor.

[0136] This implementation method records the set of feedback parameters and the confidence score vector after each control cycle to construct a feedback parameter matrix. Moving averages and standard deviations are calculated for each dimension of the data to extract the feedback statistical fluctuation vector. The relative rate of change of the confidence score vector is calculated to form a confidence change vector. A linear combination of these two vectors yields the temperature adjustment factor, which dynamically adjusts the exploration temperature parameters. This achieves control strategy adjustment based on the system feedback change trend. This method can accurately and adaptively optimize the balance between exploration and utilization of the control strategy according to the fluctuation of the air conditioning operating status and the score change trend, improving the timeliness of strategy adjustment and the stability of energy-saving control.

[0137] In this embodiment, S6 specifically includes:

[0138] S61. The environmental state vector generated in the current control cycle and the target control action are dimensionally standardized and then spliced ​​together to form a joint input vector. The splicing method is a one-dimensional ordered series arrangement with the environmental state vector first and the target control action last.

[0139] S63. Input the joint input vector into the embedded linear sensing decision network, wherein the embedded linear sensing decision network includes, in sequence, an input layer, a hidden linear sensing layer, a normalization processing layer and an output layer.

[0140] S64. In the input layer, the joint input vector is received and the input dimension is uniformly mapped.

[0141] S65. In the hidden linear perceptron layer, a fully connected structure is used to perform a linear transformation on the input to generate intermediate mapping results;

[0142] S66. In the normalization processing layer, the intermediate mapping results are normalized to adjust the data distribution.

[0143] S67. In the output layer, the normalized data is subjected to perception mapping to generate a set of control parameters. The set of control parameters is used to guide the operation settings of the air conditioning equipment in subsequent control cycles.

[0144] This implementation method standardizes the environmental state vector generated in the current control cycle and the target control action, then concatenates them in a predetermined order to form a joint input vector, which is then input into an embedded linear perception decision network. This network includes an input layer, a hidden linear perception layer, a normalization processing layer, and an output layer. It sequentially completes input mapping, linear transformation, normalization processing, and perception mapping, ultimately generating a set of control parameters to accurately guide the operation configuration of the air conditioning equipment in the next control cycle. This achieves deep fusion modeling of environmental state and control behavior, effectively improving the accuracy of control parameter decisions and the flexibility of energy-saving response.

[0145] Example 1:

[0146] To verify the feasibility of this invention in practice, it was applied to the computer room of a financial data center for system deployment and on-site testing. This data center processes more than 120 million financial transactions per day. All transaction servers are deployed in the core area of ​​the IDC with extremely high environmental control requirements. The equipment is dense, the heat load changes drastically, and it is necessary to ensure that the air conditioning system can achieve high temperature control while being highly efficient and energy-saving.

[0147] In actual deployment, sensors are first deployed in different areas of the computer room (including air intake area, air outlet area, core hot area, and peripheral cold aisle) to collect ambient temperature parameters, including the air inlet temperature of the air conditioning equipment, the air outlet temperature, the temperature of key indoor areas, and the outdoor reference temperature. Industrial-grade DHT22 and DS18B20 temperature and humidity acquisition modules are used, with the data sampling period set to once every 15 seconds, and the data is uploaded to the edge node controller to collect ambient humidity parameters, heat load parameters, and power consumption parameters.

[0148] At the beginning of each control cycle, the system inputs the environmental state vector into the improved Banditron perceptron model constructed in this invention. The model sets an independent weight structure for each preset control action, calculates the score value using a linear mapping structure, and generates a confidence score vector based on the relative magnitude of the score value. After the score value is matched with the set of control actions, it is output to the next stage.

[0149] The system inputs the set of control actions and the confidence score vector into the Thompson sampling mechanism. For each control action, a posterior probability distribution is constructed. The construction of the posterior probability is based on the score value as a prior parameter. The Thompson sampling mechanism independently samples these distributions and selects the control action corresponding to the maximum value in the sampled value set as a candidate target control action. The temperature parameter is linearly calculated based on the mean and standard deviation of the score value and dynamically adjusted in combination with the control cycle number. When the set threshold condition is reached, the candidate target action is perturbed and sampled. Finally, the target control action in the current control cycle is output.

[0150] The target control actions are applied to the air conditioning system, specifically including compressor start / stop status adjustment, fan speed control, air supply valve opening setting, and current control mode switching. After the target control actions are executed, the system automatically collects a new set of environmental temperature and humidity parameters and power consumption parameters within the control cycle. The collected temperature data is normalized to generate a temperature feedback vector, the humidity data is normalized to generate a humidity feedback vector, and the power consumption parameters are weighted and summed to obtain the power consumption value. The three are then combined to form a set of feedback parameters.

[0151] After each control cycle, the system stores and records the set of feedback parameters and the confidence score vector. Over multiple consecutive cycles, a feedback parameter matrix and a score change matrix are formed. By calculating their moving mean and standard deviation, a feedback statistical fluctuation vector is formed. Combined with the rate of change of the confidence score, a confidence change vector is constructed. The two are linearly combined to form a temperature adjustment factor, which is used to dynamically update the exploration temperature parameters and achieve continuous optimization of the exploration strategy.

[0152] The system concatenates the environmental state vector with the target control action to generate a joint input vector, which is then fed into an embedded linear sensing decision network. This network comprises an input layer, a hidden linear sensing layer, a normalization processing layer, and an output layer. After unifying the mapping dimension in the input layer, the hidden linear sensing layer employs a fully connected linear transformation, the normalization layer adjusts the data distribution to adapt to the output requirements, and finally, the output layer generates the set of control parameters required for the current control cycle, used for precise control of the equipment's operating state.

[0153] The experiment lasted for two months, with a system sampling frequency of every 15 seconds and a control cycle of every 5 minutes, generating approximately 288 control strategies per day. The following key indicators were collected for comparative analysis:

[0154] Table 1: Performance Comparison of the Invention Method and Traditional Control Methods

[0155]

[0156]

[0157] Experimental data show that the proposed solution significantly optimizes control accuracy and energy efficiency during multi-round control strategy learning. Especially during periods of frequent load changes in hotspot areas, the system can adjust the control strategy in real time through a feedback-driven temperature parameter update mechanism, maintaining the temperature within a fluctuation range of 0.4℃. This greatly reduces energy consumption while improving stability. The dynamic updating of the temperature parameters allows the system to achieve higher stability and convergence speed while ensuring exploration capabilities.

[0158] The solution of this invention has good adaptability, stability and energy saving effect in actual complex air conditioning systems. It can realize continuous control strategy optimization and respond efficiently to environmental fluctuations. The system can effectively replace traditional static control logic.

[0159] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A precision air conditioning energy-saving control optimization method based on reinforcement learning, characterized in that, Includes the following steps: S1. Collect ambient temperature parameters, ambient humidity parameters, heat load parameters, power consumption parameters and operating status parameters of the air conditioning equipment, and generate an environmental state vector; S2. Input the environmental state vector into the improved model. The perceptron model scores multiple preset control actions and outputs a set of control actions and a corresponding confidence score vector. S3. Input the confidence score vector and the set of control actions into the Thompson sampling mechanism, initialize the posterior probability distribution of the control actions, set the exploration temperature parameter, sample the set of control actions, and output the target control action. S4. Execute the target control action, and collect the ambient temperature parameters, ambient humidity parameters and power consumption parameters generated after the target control action is executed to form a set of feedback parameters. S5. Analyze and process the distribution changes of the feedback parameter set and confidence score vector in the continuous control cycle, and adjust the exploration temperature parameter in the Thompson sampling mechanism. S6. Construct a joint input vector consisting of the environmental state vector and the target control action, and input the joint input vector into the embedded linear perception decision network to generate a set of control parameters. S1 specifically includes: S11, The ambient temperature parameters include the air inlet temperature of the air conditioning equipment, the air outlet temperature, the temperature of key indoor areas, and the outdoor reference temperature; S12, The environmental humidity parameters include the air inlet humidity, air outlet humidity and indoor relative humidity of the air conditioning equipment; S13. The heat load parameters include personnel activity load, IT equipment operation heat load and lighting system heat load, and the heat load is expressed as the heat dissipation power per unit time. S14. The power consumption parameters include compressor power consumption, fan power consumption, condenser power consumption, and auxiliary control unit power consumption, in kilowatt-hours. S15. The operating status parameters of the air conditioning equipment include compressor start / stop status, fan speed, condenser heat exchange efficiency, air supply valve opening degree and current control mode. S2 specifically includes: S21. Input the environmental state vector into the improved model. Perceptron model; S22, in the improved version In the perceptron model, a corresponding scoring weight structure is assigned to each of the multiple preset control actions, and the score value of each preset control action is calculated based on the linear mapping relationship between the environmental state vector and the scoring weight vector. S23. Construct a set of control actions based on the score values ​​of multiple preset control actions, and generate a confidence score vector representing the relative magnitude of the score values. The confidence score vector is used to characterize the response magnitude of each preset control action under the current environmental state vector. S24. Output the set of control actions corresponding to all rating values ​​and the confidence rating vector.

2. The precision air conditioning energy-saving control optimization method based on reinforcement learning according to claim 1, characterized in that, The improved version The perceptron model specifically includes: Construct a scoring weight matrix, where each row corresponds to a scoring weight vector for a preset control action, used to receive environmental state vectors and perform scoring calculations; The environmental state vector at the current time step is linearly multiplied by each scoring weight vector in the scoring weight matrix to calculate the score value of each preset control action. Sort all scores and select the preset control action with the highest score as the predicted action at the current time step; Based on the preset exploration probability, a predicted action is selected, and a random action is selected from other preset control actions whose score value is not the maximum according to the exploration probability as the action to be executed at the current time. Compare the current action with the predicted action. If they match, record a feedback value of 1; otherwise, record a feedback value of 0. The state confidence factor is calculated based on the proportional relationship between the prediction error variation range corresponding to the current environmental state vector and the reference disturbance amplitude. The prediction error variation range is based on the difference between the environmental state vector at the current time step and the prediction score value at the previous time step. The reference disturbance amplitude is set as the reference disturbance amplitude in combination with the preset environmental state reference fluctuation amplitude. When the feedback value is 1, the scoring weight vector corresponding to the action executed at the current time step is updated positively; when the feedback value is 0, the scoring weight vector is updated negatively.

3. The precision air conditioning energy-saving control optimization method based on reinforcement learning according to claim 1, characterized in that, S3 specifically includes: S31. Map the confidence score vector to the set of control actions, and construct the posterior probability distribution for each control action in the set of control actions. S32. In the Thompson sampling mechanism, an independent sampling is performed based on the posterior probability distribution corresponding to each control action in the set of control actions to generate a set of sampled values. S33. Determine the control action with the largest sample value in the sample value set, and take the corresponding control action as the candidate target control action under the current time step. S34. Set the initial value of the exploration temperature parameter. The initial value of the exploration temperature parameter is calculated based on the mean and variance of the score distribution of the control action set, and satisfies the following constraint: Initial value of exploration temperature parameter = linear combination of mean and variance. S35. Compare the explored temperature parameter at the current time step with the set temperature threshold. If the explored temperature parameter is less than the set temperature threshold, directly output the candidate target control action. If the explored temperature parameter is greater than the set temperature threshold, take the candidate target control action as the center, perform disturbance sampling in the control action set based on the current explored temperature parameter, determine and output the target control action.

4. The precision air conditioning energy-saving control optimization method based on reinforcement learning according to claim 1, characterized in that, Step S4 specifically includes: S41. Apply the target control action to the air conditioning equipment, wherein the target control action includes setting operations for compressor start / stop status, fan speed, air supply valve opening and current control mode; S42. After the target control action is completed, collect the ambient temperature parameters, ambient humidity parameters and power consumption parameters within the corresponding control cycle. S43. Normalize the air inlet temperature, air outlet temperature, indoor key area temperature and outdoor reference temperature of the air conditioning equipment in the ambient temperature parameters to generate the temperature feedback vector under the current control cycle. S44. Normalize the air inlet humidity, air outlet humidity and indoor relative humidity of the air conditioning equipment in the environmental humidity parameters to generate a humidity feedback vector under the current control cycle. S45. Weighted summation of compressor power consumption, fan power consumption, condenser power consumption and auxiliary control unit power consumption in the power consumption parameters to generate the power consumption value under the current control cycle. S46. Connect the temperature feedback vector, humidity feedback vector, and power consumption value in series to construct a set of feedback parameters.

5. The precision air conditioning energy-saving control optimization method based on reinforcement learning according to claim 1, characterized in that, S5 specifically includes: S51. After each control cycle ends, record the set of feedback parameters and the confidence score vector for the current control cycle. S52. Construct a feedback parameter matrix based on the set of feedback parameters recorded in multiple consecutive control cycles, wherein each row of the feedback parameter matrix corresponds to a control cycle; S53. Calculate the moving average and standard deviation of the data in each dimension of the feedback parameter matrix, and extract them to form the feedback statistical fluctuation vector. S54. Based on the confidence score vectors in two adjacent control cycles, calculate the relative rate of change of the score values ​​corresponding to each control action, and construct the confidence change vector. S55. Linearly combine the feedback statistical fluctuation vector and the confidence change vector to generate the temperature adjustment factor. S56. Based on the initial value of the exploration temperature parameter, and combined with the current temperature adjustment factor, the parameter is dynamically updated. The update method satisfies the following linear relationship: Current exploration temperature parameter = Initial value of exploration temperature parameter × Temperature adjustment factor.

6. The precision air conditioning energy-saving control optimization method based on reinforcement learning according to claim 1, characterized in that, S6 specifically includes: S61. The environmental state vector generated in the current control cycle and the target control action are dimensionally standardized and then spliced ​​together to form a joint input vector. The splicing method is a one-dimensional ordered series arrangement with the environmental state vector first and the target control action last. S63. Input the joint input vector into the embedded linear sensing decision network, wherein the embedded linear sensing decision network includes, in sequence, an input layer, a hidden linear sensing layer, a normalization processing layer and an output layer. S64. In the input layer, the joint input vector is received and the input dimension is uniformly mapped. S65. In the hidden linear perceptron layer, a fully connected structure is used to perform a linear transformation on the input to generate intermediate mapping results; S66. In the normalization processing layer, the intermediate mapping results are normalized to adjust the data distribution. S67. In the output layer, the normalized data is perceptually mapped to generate a set of control parameters.

Images