Intelligent control method of evaporative cooling air conditioning system based on deep reinforcement learning

Abstract

An intelligent control method of an evaporative cooling air conditioning system based on deep reinforcement learning belongs to the technical field of refrigeration equipment regulation and control, and comprises the following steps: acquiring a space element of a refrigeration area, and converting the space element into a digital model, so as to generate a functional block division strategy of the refrigeration area based on the digital model. Based on a functional block division strategy, acquiring sensor data on each functional block fusion node, and preprocessing the sensor data to obtain a multi-dimensional feature data set; and fusing the multi-dimensional feature data set into a performance index to output a performance evaluation vector of each functional block. And based on the performance evaluation vector, constructing an experience playback pool fusion global state vector so as to carry out offline training on a DDPG algorithm and obtain an intelligent control strategy network. And the real-time performance index of the refrigeration area is input into an intelligent control strategy network to be processed, and a control instruction of the evaporative cooling air conditioning system is output. And controlling the valve opening and the fan frequency of each functional block in response to the control instruction.

Description

Technical Field

[0001] The present invention belongs to the technical field of refrigeration equipment control, and more specifically, relates to an intelligent control method for an evaporative cooling air conditioning system based on deep reinforcement learning. Background Art

[0002] In an evaporative cooling air conditioning system, if the supply air temperature is simply controlled within a fixed range, various coupling complexities and dynamic disturbances will be faced during actual operation, making it difficult for traditional PID control to take into account multiple objective requirements such as optimal energy consumption, humidity control, equipment wear, and load fluctuations.

[0003] Currently, when traditional air conditioning systems mainly control modes such as refrigeration, heating, ventilation, and dehumidification through a remote control, only beeping sounds and simple symbols are used to indicate mode switching, and it is difficult to accurately obtain parameters such as the current operating voltage of the unit, compressor operation, indoor and outdoor motor operation, indoor and outdoor temperatures, supply air temperature, and power consumption. It can only be collected by human senses or with the help of other devices, resulting in users being difficult to effectively and scientifically maintain and use the air conditioning system. Such problems are particularly significant in the maintenance and use of evaporative cooling air conditioning systems.

[0004] First of all, in an evaporative cooling air conditioning system, in addition to the supply air temperature, it is also necessary to maintain operating condition variables such as indoor humidity, cooling water flow rate, and pressure difference before and after the tower. These variables are highly coupled, and the operation of each actuator (fan, water pump, valve) does not have a linear superposition effect on the entire evaporative cooling air conditioning system. For example, increasing the fan speed is beneficial to the decrease of the supply air temperature, but it is more likely to cause humidity fluctuations and energy consumption increase. Secondly, the evaporative cooling air conditioning system is more susceptible to environmental changes such as outdoor temperature, relative humidity, indoor human flow, and window opening. Traditional PID control technology requires frequent manual readjustment of gain parameters and is difficult to achieve self-adaptation. Finally, the heat transfer efficiency of the evaporative cooling tower, the water film thickness, and the fan performance curve all have strong nonlinearity and uncertainty, and the actual calibration is difficult. Summary of the Invention

[0005] To solve the deficiencies in the prior art, the purpose of the present invention is to solve the above-mentioned defects, and further propose an intelligent control method for an evaporative cooling air conditioning system based on deep reinforcement learning.

[0006] The present invention adopts the following technical solutions.

[0007] The first aspect of the present invention discloses an intelligent control method for an evaporative cooling air conditioning system based on deep reinforcement learning, and the method includes: Obtain the spatial elements of the refrigeration area, and convert the spatial elements into a digital model to generate a functional block division strategy for the refrigeration area based on the digital model; Based on the above functional block division strategy, obtain the sensor data on each functional block fusion node, and preprocess the sensor data to obtain a multi-dimensional feature dataset; Fuse the multi-dimensional feature dataset into performance indicators to output the performance evaluation vectors of each functional block; Based on the performance evaluation vectors, construct an experience replay pool to fuse the global state vectors, and perform offline training on the DDPG algorithm to obtain an intelligent control strategy network; Input the real-time performance indicators of the refrigeration area into the intelligent control strategy network for processing, and output the control instructions for the evaporative cooling air conditioning system; Respond to the control instructions to control the valve opening and fan frequency of each functional block; Among them, the spatial elements include building floor plans, cooling load distributions, and pipe network structures, and the digital model includes digitized spatial coordinate sets, load mappings, and pipe network topologies; the sensor data includes temperature, humidity, flow rate, and pressure difference.

[0008] Further, the method for obtaining the spatial elements of the refrigeration area, converting the spatial elements into a digital model, and generating the functional block division strategy for the refrigeration area based on the digital model includes: Based on the digitized spatial coordinate set, load mapping, and pipe network topology, divide the refrigeration area into multiple functional blocks; Select key nodes from each functional block, and set sensor units at the key nodes to generate the functional block division strategy; the sensor units include temperature and humidity sensors, flow meters, and differential pressure gauges.

[0009] Further, the method for obtaining the sensor data on each functional block fusion node based on the functional block division strategy and preprocessing the sensor data to obtain a multi-dimensional feature dataset includes: Receive the sensor data from each sensor unit, and construct an initial sequence matrix based on the sensor data according to the sampling period and time series; Perform moving average filtering on each column of sensor data in the initial sequence matrix to obtain a smoothed sequence matrix; Fill in the missing values in the smoothed sequence matrix, and perform normalization processing on the filled smoothed sequence matrix through linear mapping to obtain the multi-dimensional feature dataset.

[0010] Further, the method for fusing the multi-dimensional feature dataset into performance indicators to output the performance evaluation vectors of each functional block includes: Timestamp alignment is performed on the time-series data of each functional block in the multi-dimensional feature dataset through the Network Time Protocol, and linear interpolation is used to fill in the sensor data with a time error exceeding the first threshold; The health status of the sensor device is judged by the heartbeat detection frequency and residual statistics, and health weights are assigned to each sensor data according to the degree of health, so as to fill in the sensor data with a health weight lower than the second threshold to obtain health-weighted time-series data; Perform sliding window median filtering on the health-weighted time-series data, and perform weighted averaging on the sensor data at the same moment according to the health weight to obtain the fusion time-series curve of each functional block; Statistical performance indicators of the fusion time-series curve according to the set sliding window; the performance indicators include cooling efficiency, temperature and humidity deviation, and energy load; Mark the performance indicators with a sliding average abnormal jump exceeding the third threshold as abnormal indicators, and package the abnormal indicators and performance indicators to obtain the performance evaluation vector.

[0011] Further, constructing an experience replay pool based on the performance evaluation vector to fuse the global state vector, and performing offline training on the DDPG algorithm to obtain an intelligent control strategy network, including: Splice the performance evaluation vectors of each functional block according to time series into the global state vector of the refrigeration area; When the experience replay pool reaches the capacity limit, sort each global state vector according to the TD error size of the global state vector, and retain the global state vectors with a TD error exceeding the fourth threshold to obtain global state samples.

[0012] Further, constructing an experience replay pool based on the performance evaluation vector to fuse the global state vector, and performing offline training on the DDPG algorithm to obtain an intelligent control strategy network, further including: Judge whether the temperature difference or energy consumption of each functional block exceeds the preset high warning threshold; if so, mark the functional block as the first label; if not, mark the functional block as the second label; When multiple adjacent functional blocks are marked as the first label, add coupling weights to the multiple adjacent functional blocks, and splice the functional blocks carrying the coupling weights with other functional blocks to obtain a fusion state vector; Use the fusion state vector as the DDPG algorithm with the Actor-Critic framework for offline training to obtain the intelligent control strategy network.

[0013] Further, inputting the real-time performance indicators of the refrigeration area into the intelligent control strategy network for processing, and outputting control instructions for the evaporative cooling air conditioning system, including: Obtain the real-time performance indicators of each functional block in the refrigeration area, and screen the real-time performance indicators according to the health labels of the real-time performance indicators, so as to eliminate and fill the sensor data with a health level lower than the fifth threshold, and obtain the quaternary performance values of each functional block; Concatenate the quaternary performance values of all functional blocks, and carry the global state vector and health weight of each functional block during concatenation to output the real-time global state vector; Call the intelligent control strategy network to process the real-time global state vector, so as to generate the recommended valve opening and fan frequency of each functional block of the evaporative cooling air conditioning system, and output the control instructions executed according to the recommended valve opening and fan frequency.

[0014] Further, the method further includes: When a sensor device of any functional block fails, it is determined that the functional block is a faulty block, switch the faulty block to a preset PID redundant loop, and send an alarm message to the operation and maintenance terminal at the same time; Generate a global action vector based on the recommended valve opening and fan frequency, and send the global action vector to the control center, and the control center issues the control instructions to the PLC through the field bus.

[0015] A second aspect of the present invention discloses a terminal, including a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method described in the first aspect.

[0016] A third aspect of the present invention discloses a computer-readable storage medium, on which a computer program is stored, characterized in that the program implements the steps of the method described in the first aspect when executed by a processor.

[0017] The beneficial effects of the present invention are that, compared with the prior art, the present invention has the following advantages: (1) In view of the temperature and humidity gradient and pipe network diversity in the large space of the computer room, the present invention deploys data acquisition and preprocessing units in zones, realizes near-real-time perception and preliminary fusion, and provides a data basis for subsequent space cooling analysis and control.

[0018] (2) The present invention integrates multi-dimensional raw data into comparable performance indicators, which can not only reflect the cooling efficiency of each area but also measure energy consumption and comfort, providing a refined view for intelligent decision-making. At the same time, the DDPG algorithm of the Actor-Critic architecture is introduced to optimize and verify the strategy. Through training, an intelligent control strategy network that can be directly deployed is obtained, which can learn the optimal operation strategy in the high-dimensional state space, balance the conflicts between various regulation objectives, and achieve the adaptive regulation of the evaporative cooling air-conditioning system. Description of the Drawings

[0019] Figure 1 It is a schematic flow chart of the intelligent control method for the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention. Detailed Embodiments

[0020] The following further describes the present application with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present application.

[0021] As Figure 1 shown, in one embodiment, an intelligent control method for an evaporative cooling air-conditioning system based on deep reinforcement learning includes the following steps: Step S110, obtain the spatial elements of the refrigeration area, convert the spatial elements into a digital model, and generate a functional block division strategy for the refrigeration area based on the digital model.

[0022] Among them, the spatial elements include the building floor plan, cooling load distribution, and pipe network structure, and the digital model includes a digitalized spatial coordinate set, load mapping, and pipe network topology diagram.

[0023] In some embodiments, for the intelligent control method for the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention, step S110 specifically includes the following steps: Step S111, divide the refrigeration area into multiple functional blocks based on the digitalized spatial coordinate set, load mapping, and pipe network topology diagram.

[0024] Step S112, select key nodes from each functional block, and set sensor units at the key nodes to generate a functional block division strategy; the sensor units include temperature and humidity sensors, flow meters, and differential pressure gauges.

[0025] In a specific embodiment, the intelligent control method for the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention includes steps 1 to 5: Step 1, system block division and fusion node deployment.

[0026] Specifically, in view of the temperature and humidity gradients and pipeline diversities in the large space of the computer room, the data acquisition and preprocessing units are deployed in zones to achieve near-real-time perception and preliminary fusion in the vicinity.

[0027] It includes the following steps: Step 1.1, basic data acquisition and digital modeling.

[0028] Specifically, first, obtain the building floor plan: including the two-dimensional plane layout of each air supply outlet, air return outlet, and wet curtain in the computer room; cold load distribution: the instantaneous cold load index corresponding to each air supply outlet (such as the cooling capacity collected in real time by the energy consumption management system); pipeline structure: the main and branch pipeline directions and valve node positions between the evaporative cooling tower and the air supply outlet. Convert the paper or CAD drawings and operation data into a computable digital model, laying the foundation of "where, how much load, and how the pipelines are connected", and providing the three elements of space-thermal-connection for the subsequent functional block division.

[0029] Step 1.2, functional block division.

[0030] Specifically, classify the air supply outlets and air return circuits with small heat load differences into the same block to avoid uneven heating and cooling in the large area, ensure that the air supply circuit and the cooling water circuit in the same block are connected or nearly connected in the physical pipeline network, reduce the response delay caused by long-distance transmission of cross-region pipelines, and try to keep each block coherent in the room layout and reduce the situation where one block spans multiple rooms. Finally, output several functional block division schemes. The present invention solves the problems of excessive temperature and humidity gradients and chaotic cross-control of pipeline networks in a single area, and provides clear spatial boundaries for subsequent parallel processing and local optimization in different zones.

[0031] Step 1.3, precise positioning of edge fusion nodes.

[0032] Specifically, in each functional block, select the following "key points" as the primary deployment locations: At the wet curtain inlet: This is the key position where the evaporative cold water directly contacts the air, and the temperature and humidity of the inlet and outlet water need to be monitored; At the pump outlet branch point: The intersection of the main water supply pipe and the branch pipe, and the water volume distribution of each branch is monitored through a flow meter; At the air supply pipeline bifurcation: The node before the main air supply pipe bifurcates to each air supply outlet, and the overall air supply characteristics are monitored with temperature and humidity sensors; At the valve control node: The position of each regulating valve, and the flow resistance change caused by the valve opening is detected with a differential pressure sensor.

[0033] After that, a fusion node (including a temperature and humidity sensor, a flow meter, a differential pressure gauge, and an edge computing unit) is preferentially deployed at each key point. If the area or load of a block is large, additional nodes are added in the middle of the area or at the remaining hot spots according to equidistant or load gradient distribution to ensure that there is no monitoring blind spot in the entire block. This process ensures that the key parts of heat exchange and water circuits in each block are monitored in real time with high precision. The edge nodes complete preliminary data cleaning and fusion locally, reducing the burden on the central system and accelerating the response speed.

[0034] Step S120: Based on the functional block division strategy, obtain the sensor data on the fusion nodes of each functional block, and preprocess the sensor data to obtain a multi-dimensional feature data set.

[0035] Among them, the sensor data includes temperature, humidity, flow rate, and differential pressure.

[0036] In some embodiments, for the intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning provided by the present invention, step S120 specifically includes the following steps: Step S121: Receive the sensor data from each sensor unit, and construct an initial sequence matrix based on the sensor data according to the sampling period and time series.

[0037] Step S122: Perform a moving average filtering process on the sensor data in each column of the initial sequence matrix to obtain a smoothed sequence matrix.

[0038] Step S123: Fill in the missing values in the smoothed sequence matrix, and perform normalization processing on the filled smoothed sequence matrix through linear mapping to obtain a multi-dimensional feature data set.

[0039] In a specific embodiment, for the intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning provided by the present invention, step 2: Multi-dimensional feature acquisition and preprocessing. Step 2 is used to clean and unify the data formats of various sensors, solve noise and missing interference, and provide high-quality input for subsequent fusion analysis, including the following steps: Step 2.1: Original signal acquisition.

[0040] Specifically, first obtain the sensor data on the fusion nodes of each functional block, including the supply air temperature sensor data , relative humidity sensor data , circulating water flow meter output data , and wet curtain differential pressure sensor data , and construct an original time series matrix according to the time series of these sensor data: Among them, is the total time and t is the instant.

[0041] The above original time series matrix The four types of sensor data are synchronously read at a fixed sampling period (e.g., 1s) and cached locally. This process unifies the analog / digital signals of heterogeneous sensors into a structured time series, laying the foundation for subsequent cleaning and analysis.

[0042] Step 2.2, noise filtering.

[0043] Specifically, a sliding average filter is applied to each column of data in the matrix R to process temperature, humidity, flow rate, and pressure difference independently, and a window length w (such as 5 to 15 points) is selected, and its expression is: in, is the smoothed result, as the element in the smoothed matrix S, t is the time, and k represents the unit time corresponding to the window length.

[0044] This process eliminates the spikes and jitters caused by sensor noise and pipeline pulsation, improves the stability of the sensor signal, and avoids subsequent misjudgments.

[0045] Step 2.3: fill in missing values.

[0046] Specifically, when a single point is missing in the smoothed matrix S, linear interpolation is used to complete it. If more than three points are lost in succession, the historical average value of the functional block or the data of the adjacent synchronous block is used to backfill to ensure continuity, and finally output a complete matrix I without missing points. This process fills the data gaps caused by network jitter or short-term sensor failure, ensures the integrity of the feature sequence, and prevents NAN from occurring during training.

[0047] Step 2.4, feature normalization.

[0048] Specifically, each channel in the complete matrix I is linearly mapped to the interval [0,1], and the mapping formula is: in, represents the value of the jth feature (one of temperature, humidity, flow rate, and pressure difference) at time t, Indicates the mapping result corresponding to the value of the jth feature (one of temperature, humidity, flow rate, and pressure difference) at time t. , They respectively represent the lower and upper limits of the sensor range corresponding to this feature.

[0049] This process unifies physical quantities of different dimensions, eliminates differences in numerical scales, and can ensure stable convergence during the training of deep reinforcement learning models.

[0050] Step S130, fuse the multi-dimensional feature dataset into performance metrics to output the performance evaluation vectors of each functional block.

[0051] In some embodiments, for the intelligent control method of the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention, step S130 specifically includes the following steps: Step S131, perform timestamp alignment on the time-series data of each functional block in the multi-dimensional feature dataset through the Network Time Protocol, and perform linear interpolation to fill in the sensor data with a time error exceeding the first threshold.

[0052] Step S132, judge the health status of the sensor devices through the heartbeat detection frequency and residual statistics, and assign health weights to each sensor data according to the degree of health, so as to fill in the sensor data with a health weight lower than the second threshold to obtain health-weighted time-series data.

[0053] Step S133, perform moving window median filtering on the health-weighted time-series data, and perform weighted averaging on the sensor data at the same moment according to the health weights to obtain the fused time-series curve of each functional block.

[0054] Step S134, count the performance metrics of the fused time-series curve according to the set sliding window; the performance metrics include cooling efficiency, temperature and humidity deviation, and energy load.

[0055] Step S135, mark the performance metrics with a moving average abnormal jump exceeding the third threshold as abnormal metrics, and package the abnormal metrics and performance metrics to obtain the performance evaluation vector.

[0056] In a specific embodiment, for the intelligent control method of the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention, step 3, distributed fusion analysis generates regional performance metrics. Step 3 is used to fuse the multi-dimensional raw data into comparable performance metrics, which can not only reflect the cooling efficiency of each area, but also measure the energy consumption and comfort, and provide a refined view for intelligent decision-making, including the following steps: Step 3.1, time alignment and robustness weighting.

[0057] Specifically, first, align the timestamps of each node with the Network Time Protocol (NTP), and perform linear interpolation to fill in the data with an error exceeding half of the sampling period. For sensor drift or disconnection, judge the health status through the heartbeat detection frequency and residual statistics within the previous and next two minutes, and dynamically assign weights according to the degree of health. For the node data with extremely low weights or long-term dropouts, directly set the weights to zero and use the data of the nearest neighbor node to fill in, and finally output the health-weighted time-series data. This process can ensure the time consistency of the fused basic data, dynamically reduce the influence of unreliable nodes, and improve the subsequent fusion quality.

[0058] Step 3.2, Abnormal Elimination and Weighted Fusion.

[0059] Specifically, for the time series in the health-weighted time series data output in Step 3.1, the sliding window median filter (window length 10s) is used to eliminate extreme pulses in the time series. The readings of each node at the same moment are weighted and averaged according to the health weights to obtain a fusion value sequence representing the entire block. Finally, the fused time series curves of each physical quantity (temperature difference, humidity, flow rate, pressure difference) are integrated and output. This process uses median filtering to eliminate accidental spikes, and weighted averaging ensures that the fused readings are both stable and representative of the overall situation.

[0060] Step 3.3, Key Index Period Statistics.

[0061] Specifically, a 1-minute sliding window is used for the fused time series curve, and the indicators within each window are calculated at a 10s step length, and a rolling queue of the last 6 windows is maintained. If the sliding average abnormal jump exceeds 10%, it is marked as "indicator abnormal" and enters the priority alarm flow, and three indicators within the sliding statistical window are obtained: cooling efficiency (average temperature difference), humidity deviation (average humidity deviation), and energy consumption load (average of flow rate × pressure difference). This process provides a multi-period comparison basis for comprehensive evaluation by regularly evaluating the short-term performance of the block and capturing sudden anomalies.

[0062] Step 3.4, Abnormal Priority and Comprehensive Vector Packing.

[0063] Specifically, based on the three indicators and the abnormal marks obtained in Step 3.3, if the "cooling efficiency" or "humidity deviation" triggers an abnormal mark, the comprehensive score directly takes this indicator and enters the priority adjustment. Otherwise, the weighted sum is obtained by normalizing the three indicators according to the preset comfort weight (0.6) and energy-saving weight (0.4), and the three original statistical values and the comprehensive score are output together as a performance vector. This process achieves a balance between comfort and energy saving under most normal operating conditions. Once serious discomfort or energy consumption anomalies occur, the specific problem item is preferentially fed back to guide the intelligent agent to focus on correction.

[0064] Step S140, Construct an experience replay pool based on the performance evaluation vector to fuse the global state vector, and perform offline training on the DDPG algorithm to obtain an intelligent control strategy network.

[0065] In some embodiments, for the intelligent control method of the evaporative cooling air conditioning system provided by the present invention, Step S140 specifically includes the following steps: Step S141, Concatenate the performance evaluation vectors of each functional block in time series to form a global state vector of the refrigeration area.

[0066] Step S142, when the experience replay pool reaches the capacity limit, sort the global state vectors according to the TD error magnitude of the global state vectors, so as to retain the global state vectors whose TD errors exceed the fourth threshold, and obtain global state samples.

[0067] In some embodiments, for the intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning provided by the present invention, step S140 specifically further includes the following steps: Step S143, determine whether the temperature difference or energy consumption of each functional block exceeds a preset high warning threshold; if so, label the functional block with a first label; if not, label the functional block with a second label.

[0068] Step S144, when multiple adjacent functional blocks are labeled with the first label, add coupling weights to the multiple adjacent functional blocks, and splice the functional blocks carrying the coupling weights with other functional blocks to obtain a fused state vector.

[0069] Step S145, use the fused state vector for offline training with the DDPG algorithm based on the Actor-Critic framework to obtain an intelligent control policy network.

[0070] In a specific embodiment, for the intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning provided by the present invention, step 4, offline deep reinforcement learning policy training. Step 4 takes the performance evaluation vectors of each functional block output in step 3, the corresponding historical control actions, and reward signals as inputs, performs offline training through constructing an experience replay pool, fusing global states, and using the DDPG algorithm with the Actor–Critic architecture, and tunes and validates the policy, and finally outputs an intelligent control policy network that can be directly deployed. It includes the following steps: Step 4.1, construct an experience replay pool.

[0071] Specifically, first, determine the performance evaluation vectors (quadruple: temperature difference, humidity deviation, energy consumption load, comprehensive score) of all blocks in each control period (for example, every 1 minute); the corresponding action vectors (cooling water valve opening and fan frequency of each block) in the same period, and the reward value (the global reward scalar synthesized according to the regional priority after calculating the "comfort - energy consumption" return of each block with a reward function). Secondly, for the recorded offline simulation or on-site log, sequentially read the performance vectors of each block in time sequence and splice them into a global state, and at the same time read the corresponding actions and rewards, and then read the state of the next period to generate a complete sample.

[0072] To prevent the policy from only learning normal working conditions, a "high-return sample pool" and a "low-return sample pool" are established. Samples with returns greater than the upper percentile and lower than the lower percentile are saved with a higher probability, while ordinary samples are put into the pool with a lower probability, ensuring that the replay pool takes into account both extreme and normal experiences. When the replay pool reaches its capacity limit, the samples are sorted according to the magnitude of their TD errors (i.e., the Critic estimation errors), and samples with larger errors are preferentially retained, while the samples with the lowest priority are eliminated.

[0073] Periodically save the snapshot of the replay pool to the database for easy recovery when training restarts, support multi-version management, and record the sample range used in each batch of training for traceability. Finally, output an experience replay pool DDD in the form of a double-ended circular queue, storing the quadruple of "global state - action - return - next global state", with a configurable capacity (e.g., 100,000 entries).

[0074] This process breaks the temporal correlation, improves the training stability and efficiency, and ensures that the policy has sufficient learning samples under various working conditions (extreme and normal) through the balancing and prioritization strategies. The persistence of the replay pool provides a data basis for experiment reproducibility and cross-team collaboration.

[0075] Step 4.2, Global state fusion and regional coupling.

[0076] Specifically, first determine the original global state vector in the replay pool (the concatenation of the quadruples of the performance of each block) and the predefined "high warning threshold" and "coupling regulation threshold" - such as a temperature difference exceeding 3°C or an energy consumption index exceeding 80% of the historical peak. Perform threshold judgment on the temperature difference and energy consumption indicators of each block: If the temperature difference or energy consumption of a certain block exceeds the "high warning threshold", the label of this block is "urgently in need of adjustment"; otherwise, the label is "normal". If multiple (≥2) adjacent blocks (connected positions in the pipe network topology) are simultaneously marked as "urgently in need of adjustment", the "coupling regulation" mode is triggered; an additional one-dimensional "coupling weight" is added to these blocks, and the value can be allocated according to the proportion of the number of blocks exceeding the threshold (e.g., if two blocks exceed the threshold simultaneously, the weight is 0.8; if three blocks exceed the threshold, the weight is 0.9). In addition, the quadruple performance vectors of all blocks need to be concatenated in a fixed order together with their respective "urgently in need of adjustment" binary labels and "coupling weights". If the overall is a "coupling regulation" scenario, an additional "coupling trigger" flag bit is appended at the end of the vector. Finally, output a fused global state vector with coupling marks and priority attention coefficients for input to the Actor network.

[0077] This process enables the coexistence of block independence and coupling scenarios, enabling the policy to "locally independently adjust" in general situations and "cooperatively optimize" when the adjacent area linkage is unbalanced. The label and weight mechanism allows the network to automatically identify and focus on the blocks that are urgently in need of adjustment and their mutual influences, effectively avoiding isolated decision-making for each area.

[0078] Step 4.3, Training of DDPG algorithm and cross-region interaction modeling.

[0079] Specifically, based on the fused global state vector (including coupling markers), the corresponding global action vector, and the global rewards and next global states in the replay pool obtained from the foregoing steps, an Actor-Critic architecture is selected. Among them, the Actor takes the fused state as the input and outputs continuous global actions (valve openings and fan frequencies of all blocks), and has a coupling perception layer for processing adjacent block interaction features; the Critic takes the state-action pair as the input and outputs the estimated reward value of this combination under the off-policy for evaluating the quality of actions.

[0080] In this embodiment, during the update process of the Critic, a small batch (such as 64) of samples is randomly selected to calculate the target reward (the estimated reward based on the next state and the target network), and the mean square loss of the difference between the current Critic output and the target reward is minimized. During the update process of the Actor, the gradient of the current policy output action by the Critic is used to adjust the Actor parameters to maximize the expected reward estimated by the Critic. An "adjacent convolution" or "self-attention" module is added after the input layer of the Actor network to perform local interaction learning on the concatenated state vectors of all blocks, enabling the network to automatically extract the coupling features between adjacent blocks. At the beginning of training, time-related noise (such as the Ornstein-Uhlenbeck process) is added to the Actor output to promote diverse exploration. At the same time, soft update (τ = 0.005) is used to synchronize the target network regularly to reduce the risk of training divergence.

[0081] In this embodiment, DDPG can handle high-dimensional continuous action spaces and directly output multi-block collaborative adjustment instructions. The separation of Actor-Critic and the coupling perception layer jointly ensure that the policy has strong coordination and generalization capabilities in cross-region coupling scenarios. Details such as exploration and soft update of the target network improve the stability of the training process and the robustness of the final policy.

[0082] Step 4.4, Offline verification, tuning, and model solidification.

[0083] Specifically, based on the multi-scenario simulation environment: Scenario A: A sudden increase in heat load of two-in-one adjacent blocks (coupling trigger); Scenario B: A single block has a high load, and the rest of the blocks have a low load (independent scenario); Scenario C: The entire area has a uniform medium load (conventional scenario). Run at least 2,000 simulation steps in each scenario, and periodically record the temperature and humidity difference, energy consumption load, and coupling changes of each block. Coupling scenario: measure whether the strategy synchronously increases the water flow and fan frequency in two adjacent blocks with a sudden increase in heat load without accidentally hurting other blocks; independent scenario: verify whether the strategy only adjusts the high-load block and maintains or fine-tunes the actions of other blocks; conventional scenario: test whether the overall energy consumption reduction rate and comfort deviation have reached the predetermined target (for example, comfort deviation <1°C, energy consumption reduction>10%). If the performance indicators are not met in any scenario, analyze the deviation between the Critic value estimate and the actual return, appropriately adjust the penalty weight or resample the specific scenario samples, and retrain. In addition, after all scenarios are passed, the final model weight file is output and a detailed verification report including scenario comparison curves, convergence trends and sample coverage is generated.

[0084] During this process, offline verification ensures that the strategy can respond correctly in both coupled and independent complementary scenarios, taking into account both local and overall optimality. The tuning mechanism enables the model to be continuously improved until it meets multi-dimensional deployable standards. Model solidification and reporting provide a reliable basis for on-site one-click deployment and subsequent maintenance.

[0085] Step S150, input the real-time performance index of the refrigeration area into the intelligent control strategy network for processing, and output the control instructions of the evaporative cooling air conditioning system.

[0086] In some embodiments, the intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning provided by the present invention, step S150 specifically includes the following steps: Step S151, obtaining the real-time performance indicators of each functional block in the refrigeration area, and screening the real-time performance indicators according to the health labels of the real-time performance indicators to eliminate and fill the sensor data whose health is lower than the fifth threshold, and obtain the quaternary performance value of each functional block.

[0087] Step S152, concatenate the quaternary performance values ​​of all functional blocks, and carry the global state vector and health weight of each functional block during concatenation to output a real-time global state vector.

[0088] Step S153, calling the intelligent control strategy network to process the real-time global state vector to generate recommended valve openings and fan frequencies for each functional block of the evaporative cooling air conditioning system, and outputting control instructions executed according to the recommended valve openings and fan frequencies.

[0089] Step S160, in response to the control instruction, control the valve opening degree and the fan frequency of each functional block.

[0090] In some embodiments, the intelligent control method of the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention further includes the following steps: Step S210, when a sensor device of any functional block fails, determine that the functional block is a faulty block, switch the faulty block to a preset PID redundant loop, and simultaneously send an alarm message to the operation and maintenance terminal.

[0091] Step S220, generate a global action vector based on the recommended valve opening degree and the fan frequency, and send the global action vector to the control center. The control center issues a control instruction to the PLC through the fieldbus.

[0092] In a specific embodiment, for the intelligent control method of the evaporative cooling air-conditioning system based on deep reinforcement learning provided by the present invention, step 5, online predictive control and fault emergency switching. Step 5 issues the optimal control action predicted by the policy network to respond to the fluctuations of the cooling and heating loads in advance. When it detects that the water pump stops rotating, the sensor fails, or the performance index exceeds the limit, it automatically triggers the preset PID redundant loop and alarms to ensure the basic cooling demand, realizes predictive closed-loop control and has the ability of fault tolerance and degradation, and ensures the stable operation of the system under sudden conditions. It includes the following steps: Step 5.1, obtain the real-time performance vector.

[0093] Specifically, based on the fused performance evaluation vector (quadruple: average temperature difference, humidity deviation, energy consumption load, comprehensive score) of each functional block at the current moment and the latest sensor health and weight information (used to verify the data reliability), each edge fusion node immediately uploads the quadruple performance evaluation of this block to the control center. The control center checks the health labels of each node, eliminates the data with a health lower than the threshold (such as 0.7), and fills it with the adjacent area or historical value. Finally, the quadruple performance values of all blocks are spliced in a predetermined order and an urgent adjustment or coupling trigger flag is added to form a global state vector. This process can ensure that the state data used for online control is the latest and reliable, provide a clear and comparable full-system observation for the subsequent policy network, and support cross-block collaborative optimization.

[0094] Step 5.2, generate the predictive control action.

[0095] Specifically, based on the aforementioned global state vector and the pre-trained Actor network, the control center inputs the global state vector into the pre-trained Actor network to output an initial action recommendation, that is, to recommend a global action vector, including the valve opening degrees and fan frequencies of each block. If any two adjacent blocks are simultaneously marked as "coupling trigger", then a coupling adjustment strategy is applied to their corresponding actions, that is, the fan frequencies of the two blocks are slightly synchronously increased or the valve opening degrees are coordinated and balanced to jointly relieve the high load; otherwise, the actions of each block are independently optimized to ensure local fast response. In addition, range constraints (such as the valve opening degree being in [0,1] and the fan frequency being in [10,60] Hz) and slope limitations (to avoid sudden changes) need to be imposed on each path of the recommended global action vector. This process realizes online predictive control through the policy network, responds to load fluctuations in advance, and the conditional judgment and coupling adjustment also ensure that when the cooperation between adjacent blocks is unbalanced, they can exert force simultaneously to avoid the performance deterioration of other blocks caused by the actions of a single block.

[0096] Step 5.3, Fault detection and emergency switching.

[0097] Specifically, based on the real-time sensor raw data, the fusion performance vector, and the device operation status alarm signals (such as the water pump flow dropping to zero and the abnormal wet curtain pressure difference), if it is detected that the water flow in a certain block drops to zero short-term or the pressure difference exceeds the maximum safety threshold (such as 5 kPa), it is determined that there is a fault in the waterway or sensor of this block. If the wet curtain inlet humidity sensor continuously reads out of range three times, it is determined that the measurement point fails. The faulty block is immediately switched to the pre-tuned "PID redundant loop", which will use fixed PID parameters to maintain the minimum supply air temperature and the lowest energy consumption. Other normal blocks continue to adopt the actions output by the deep policy network. At the same time, an operation and maintenance alarm is triggered, and the fault information is pushed to the operation and maintenance personnel through the SCADA system. This process can instantly restore to the controllable traditional PID control when key equipment or measurement points fail, ensure the basic cooling demand, isolate the non-faulty blocks from the faulty blocks, and avoid the impact of fault propagation on the overall system.

[0098] Step 5.4, Control command execution and closed-loop feedback.

[0099] Specifically, based on the finally selected global action vector (including predictive and emergency commands), the control center issues valve and fan commands to the PLC through the fieldbus (such as Modbus / TCP). The PLC performs rate limitation and anti-jitter processing on the commands and then outputs them to the frequency converter and servo valve. After execution, the actual execution values are immediately collected, compared with the command values, an error report is generated and returned to the edge fusion node. This process realizes the implementation of actions and visual monitoring, ensures that the commands are correctly executed, detects the command and execution deviation through closed-loop feedback, and provides real error data for the preprocessing and training of the next cycle.

[0100] Step 5.5, Online Logging and Policy Evaluation.

[0101] Specifically, write the global state vector, action, reward, and execution feedback of each cycle into the time series database, and automatically count the energy consumption, comfort deviation, and number of faults per hour / day to generate KPI Charts. If it is found that the policy's continuous reward drops severely under a certain working condition, immediately trigger the "online policy switching" or "re-training offline" process. This process can ensure the traceability and transparency of the policy's online operation, timely detect policy failures or system changes through real-time evaluation, and support online fine-tuning or trigger the next round of offline training.

[0102] The present disclosure may be a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0103] The computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in a groove having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0104] The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. The network adapter or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0105] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0106] Aspects of the present disclosure are described herein with reference to the flowchart and / or block diagram of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart and / or block diagram, and the combinations of blocks in the flowchart and / or block diagram, can be implemented by computer - readable program instructions.

[0107] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data - processing apparatus, result in an apparatus that implements the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0108] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0109] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, and the module, segment of code, or portion of an instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific implementation manners of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. An intelligent control method for an evaporative cooling air conditioning system based on deep reinforcement learning, characterized in that: The method comprises: Acquire spatial elements of a refrigeration area, and convert the spatial elements into a digital model, so as to generate a functional block division strategy of the refrigeration area based on the digital model; Based on the functional block division strategy, sensor data on the fusion nodes of each functional block are acquired, and the sensor data are preprocessed to obtain a multi-dimensional feature data set; fusing the multidimensional feature data set into a performance index to output a performance evaluation vector for each functional block; Based on the performance evaluation vector, an experience replay pool is constructed to fuse the global state vector to perform offline training on the DDPG algorithm to obtain an intelligent control strategy network; Inputting the real-time performance index of the refrigeration area into the intelligent control strategy network for processing, and outputting control instructions for the evaporative cooling air conditioning system; Controlling the valve opening and fan frequency of each functional block in response to the control instruction; The spatial elements include building floor plans, cooling load distribution and pipe network structures; the digital model includes a digitized spatial coordinate set, load mapping and pipe network topology; and the sensor data includes temperature, humidity, flow and pressure difference.

2. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 1 is characterized in that: The acquiring of spatial elements of the refrigeration area and converting the spatial elements into a digital model to generate a functional block division strategy for the refrigeration area based on the digital model includes: Based on the digital spatial coordinate set, load mapping and pipe network topology, the refrigeration area is divided into a plurality of functional blocks; A key node is selected from each functional block, and a sensor unit is set at the key node to generate the functional block division strategy; the sensor unit includes a temperature and humidity sensor, a flow meter and a differential pressure meter.

3. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 2 is characterized in that: Based on the functional block division strategy, the sensor data on each functional block fusion node is obtained, and the sensor data is preprocessed to obtain a multi-dimensional feature data set, including: Receiving sensor data from each sensor unit, and constructing an initial sequence matrix based on the sensor data according to a sampling period and a time sequence; Performing sliding average filtering on each column of sensor data in the initial sequence matrix to obtain a smoothed sequence matrix; The missing values ​​of the smooth sequence matrix are filled, and the filled smooth sequence matrix is ​​normalized by linear mapping to obtain the multidimensional feature data set.

4. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 3 is characterized in that: The step of fusing the multi-dimensional feature data set into a performance index to output a performance evaluation vector of each functional block includes: Performing time stamp alignment on the time series data of each functional block in the multi-dimensional feature data set through a network time protocol, and performing linear interpolation to fill the sensor data whose time error exceeds a first threshold; The health status of the sensor device is determined by the heartbeat detection frequency and residual statistics, and a health weight is assigned to each sensor data according to the health degree, so as to fill the sensor data whose health weight is lower than the second threshold value, and obtain health-weighted time series data; Performing sliding window median filtering on the health-weighted time series data, and performing weighted averaging on the sensor data at the same time according to the health weight, to obtain a fusion time series curve of each functional block; According to the set sliding window, the performance indicators of the fusion timing curve are counted; the performance indicators include cooling efficiency, temperature and humidity deviation, and energy load; The performance indicator whose sliding average abnormal jump exceeds the third threshold is marked as an abnormal indicator, and the abnormal indicator and the performance indicator are packaged to obtain the performance evaluation vector.

5. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 4 is characterized in that: The experience replay pool is constructed based on the performance evaluation vector to fuse the global state vector, so as to perform offline training on the DDPG algorithm to obtain an intelligent control strategy network, including: splicing the performance evaluation vectors of each functional block into a global state vector of the refrigeration area in time sequence; When the experience replay pool reaches an upper capacity limit, the global state vectors are sorted according to the TD error of the global state vector to retain the global state vectors whose TD error exceeds a fourth threshold, thereby obtaining a global state sample.

6. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 5 is characterized in that: The process of constructing an experience replay pool based on the performance evaluation vector to fuse the global state vector so as to perform offline training on the DDPG algorithm to obtain an intelligent control strategy network also includes: Determine whether the temperature difference or energy consumption of each functional block exceeds a preset high warning threshold; if so, mark the functional block as a first label; if not, mark the functional block as a second label; When a plurality of adjacent functional blocks are marked as the first label, coupling weights are added to the plurality of adjacent functional blocks, and the functional blocks carrying the coupling weights are spliced ​​with other functional blocks to obtain a fusion state vector; The fusion state vector is used as a DDPG algorithm with an Actor-Critic framework for offline training to obtain the intelligent control strategy network.

7. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 6 is characterized in that: The real-time performance index of the refrigeration area is input into the intelligent control strategy network for processing, and the control instruction of the evaporative cooling air conditioning system is output, including: Acquire the real-time performance indicators of each functional block of the refrigeration area, and filter the real-time performance indicators according to the health labels of the real-time performance indicators to remove and fill the sensor data whose health is lower than the fifth threshold, so as to obtain the quaternary performance value of each functional block; The quaternary performance values ​​of all functional blocks are spliced, and the global state vector and health weight of each functional block are carried during the splicing to output a real-time global state vector; The intelligent control strategy network is called to process the real-time global state vector to generate recommended valve openings and fan frequencies for each functional block of the evaporative cooling air-conditioning system, and output control instructions executed according to the recommended valve openings and fan frequencies.

8. The intelligent control method of the evaporative cooling air conditioning system based on deep reinforcement learning according to claim 7 is characterized in that: The method further comprises: When a sensor device of any functional block fails, the functional block is determined to be a faulty block, the faulty block is switched to a preset PID redundant loop, and an alarm message is sent to the operation and maintenance terminal; A global action vector is generated based on the recommended valve opening and fan frequency, and the global action vector is sent to a control center, and the control center sends the control instruction to the PLC through a field bus.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images