Roadway local cooling control method and system based on deep reinforcement learning

Abstract

The invention relates to the technical field of intelligent control, in particular to a roadway local cooling control method and system based on deep reinforcement learning. The method comprises the following steps: acquiring historical data of a temperature field, a wind speed field, humidity and energy consumption in a roadway environment; preprocessing the historical data to obtain standardized data; based on the standardized data, constructing an optimization prediction model of the deep learning temperature field; designing a reinforcement learning control environment based on the optimization prediction model; constructing a reinforcement learning strategy network based on the reinforcement learning control environment; and on the basis of the reinforcement learning strategy network, real-time optimization control of the local temperature of the roadway is realized. According to the system, the double axial flow fans, the air deflectors and the cold wall system are cooperatively arranged, so that the temperature in the complex environment is dynamically regulated and controlled. According to the method, the temperature control precision is effectively improved, the system energy consumption is reduced, the control response time is shortened, and a reliable solution is provided for underground high-temperature treatment.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a tunnel local cooling control method and system based on deep reinforcement learning. Background Art

[0002] With the in-depth development of mining and tunnel engineering, the high temperature environment caused by geothermal gradient has become a key factor restricting safe production. The local cooling control in deep tunnels faces severe challenges due to their narrow and long space, complex heat sources, uneven temperature distribution and difficult ventilation.

[0003] In the existing technology, fixed-point air supply, segmented refrigeration and full-section ventilation technologies are mainly used to cool the tunnel. Although the fixed-point air supply technology is simple to operate, there are dead corners for air supply, and it is difficult to achieve uniform temperature field control; segmented refrigeration technology expands the refrigeration range through multi-point arrangement, but the system has high energy consumption and is prone to interference between refrigeration points; full-section ventilation technology has a wide coverage range, but requires high-power equipment, serious energy waste, and it is difficult to meet the precise temperature control needs of local areas. With the development of intelligent control technology, deep learning methods have gradually been applied in temperature field prediction and control. However, this method still has some problems: first, it is difficult for deep neural networks to accurately characterize the dynamic characteristics of complex temperature fields; second, the reinforcement learning strategy converges slowly and is prone to fall into local optimality; third, the lack of physical model constraints makes the prediction results violate the actual physical laws; fourth, the separation of prediction and control leads to delayed system response and low energy efficiency. In addition, the combination of deep learning and traditional control theory mainly remains at the simple combination level. When dealing with highly nonlinear and strongly coupled tunnel temperature field problems, it is difficult to take into account the control accuracy goals, system energy efficiency goals and dynamic adaptability goals.

[0004] At present, there is not much research on local cooling control in deep tunnels, and there is no specific temperature field control method that combines deep learning, reinforcement learning and model predictive control. Summary of the invention

[0005] In view of the defects in the prior art, the present invention provides a tunnel local cooling control method and system based on deep reinforcement learning.

[0006] In the first aspect, the present invention provides a local cooling control method for a lane based on deep reinforcement learning, comprising the following steps: obtaining historical data of temperature field, wind speed field, humidity and energy consumption in the lane environment; preprocessing the historical data to obtain standardized data; constructing an optimized prediction model of deep learning temperature field based on the standardized data; designing a reinforcement learning control environment based on the optimized prediction model; constructing a reinforcement learning strategy network based on the reinforcement learning control environment; and realizing real-time optimization control of the local temperature of the lane based on the reinforcement learning strategy network. The present invention provides a solid foundation for subsequent accurate prediction and control by collecting historical data of temperature field, wind speed field, humidity and energy consumption in the lane; by preprocessing the historical data, ensuring the standardization and consistency of the data, and improving the prediction accuracy of the model; by using the deep learning method, constructing an optimized prediction model of the temperature field, efficiently capturing the complex laws of environmental changes; by optimizing the prediction model, designing a reinforcement learning control environment, providing a simulation and verification platform for the formulation of intelligent control strategies; by constructing a reinforcement learning strategy network, responding to changes in the lane environment in real time, automatically adjusting the cooling strategy, and realizing accurate and efficient control of the local temperature of the lane.

[0007] Optionally, the preprocessing of the historical data to obtain standardized data includes: performing data cleaning and outlier processing on the historical data to obtain optimized data; and performing standardization processing on the optimized data to obtain standardized data. The present invention effectively eliminates erroneous, duplicated or unreasonable data points through data cleaning and outlier processing, ensures the quality and reliability of the data, and provides a solid foundation for subsequent modeling and control; by standardizing the optimized data, the dimensional differences between different variables are eliminated, which not only helps to improve the training efficiency and prediction accuracy of the model, but also makes the model more robust and easy to explain.

[0008] Optionally, the method of constructing an optimized prediction model of a deep learning temperature field based on the standardized data includes: constructing a deep neural network structure based on the standardized data; designing an input layer of temperature field and wind speed field data based on the deep neural network structure; designing a feature extraction layer and a time series prediction layer based on the input layer; designing a loss function with physical constraints based on the feature extraction layer and the time series prediction layer; determining model training parameters and optimization methods based on the loss function with physical constraints; and constructing an optimized prediction model of a deep learning temperature field through the model training parameters and the optimization method. The present invention fully captures the spatiotemporal characteristics of the temperature field and the wind speed field by designing a deep neural network structure, thereby improving the prediction ability of the model; ensures that the temperature field and wind speed field data are accurately and efficiently input into the model by designing the input layer; deeply explores the potential laws and trends of the data by designing the feature extraction layer and the time series prediction layer; ensures that the prediction results of the model conform to the physical laws by introducing the loss function with physical constraints; and constructs an optimized prediction model of a deep learning temperature field by determining the model training parameters and the optimization method, thereby providing strong support for the real-time optimization control of the local temperature of the lane.

[0009] Optionally, the design of the reinforcement learning control environment based on the optimization prediction model includes: according to the optimization prediction model, designing the environment state space, action space and reward function, the environment state space includes temperature field distribution, wind speed field distribution, fan speed and wind deflector angle, the action space includes the adjustment range of fan speed and the adjustment range of wind deflector angle, and the reward function includes target temperature and current energy consumption. The present invention covers the temperature field distribution, wind speed field distribution, fan speed and wind deflector angle information by designing the environment state space, providing comprehensive environmental perception for the reinforcement learning algorithm; by designing the action space, the adjustment range of fan speed and wind deflector angle is clarified, providing an effective control means for the temperature control system; by designing the reward function, the target temperature and current energy consumption are comprehensively considered, ensuring that the temperature control system can take into account energy efficiency while pursuing the temperature control effect.

[0010] Optionally, the construction of a reinforcement learning strategy network based on the reinforcement learning control environment includes: designing a network structure and loss function of deep reinforcement learning based on the reinforcement learning control environment; constructing a reinforcement learning strategy network based on the network structure and loss function of deep reinforcement learning. The present invention designs a network structure of deep reinforcement learning based on the characteristics of the reinforcement learning control environment, efficiently handles complex environmental states and action spaces, and provides a powerful decision-making capability for the temperature control system; by designing a loss function that matches it, it ensures that the strategy network continuously approaches the optimal strategy during the training process; by constructing a reinforcement learning strategy network, the fan speed and the angle of the air guide plate are intelligently adjusted according to the real-time environmental state to achieve precise control of the tunnel temperature field.

[0011] Optionally, the network structure and loss function based on the deep reinforcement learning to construct a reinforcement learning strategy network includes: based on the network structure and loss function of the deep reinforcement learning, using the experience replay pool, executing strategy training iterations to obtain iterative results; based on the iterative results, evaluating strategy performance indicators, and constructing a reinforcement learning strategy network, wherein the strategy performance indicators include target area temperature control accuracy, temperature field distribution uniformity, wind speed field uniformity, system comprehensive energy consumption, and control response time. The present invention uses the experience replay pool to perform strategy training iterations, effectively utilizes historical data, improves training efficiency and strategy stability, and obtains accurate iterative results; comprehensively evaluates strategy performance indicators, including target area temperature control accuracy, temperature field distribution uniformity, wind speed field uniformity, system comprehensive energy consumption, and control response time, to ensure the excellent performance of the strategy in practical applications; and constructs a reinforcement learning strategy network to intelligently adjust the fan speed and wind guide plate angle to achieve accurate and efficient control of the tunnel temperature field.

[0012] Optionally, the real-time optimization control of the local temperature of the tunnel based on the reinforcement learning strategy network includes: based on the reinforcement learning strategy network, constructing a prediction model that considers the dynamic characteristics of the temperature field, the prediction model sets the prediction time domain and the control time domain; based on the prediction model, designing an optimization objective function including temperature control accuracy and energy efficiency; through the optimization objective function and constraints, generating a dynamic optimal control strategy; according to the optimal control strategy, realizing real-time optimization control of the local temperature of the tunnel. The present invention provides a reference in the time dimension for real-time optimization by constructing a prediction model that considers the dynamic characteristics of the temperature field and sets the prediction time domain and the control time domain; by designing an optimization objective function including temperature control accuracy and energy efficiency, it ensures that the optimization process pursues both high-precision temperature control and energy efficiency improvement; by optimizing the objective function and constraints, a dynamic optimal control strategy is generated, and real-time optimization control of the local temperature of the tunnel is realized accordingly.

[0013] In the second aspect, the present invention provides a tunnel local cooling control system based on deep reinforcement learning, including an input device, a processor, an output device and a memory; the input device includes a data acquisition module; the processor includes a model prediction module, a decision control module and an execution control module; the input device, the processor, the output device and the memory are interconnected; the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions. The present invention adopts a modular design so that each module can be independently developed, upgraded and maintained, thereby improving the overall flexibility and scalability of the system; by integrating the model prediction module, the decision control module and the execution control module, the data collected by the input device is efficiently processed, and accurate decisions are made according to the preset algorithms and logics, ensuring that the system can respond quickly and achieve precise control when facing complex environments; through the programmed control method, the behavior of the system can be predicted and repeated, thereby enhancing the stability and reliability of the system.

[0014] Optionally, the sampling frequency of the data acquisition module is adjustable, and it has data anomaly detection and automatic calibration functions, including a temperature sensor array and a wind speed sensor array. The temperature sensor array is arranged in a gridded form at key positions in the lane, and the wind speed sensor array is arranged in key sections of the airflow channel; the model prediction module has online learning and model update functions, including a deep learning prediction unit and a reinforcement learning strategy unit, which are used to predict temperature field distribution and generate control strategies; the decision control module has multi-objective optimization and real-time response capabilities, including a model prediction control unit, which is used to optimize control strategies and generate control instructions; the execution control module has fault diagnosis and automatic protection functions, including a dual axial flow fan unit, an adjustable air guide plate group and a cold wall system. The dual axial flow fan unit and the air guide plate are arranged in a coordinated manner, and the adjustable air guide plate group includes an independently adjustable air guide plate unit, and the cold wall system provides a stable cold source output. The acquisition module of the present invention ensures the accuracy and reliability of sensor data through flexible sampling frequency, anomaly detection and automatic calibration functions, providing a solid foundation for subsequent temperature field prediction and control; the model prediction module combines the deep learning prediction unit and the reinforcement learning strategy unit to achieve accurate prediction of the temperature field and generation of intelligent control strategies, improving the prediction ability and control accuracy of the system; the decision control module and the execution control module ensure the accuracy of control instructions and the safety of the execution process through multi-objective optimization and real-time response, as well as fault diagnosis and automatic protection functions, and achieve accurate, efficient and safe control of the local temperature of the tunnel.

[0015] Optionally, the dual-axial flow fan unit includes a main blower, an auxiliary blower, a frequency conversion control unit, an angle adjustment mechanism and a position adjustment mechanism; the main blower is used to provide the main airflow; the auxiliary blower is used to adjust the airflow direction; the frequency conversion control unit is used to adjust the fan speed; the angle adjustment mechanism is used to adjust the angle within the range of 0° to 90°; the position adjustment mechanism is used to adjust the position of the air guide plate. The main blower of the present invention provides the main airflow, ensuring the basic demand for air circulation in the lane; the auxiliary blower further optimizes the air flow path and improves the ventilation efficiency by adjusting the airflow direction; the frequency conversion control unit adjusts the fan speed according to actual needs, realizing the refined management of energy consumption, which not only meets the ventilation needs but also reduces energy consumption; the angle adjustment mechanism and the position adjustment mechanism work together to accurately control the airflow distribution and ensure uniform temperature in various areas of the lane. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of a tunnel local temperature reduction control method based on deep reinforcement learning according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the overall structure of a tunnel local cooling control system based on deep reinforcement learning according to an embodiment of the present invention; Figure 3 It is a schematic diagram of the main structure of a tunnel local cooling control system based on deep reinforcement learning according to an embodiment of the present invention; Figure 4 A schematic diagram of the wind guide flow direction according to an embodiment of the present invention; Figure 5 This is an operation flow chart of a tunnel local cooling control system based on deep reinforcement learning according to an embodiment of the present invention; Figure 6 Schematic diagram of the system timing temperature field structure of an embodiment of the present invention.

[0017] Description of Reference Numerals 1 is the axial flow fan unit; 2 is the air guide plate unit; 3 is the cold arm; 4 is the monitoring point; 5 is the palm face. DETAILED DESCRIPTION

[0018] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details do not need to be adopted to implement the present invention. In other examples, in order to avoid confusing the present invention, known circuits, software or methods are not specifically described.

[0019] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. In addition, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and / or subcombination. In addition, it should be understood by those of ordinary skill in the art that the figures provided herein are for illustrative purposes and that the figures are not necessarily drawn to scale.

[0020] See also Figure 1 , an embodiment of the present invention provides a tunnel local temperature reduction control method based on deep reinforcement learning, the method comprising the following steps: S1. Obtain historical data of temperature field, wind speed field, humidity and energy consumption in the tunnel environment.

[0021] Among them, S1 includes the following steps: S11. Define system parameters.

[0022] In one embodiment, the operating range of the fan is clarified, including the upper and lower limits of the adjustable speed, the rated air volume and the corresponding wind pressure range, and the layout, quantity and spacing of the fan are reasonably planned to ensure the airflow coverage of the target area. For the wind guide plate, the adjustment range of the deflection angle is defined from zero to ninety degrees, and the kinetic energy recovery coefficient and friction coefficient required for the wind flow distribution are set in combination with the material. In terms of tunnel parameters, the cross-sectional dimensions, surface roughness and ventilation distance are clarified to provide support for the calculation of resistance along the way and the prediction of wind flow attenuation. These parameters lay the foundation for model construction and system optimization.

[0023] S12. Perform data acquisition configuration based on the system parameters.

[0024] Specifically, the data acquisition configuration includes temperature sensor arrangement, humidity sensor arrangement, wind speed sensor arrangement and sampling frequency setting.

[0025] The temperature sensor arrangement method is as follows: According to the temperature control requirements in the lane, select the appropriate sensor type and quantity to ensure coverage of key areas, including the fan outlet, the front and rear of the air guide plate, and the target cooling area. At the same time, set the accuracy and range of the sensor to meet the monitoring requirements of actual temperature changes. The layout follows the principle of reasonable distribution: densely arrange sensors in areas with large temperature differences, and appropriately reduce the number of sensors in areas with gentle temperature changes, so as to take into account both monitoring accuracy and cost efficiency.

[0026] The wind speed sensor arrangement method is as follows: focus on covering the fan outlet, key positions of the wind deflector and the target cooling area to capture the dynamic changes in wind speed distribution. According to the characteristics of the tunnel wind flow, determine the measurement range and accuracy of the sensor to ensure that it can accurately reflect the fluctuation and distribution of wind speed. Reasonable distribution of sensor positions must ensure the comprehensiveness of the data while taking into account the complexity of the tunnel structure and actual needs.

[0027] The humidity sensor arrangement method is as follows: the humidity sensor is mainly arranged in a small area near the fan to monitor the change of humidity around the fan in real time, providing a basis for the subsequent acquisition of energy consumption data.

[0028] The sampling frequency setting method is as follows: the temperature sensor, humidity sensor and wind speed sensor are synchronously sampled once per second (1 Hz) or higher to ensure the dynamic consistency of the data and provide real-time environmental feedback for the control system. At the same time, the data storage path should be configured to support the transmission of sensor data to the processing center via wireless or wired methods to achieve real-time monitoring and data analysis functions.

[0029] S13. Based on the data acquisition configuration, historical data of temperature field, wind speed field, humidity field and energy consumption in the tunnel environment are obtained.

[0030] In one embodiment, by arranging temperature sensors, wind speed sensors and humidity sensors, historical data of temperature field, wind speed field and humidity in the tunnel environment are directly acquired.

[0031] Furthermore, the historical data of the temperature field, the wind speed field and the humidity are substituted into the energy consumption calculation expression of the fan system to obtain the historical data of energy consumption.

[0032] S2. Preprocess the historical data to obtain standardized data.

[0033] Among them, S2 includes the following steps: S21. Perform data cleaning and outlier processing on the historical data to obtain processing results.

[0034] In one embodiment, data cleaning is performed on the historical data.

[0035] Specifically, the collected data is imported into a data cleaning tool or programming environment, such as Python's Pandas library.

[0036] Furthermore, check the missing values ​​in the data. For records with fewer missing values, consider using linear interpolation or mean filling. If there are many missing values, you need to delete these records or collect data again.

[0037] Furthermore, check for duplicate values ​​in the data. For identical records, only one is retained. If the records are similar but not identical, further analysis is required to determine whether to retain them.

[0038] Furthermore, ensure that all data is converted to a uniform format and unit. For example, temperature is converted to degrees Celsius, wind speed is converted to meters per second, humidity is converted to percentage, and energy consumption is converted to kilowatt-hours.

[0039] In another embodiment, outlier processing is performed on the historical data that has undergone data cleaning.

[0040] Specifically, use visualization tools such as box plots and scatter plots to visually identify outliers. Box plots can show the distribution and median of the data, as well as the range between the first and third quartiles. Scatter plots can show the relationship between variables and help identify possible outliers.

[0041] Furthermore, the physical meaning of the tunnel environment and common sense are combined to judge abnormal values. For example, if a temperature value is much lower or much higher than other records and there is no reasonable explanation, this value is judged as an abnormal value.

[0042] Furthermore, the identified outliers are replaced, deleted or marked.

[0043] It should be noted that the processing of historical data of temperature field includes: dividing the temperature field distribution data in the tunnel into regions, extracting the temperature characteristics of key regions, ensuring balanced data distribution, and avoiding excessive or insufficient data in a certain region. The processing of historical data of wind speed field includes: extracting the characteristics of wind speed distribution in the tunnel, capturing cross-sectional wind speed gradient data, smoothing wind speed data, and performing noise removal.

[0044] S22. Standardize the processing results to obtain standardized data.

[0045] In one embodiment, the preprocessed historical data in step S21 is standardized to eliminate data dimension differences and improve the stability and convergence speed of model training.

[0046] Furthermore, the data distribution is adjusted to standardized data with a mean of 0 and a standard deviation of 1.

[0047] S3. Based on the standardized data, a deep learning optimization prediction model is constructed.

[0048] Among them, S3 includes the following steps: S31. Construct a deep neural network structure based on the standardized data.

[0049] Wherein, S31 further comprises the following steps: S311. Based on the standardized data, a physical model of ventilation and cooling is established.

[0050] In one embodiment, based on the standardized data and in combination with the parameters of the fan system, a physical model of ventilation and cooling is established, which includes a calculation model of ventilation resistance, a calculation model of wind speed distribution, and a temperature field prediction model.

[0051] Specifically, the expression of ventilation resistance is as follows:

[0052] in, is the pressure drop caused by the resistance along the way, is the resistance coefficient along the way, is the air density, is the wind speed.

[0053] Furthermore, the expression of the fan outlet flow rate is as follows:

[0054] in, is the fan outlet volume flow rate, is the fan outlet wind speed, is the fan outlet cross-sectional area.

[0055] Furthermore, the expression of wind pressure is as follows:

[0056] in, is the dynamic pressure at the fan outlet, is the air density, is the fan outlet wind speed.

[0057] Furthermore, the decomposition expressions of the wind deflector reflection and incident wind speed are as follows:

[0058]

[0059] in, is the component of the incident wind speed in the normal direction (perpendicular to the wind deflector), is the component of the incident wind speed in the tangential direction (parallel to the wind deflector), It is the angle between the air guide plate and the fan airflow.

[0060] Furthermore, the reflected wind speed component satisfies the following expression:

[0061] in, is the normal component of the reflected wind speed, is the coefficient of restitution.

[0062] Furthermore, the expression of the tangential component is as follows:

[0063] in, is the tangential component of the reflected wind speed, is the friction coefficient.

[0064] Furthermore, the wind speed and direction are combined to obtain the horizontal component and the vertical component, which are expressed as follows:

[0065]

[0066] in, , is the horizontal component of the combined wind speed, , are the velocity components of the two wind flows in the horizontal direction, , are the velocity components of the two wind flows in the vertical direction, , is the mass flow rate of the two wind flows.

[0067] Furthermore, the combined wind speed expression is as follows:

[0068] in, is the total wind speed after merging.

[0069] Furthermore, the combined direction angle satisfies the following expression:

[0070] in, is the wind direction angle after the merger.

[0071] Furthermore, the temperature field distribution formula is as follows:

[0072] in, is the temperature potential field, is the time variable, is the velocity field vector, is the diffusion term, is the thermal diffusivity, is the heat source sink function.

[0073] Furthermore, the thermal diffusivity satisfies the following relationship:

[0074] in, is the thermal conductivity, is the fluid density, is the specific heat at constant volume.

[0075] S312. Determine performance indicators based on the physical model of ventilation and cooling.

[0076] In one embodiment, according to the method design objectives and actual needs, the core performance indicators of the wind flow optimization and cooling system are defined to evaluate the cooling effect and optimize the space. These indicators mainly include cooling effect, wind speed uniformity and system energy consumption.

[0077] Specifically, the temperature control deviation is the core indicator for evaluating the cooling effect, and the temperature control deviation satisfies the following expression:

[0078] in, is the temperature control deviation index, is the total number of monitoring points, For the The actual temperature of each monitoring point, To predict temperature.

[0079] Furthermore, the wind speed distribution standard deviation is a core indicator for evaluating wind speed uniformity, and the wind speed distribution standard deviation satisfies the following expression:

[0080] in, is the standard deviation of wind speed distribution, For the The wind speed at each monitoring point, is the average wind speed in the section.

[0081] Furthermore, the wind speed direction is combined to form consistency, and the corresponding indicator expression is as follows:

[0082] in, is the wind speed direction consistency index, After the merger The wind flow direction angle, is the target wind direction angle.

[0083] Furthermore, the fan energy consumption satisfies the following expression:

[0084] in, is the total energy consumption of the fan, is the air volume, is the static pressure power consumption of the fan, is the fan efficiency, is the initial time, The end time of the operation.

[0085] Furthermore, the energy consumption of the cooling system satisfies the following expression:

[0086] in, is the total energy consumption of the cooling system, is the cooling power per unit time.

[0087] Furthermore, the system energy efficiency ratio is as follows:

[0088] in, is the cooling capacity of the system; is the total energy consumption of the system, satisfying the following expression:

[0089] S32. Based on the deep neural network structure, design the input layer of temperature field and wind speed field data.

[0090] In one embodiment, based on the deep neural network structure, an input layer of temperature field and wind speed field data is designed. The design of the input layer is centered on the key features of the tunnel environment, and uses control variables such as temperature field (two-dimensional or three-dimensional matrix distribution), wind speed field (horizontal and vertical components), and fan parameters and wind deflector status. The standardized data is organized and structured into a form that can be directly processed by the network to provide support for subsequent feature extraction and dynamic prediction.

[0091] S33. Based on the input layer, a feature extraction layer and a time series prediction layer are designed to obtain a deep learning prediction model for temperature field and wind speed field.

[0092] In one embodiment, based on the input layer, the hidden layer is designed through convolution operation and time series processing, wherein the feature extraction layer of the hidden layer is obtained through convolution operation, and the time series prediction layer of the hidden layer is obtained through time series processing.

[0093] Specifically, the convolution operation is implemented by a convolution model, and the convolution model satisfies the following expression:

[0094] in, is the output feature after the convolution operation, is the input feature, is the convolution kernel weight, is the bias term, , , , is the feature sequence before the convolution operation, , are two different maximum characteristic sequences, , is the sequence corresponding to the input feature, The time series processing is implemented by a time series model, and the time series model satisfies the following expression:

[0095] in, is the hidden state at the current time step, is the output gate, is the hyperbolic tangent activation function, is the memory unit state at the current time step, and its update mechanism satisfies the following expression:

[0096] in, is the memory cell state at the current time step, For the Gate of Forgetfulness, is the memory cell state at the previous time step, is the input gate, is the hyperbolic tangent activation function, is the weight matrix, is the hidden state of the previous time step, is the input of the current time step, is the bias term. Through time series processing, the time series prediction layer is obtained.

[0097] Furthermore, based on the designed input layer and hidden layer, combined with other layer structures of the neural network structure, a deep learning prediction model for temperature field and wind speed field is jointly constructed.

[0098] S34. Train the prediction model to obtain an optimized prediction model.

[0099] In one embodiment, the deep learning prediction model of temperature field and wind speed field is trained, wherein the implementation steps of the training include: training data set division, learning parameter setting, model training process and training effect evaluation.

[0100] Specifically, the training data set division includes: dividing the processed temperature field and wind speed field data sets into training set, validation set and test set. Among them, the training set is used for model parameter optimization, accounting for 70% to 80% of the total data; the validation set is used for evaluating the performance of the model during the training process, accounting for 10% to 15% of the total data; the test set is used for evaluating the final model performance, accounting for 10% to 15% of the total data. The data is divided by random sampling, and at the same time, it is ensured that there are no cross samples between different data sets to ensure the accuracy of the evaluation of the generalization ability of the model. In order to avoid model deviations caused by uneven data distribution, the data is evenly sampled according to the regional characteristics of the temperature field and wind speed field, so that the data ratio of each sub-region is consistent, thereby improving the adaptability of the model to different regional environments.

[0101] The learning parameter setting includes: using a custom loss function in the training process to combine the prediction error with the physical consistency constraint, and the loss function is in the following form:

[0103] in, is the loss function, is the mean square error between the model prediction value and the true value, To adjust the weight, is the momentum conservation equation for the thermal diffusion law of the temperature field, which is defined as:

[0104] in, is the time rate of change of temperature, is the fluid velocity vector, is the gradient of the temperature field, is the thermal diffusion coefficient, is the Laplace operator of the temperature field, is the heat source term, is the square of the two norm.

[0105] Furthermore, the Adam optimizer is used to optimize the algorithm, and the parameter update is expressed as:

[0106] in, For time The temperature value at the moment, For time The temperature value at the moment, is the learning rate, is the first-order momentum, is the second-order momentum, To prevent small values ​​of the denominator being zero.

[0107] The model training process includes the following three key steps: forward propagation, error calculation and back propagation. In the forward propagation stage, the training data passes through the input layer, hidden layer and output layer in turn to generate a predicted value. Subsequently, based on the designed loss function, the error between the predicted value and the true value is calculated. Next, the weights are updated by calculating the gradient of the loss to the model parameters, and the optimization algorithm is used for multiple rounds of iterations to gradually reduce the training error, thereby achieving back propagation. In order to accelerate convergence and avoid overfitting, the learning rate is dynamically adjusted, and the model performance is monitored on the validation set at the same time, the early stopping mechanism is triggered, and the optimal model parameters are saved. After training is completed, the model prediction accuracy and physical consistency are evaluated through the test set to ensure the generalization ability and reliability of the model in actual scenarios.

[0108] The training effect evaluation is completed through the verification data set and the test data set. After each complete training of a data set, the verification data is input into the model, the verification error is calculated to evaluate the generalization ability of the model, and the training status of the model is judged according to the changing trend of the verification error. In order to avoid overfitting of the model, an early stopping mechanism is adopted. When the verification error no longer decreases after multiple consecutive complete trainings, the training is stopped and the current optimal model parameters are saved. After the training is completed, the final performance of the model is evaluated using the test data set, and the prediction error and physical consistency index are calculated to verify the prediction accuracy of the model in the actual application scenario and the degree of compliance with the physical laws, so as to ensure its reliability and applicability in practical applications. When the training effect is the best, the optimized prediction model is obtained.

[0109] It should be noted that in the optimization process of the prediction model, the performance of the model is determined by the size of the relative error, which satisfies the following expression:

[0110] in, is the relative error, is the actual value, is the predicted value.

[0111] Furthermore, in the output layer, the predicted values ​​of the temperature field and wind speed field are used to correct the difference with the measured values ​​of the sensor to achieve real-time optimization of the prediction results. The expression of the difference correction is as follows:

[0112] in, is the corrected temperature, is the initial temperature predicted by the model, is the actual temperature measured by the sensor, is the current temperature predicted by the model, is the corrected wind speed vector, is the initial wind speed vector predicted by the model, is the current wind speed vector predicted by the model, is the true wind speed vector measured by the sensor.

[0113] Furthermore, the fan speed and the wind deflector angle are adjusted, and the expression of the adjustment is as follows:

[0114] in, is the predicted value of wind speed field, is the target value of the speed field, To adjust the fan speed, is the current fan speed.

[0115] Furthermore, the angle of the air guide plate is adjusted, and the expression of the adjustment is as follows:

[0116] in, The angle deviation of the wind deflector needs to be adjusted. , is the wind speed vector in the horizontal direction , The weight, is the prediction angle, is the target angle.

[0117] S4. Based on the optimization prediction model, a reinforcement learning control environment is designed.

[0118] In one embodiment, the design of the reinforcement learning control environment is achieved by designing the state space, action space and reward function.

[0119] Specifically, in the process of designing the state space, the state vector includes the current temperature field distribution, wind speed field distribution, fan speed and wind deflector angle, and the specific form is as follows:

[0120] in, is the state vector, is the temperature field distribution, is the wind speed field distribution, is the current speed of the fan, is the current angle of the air deflector.

[0121] Furthermore, in the process of designing the action space, the action vector includes the adjustment range of the fan speed and the adjustment range of the wind deflector angle, and the specific form is as follows:

[0122] in, is the action vector, is the change of fan speed, is the change of the angle of the wind deflector.

[0123] Furthermore, in the process of designing the reward function, the reward function takes temperature control effect and energy consumption optimization as the goal, and the specific form is as follows:

[0124] in, is the reward function, , is the weight coefficient, is the current energy consumption, is the target temperature, is the current temperature.

[0125] S5. Based on the reinforcement learning control environment, a reinforcement learning strategy network is constructed.

[0126] Among them, S5 includes the following steps: S51. Based on the reinforcement learning control environment, design the network structure and loss function of deep reinforcement learning.

[0127] In one embodiment, after the reinforcement learning control environment has been completed, the reinforcement learning policy network is constructed, including constructing a network structure and a loss function.

[0128] Specifically, the network structure uses a deep reinforcement learning network, including an input layer, a hidden layer, and an output layer. The loss function of the policy network is as follows:

[0129] in, is the loss function of the policy network, Status distributed The expectation under sampling, For the current strategy, is the time step status, is the time step Actions, is the state distribution, is the action value function.

[0130] S52. Based on the network structure and loss function of the deep reinforcement learning, a reinforcement learning strategy network is constructed.

[0131] Wherein, S52 further comprises the following steps: S521. Based on the network structure and loss function of the deep reinforcement learning, using the experience replay pool, perform strategy training iterations to obtain iterative results; In one embodiment, the experience replay and policy update methods are used to train and optimize the reinforcement learning policy network.

[0132] Specifically, the experience replay method includes: using the experience replay pool to store historical quadruplets, and randomly sampling for training. The strategy update method adopts a strategy update model, and the strategy update model satisfies the following expression:

[0133] in, is the policy gradient, is the expected value of the subsequent random variable, is the policy objective function, For parameterized strategies, is the gradient of the policy probability, is the action value function.

[0134] S522. Based on the iteration results, the strategy performance indicators are evaluated and a reinforcement learning strategy network is constructed. The strategy performance indicators include the temperature control accuracy of the target area, the uniformity of the temperature field distribution, the uniformity of the wind speed field, the comprehensive energy consumption of the system and the control response time.

[0135] In one embodiment, the effect of the reinforcement learning strategy is evaluated, mainly from the following aspects: first, the temperature control effect is evaluated by the deviation between the target temperature and the actual temperature to measure the temperature control accuracy of the system; second, the performance of the strategy in energy consumption optimization is evaluated by calculating the total energy consumption of the fan and the cold source system; finally, the convergence and stability of the strategy network during the training process are observed to judge its training effect and availability. In terms of evaluation methods, the reinforcement learning strategy is verified through simulation experiments before actual application. After online deployment, the actual operation data is combined to test the adaptability and stability of the strategy under different working conditions to ensure its reliability and practicality in real scenarios. The strategy performance indicators involved include the temperature control accuracy of the target area, the uniformity of the temperature field distribution, the uniformity of the wind speed field, the comprehensive energy consumption of the system, and the control response time. When the evaluation result of the strategy network is good, the strategy network is determined to be a reinforcement learning strategy network.

[0136] S6. Based on the reinforcement learning strategy network, the real-time optimization control of the local temperature of the tunnel is realized.

[0137] Wherein, S6 further comprises the following steps: S61. Based on the evaluation results, a prediction model that takes into account the dynamic characteristics of the temperature field is constructed.

[0138] In one embodiment, based on obtaining better results by evaluating the reinforcement learning strategy, a system prediction model considering the dynamic characteristics of the temperature field is constructed. The construction process of the system prediction model includes the definition of the system dynamic model, the design of the disturbance model, and the setting of the prediction time domain and the control time domain.

[0139] Specifically, the definition of the system dynamic model is mainly in the form of state space equation, which is used to describe the law of change of the system state. Its specific form can be expressed as:

[0140]

[0141] in, is the input variable, is the state variable, is the output variable, is the control variable, is the disturbance variable, , , , is the system parameter matrix, For time.

[0142] Furthermore, the purpose of the design of the disturbance model is to describe the impact of the external environment on the system. The disturbance model satisfies the following expression:

[0143] in, is the disturbance variable, is the perturbation function, For external environment variables.

[0144] Furthermore, the purpose of setting the prediction time domain and the control time domain is to enable the system to take into account system performance, real-time performance and computational efficiency, make control decisions more accurate, ensure the robustness and adaptability of the system in complex dynamic environments, and provide reliable basic predictions for temperature control and energy consumption optimization of the tunnel.

[0145] Specifically, the specific form of the prediction time domain is as follows:

[0146] in, For the predicted state, is the state transfer function, is the time variation.

[0147] The specific form of the control time domain is as follows:

[0148] in, For the current moment The optimal control input is is the objective function. is the number of input features, Indicates the parameter to be minimized.

[0149] S62. Based on the prediction model, construct an optimization problem.

[0150] In one embodiment, an optimization problem is constructed by designing an objective function, defining constraints, and optimizing variables and solving them. The construction of the optimization problem is the core step of model predictive control, which aims to formalize the temperature control requirements and energy consumption constraints of the system into a mathematical model, and further generate the optimal control strategy through processing.

[0151] Specifically, the purpose of designing the objective function is to clarify the optimization direction and balance the temperature control accuracy and energy consumption, so as to achieve precise cooling while maximizing energy conservation. The objective function is in the following form:

[0152] in, is the objective function value, To predict the temperature, is the target temperature, is the control variable, is the temperature control deviation weight coefficient, is the energy consumption weight coefficient, is the time variation, is the total amount of time change.

[0153] Furthermore, the purpose of defining constraints is to ensure that the optimization results are within the range of equipment operation and physical model, avoid infeasible control schemes, and limit the range of temperature and wind speed changes to ensure the safety and reliability of the control strategy. The constraints are obtained based on the equipment operation restrictions and are in the following form:

[0154]

[0155] in, , are the minimum and maximum values ​​of the fan speed, respectively. , are the minimum and maximum values ​​of the wind deflector angle respectively.

[0156] Furthermore, the optimization variables include fan speed and air guide angle, which are used to adjust the speed and direction of the system airflow respectively. By setting the upper and lower limits of the fan speed and air guide angle, the control strategy is physically feasible within the safe range of equipment operation. The optimization problem is designed based on the objective function and constraints, and is solved in real time using quadratic programming or gradient optimization algorithms to generate a dynamic optimal control strategy to simultaneously meet the system's temperature control accuracy and energy consumption minimization requirements, and ensure the rapid response capability of the control.

[0157] S63. Implement a control strategy based on the optimization problem.

[0158] In one embodiment, based on the objective function and constraints constructed by the optimization problem, rolling optimization, real-time solution and feedback adjustment mechanism are specifically applied to dynamically generate and execute the control strategy of fan speed and wind guide plate angle. At each time step, the control system takes the objective function in S62 (balancing temperature control accuracy and energy consumption) as the optimization core, and combines the constraints (equipment operating range and physical model restrictions) to adjust the predicted temperature field and wind speed field distribution in real time, and only implements the current control quantity in the control time domain. The rest will be recalculated in the next rolling optimization. The optimization problem defined in S62 is quickly solved by quadratic programming or gradient optimization algorithm to generate a control strategy that meets the real-time requirements. After the control quantity is implemented, the system deviation is corrected using real-time sensor data according to the state transfer formula and feedback error calculation mechanism, and the fan speed and wind guide plate angle are dynamically adjusted to form a closed-loop control. This process ensures the accuracy, stability and adaptability of the control strategy to complex environments.

[0159] S64. Based on the control strategy, verify the control effect.

[0160] In one embodiment, based on the objective function and constraints defined in S62, the actual effect of the optimized control strategy is fully verified through control accuracy evaluation and performance index summary.

[0161] Specifically, the control accuracy is evaluated by calculating the temperature control error. The calculation model of the temperature control error satisfies the following expression:

[0162] in, is the mean square error, is the total number of data sampling points, is the target temperature, is the measured temperature, is a sequence of sampling points.

[0163] Furthermore, the control effect is quantified by energy efficiency to evaluate the overall performance of the system. The calculation model of energy efficiency satisfies the following expression:

[0164] in, For energy efficiency, is the fan power, is the power of the cooling system, For total time.

[0165] Furthermore, the control strategy is continuously optimized according to the control effect, so as to realize the real-time optimization control of the local temperature of the tunnel.

[0166] See also Figure 2 , Figure 2 The overall structure diagram of a tunnel local cooling control system based on deep reinforcement learning in an embodiment of the present invention is shown in FIG. The system uses the tunnel local cooling control method based on deep reinforcement learning. The system includes an input device, a processor, an output device, and a memory; the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the input device, the processor, the output device, and the memory are interconnected. The main layout of the system is as follows Figure 3 shown. Figure 3 The layout structure includes a double axial flow fan unit 1, a wind guide plate unit 2, a cold wall system 3, a monitoring point 4 and a tunnel face 5. Among them, the cross-sectional dimensions and point positions of the tunnel face are Figure 3 The dashed box is marked.

[0167] In this embodiment, the input device includes a data acquisition module.

[0168] Specifically, the sampling frequency of the data acquisition module is adjustable and has data anomaly detection and automatic calibration functions, including a temperature sensor array and a wind speed sensor array. The temperature sensor array is arranged in a gridded form at key positions in the alley, and the wind speed sensor array is arranged in key sections of the airflow channel.

[0169] The processor includes a model prediction module, a decision control module and an execution control module.

[0170] Specifically, the model prediction module has online learning and model updating functions, including a deep learning prediction unit and a reinforcement learning strategy unit, which are used to realize temperature field distribution prediction and control strategy generation; the decision control module has multi-objective optimization and real-time response capabilities, including a model prediction control unit, which is used to optimize the control strategy and generate control instructions; the execution control module has fault diagnosis and automatic protection functions, including a dual-axial flow fan unit 1, an air guide plate group 2 and a cold wall system 3. The dual-axial flow fan unit 1 and the air guide plate are arranged in coordination to realize precise control of the airflow. The cold wall system 3 is used to provide a stable cold source output, and the air guide plate group 2 includes an independently adjustable air guide plate unit. The dual-axial flow fan unit 1 includes: a main air supply fan for providing the main airflow; an auxiliary fan for adjusting the airflow direction; a frequency conversion control unit for realizing precise adjustment of the fan speed; an angle adjustment mechanism for realizing angle adjustment within a range of 0° to 90°; and a position adjustment mechanism for adjusting the position of the air guide plate according to the progress of the working surface. During the operation of the system, the wind guide direction is as follows: Figure 4 shown.

[0171] The output device uses a high-definition display screen to display the control results of local cooling in the tunnel.

[0172] The memory adopts a high-speed solid-state hard disk to store data acquired by the input device, the results displayed by the output device and the results processed by the processor. The memory has the characteristics of fast reading and writing speed, large capacity and high reliability, and can meet the needs of large data storage.

[0173] See also Figure 5 , Figure 5 This is an operation flow chart of a tunnel local cooling control system based on deep reinforcement learning in an embodiment of the present invention. The contents involved in the flow chart include system initialization and modeling, deep learning model development, model predictive control development, system operation and feedback, system evaluation and maintenance, and reinforcement learning strategy development.

[0174] See also Figure 6 , Figure 6 This is a schematic diagram of the system sequential temperature field structure of an embodiment of the present invention, and the sequential temperature field distribution of 10 seconds, 20 seconds, 30 seconds, 1 minute, 2 minutes, 3 minutes, 5 minutes, 10 minutes, 30 minutes and 60 minutes are given respectively. Figure 6 In the figure, S stands for seconds and Min stands for minutes. This figure reflects the change of temperature field over time.

[0175] In this embodiment, system operation and feedback is the final execution and closed-loop optimization stage of the model predictive control system, which aims to ensure the efficient and stable operation of the system in a dynamic and complex environment through four core links: real-time data collection and processing, state prediction analysis, control decision execution, and operation effect analysis. It mainly includes the following steps: The first step is real-time data acquisition.

[0176] Specifically, real-time data acquisition is based on the sensor layout and sampling frequency configured in step S1. Through temperature and humidity sensors and wind speed sensors, environmental data and equipment operating status (such as fan speed and wind deflector angle) are collected in real time to ensure that the dynamic feedback of data meets the real-time requirements of the system. At the same time, combined with the standardized method of data preprocessing in S2, outliers are detected and corrected, and interpolation or smoothing algorithms are used to remove noise to ensure the validity and integrity of the data.

[0177] The second step is status prediction analysis.

[0178] Specifically, the state prediction analysis is based on the optimized prediction model of the temperature field and wind speed field constructed in step S3, and the future state of the system is predicted in multiple steps. The dynamic characteristics of key areas are extracted in combination with trend analysis to identify potential risks in advance. At the same time, the real-time state is evaluated through the temperature control accuracy and energy consumption constraints defined in step S62 to determine whether the target requirements are met. When the predicted or real-time data exceeds the safety threshold set by the constraint conditions in step S62, an early warning message is generated to provide a basis for the strategy adjustment in the next step of control decision execution.

[0179] The third step is to control decision execution.

[0180] Specifically, the control decision execution is based on the objective function and constraints defined in step S62, combined with the rolling optimization and real-time solution mechanism in the control strategy implementation of step S63, and generates dynamic adjustment suggestions for fan speed and wind deflector angle through reinforcement learning strategy, and calculates the optimal control instructions using model predictive control. These instructions are transmitted to the actuator to complete real-time operation, and the execution effect is monitored through the feedback adjustment mechanism in step S63, deviations are corrected and the system status is dynamically updated to ensure the efficiency and accuracy of the control strategy in complex environments.

[0181] Step 4: Operation effect analysis. Operation effect analysis calculates the temperature control error (such as mean square error) and energy efficiency indicators (such as system energy consumption efficiency) through the objective function and constraints defined in step S62 to verify the temperature control accuracy and energy consumption optimization effect of the system. At the same time, combined with the rolling optimization mechanism and feedback adjustment process in the control strategy implementation of step S63, analyze the response time and recovery ability of the system when disturbed, evaluate the stability and accuracy of the control strategy, and provide data support for further optimizing the control logic and improving system performance.

[0182] In this embodiment, system evaluation and maintenance are key steps to ensure the long-term stable operation of the model predictive control system, covering three core links: performance evaluation, model update and optimization, and system maintenance. It ensures the long-term reliability and economy of the system in a complex environment through comprehensive analysis and optimization of system performance, combined with status checks of equipment and sensors. Specifically, it includes the following steps: The first step is performance evaluation.

[0183] Specifically, the performance evaluation defines the evaluation criteria for wind speed distribution and temperature distribution by combining the parameter settings of the system physical model in step S1 to ensure that the temperature control system design is consistent with actual needs; at the same time, combined with the data processing and feature extraction results in step S3, a comprehensive analysis of the temperature control effect and energy efficiency is performed. The temperature control effect is verified by calculating the deviation between the target temperature and the actual temperature (such as the mean square error), and the energy efficiency is combined with the objective function and the system energy efficiency ratio calculation formula in step S62 to evaluate the energy consumption of the fan and cooling system. Finally, based on the comprehensive analysis of the system response stability and energy consumption cost in the control effect verification in step S64, the performance indicators are optimized to provide data support for the reliability and economy of the overall system.

[0184] The second step is model updating and optimization.

[0185] Specifically, the model update and optimization combines the deep learning model development in step S3 and the reinforcement learning strategy development in step S5. The deep learning model is retrained through historical data and real-time data to optimize the prediction accuracy of the temperature field and wind speed field; the experience replay pool and policy gradient method in reinforcement learning are used to dynamically adjust the reward function to enhance the adaptability and convergence speed of the policy network. At the same time, based on the rolling optimization mechanism and objective function setting in the model predictive control development in step S6, the prediction time domain, control time domain and parameters of the MPC are dynamically adjusted to optimize the weight coefficient in the objective function so that the control strategy can better adapt to complex dynamic environments. Through the above comprehensive optimization, the prediction accuracy and control effect of the system are continuously improved, and its operational efficiency and adaptability are improved.

[0186] The third step is system maintenance.

[0187] Specifically, by regularly checking the operating status of equipment such as fans, air guides and cooling systems, we ensure that the equipment is working stably within a safe range, and calibrate the temperature and wind speed sensors to ensure the accuracy and consistency of data collection. Combined with fault diagnosis technology, we analyze operating data and historical records, identify potential problems (such as equipment abnormalities or sensor failures), and develop maintenance plans, including equipment cleaning, component replacement and sensor calibration, and promptly implement preventive maintenance and repair measures to ensure long-term and reliable operation of the system.

[0188] In summary, the present invention provides a local cooling control method for underground tunnels based on deep learning and reinforcement learning. By collecting historical data on temperature field, wind speed field, humidity and energy consumption in the tunnel, a solid foundation is provided for the accurate prediction and control of tunnel temperature. By preprocessing historical data, the standardization and consistency of data are ensured, thereby improving the prediction accuracy of the model. Through the deep learning method, an optimized prediction model of the temperature field is constructed to efficiently capture the complex laws of environmental changes. By optimizing the prediction model and designing the reinforcement learning control environment, a simulation and verification platform is provided for the formulation of intelligent control strategies. By constructing a reinforcement learning strategy network, the changes in the tunnel environment are responded to in real time, and the cooling strategy is automatically adjusted to achieve accurate and efficient control of the local temperature of the tunnel.

[0189] The present invention provides a local cooling control system for underground tunnels based on deep learning and reinforcement learning, which is suitable for the precise cooling requirements in high-temperature environments such as deep mine working faces and tunnel construction. The system realizes temperature control in complex environments by arranging dual axial flow fans, air guide plates and cold wall systems in a coordinated manner. The system relies on a temperature sensor array to collect real-time environmental data, uses deep learning technology to extract environmental characteristics and predict changes in temperature fields and wind speed fields, introduces physical constraints to ensure that the prediction conforms to the laws of thermodynamics, combines reinforcement learning algorithms to optimize control strategies, and realizes dynamic adjustment of fan speed, air guide plate angle and cold source output through model predictive control. The combined design of dual axial flow fans and adjustable air guide plates ensures precise guidance of airflow, and the cold wall system provides stable cold source support. The system has a flexible layout and can be adjusted according to the mining working face or construction progress, and has fault warning and adaptive adjustment capabilities. Practical application verification shows that the present invention effectively improves the temperature control accuracy, reduces system energy consumption, shortens the control response time, and provides a reliable solution for underground high temperature management.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A tunnel local cooling control method based on deep reinforcement learning, characterized in that: The method comprises the following steps: Obtain historical data on temperature field, wind speed field, humidity and energy consumption in the tunnel environment; Preprocessing the historical data to obtain standardized data; Based on the standardized data, construct an optimization prediction model of the deep learning temperature field; Based on the optimization prediction model, design a reinforcement learning control environment; Based on the reinforcement learning control environment, construct a reinforcement learning strategy network; Based on the reinforcement learning strategy network, real-time optimization control of the local temperature of the tunnel is achieved.

2. According to the method of claim 1, the local cooling control method of the tunnel based on deep reinforcement learning is characterized in that: The preprocessing of the historical data to obtain standardized data includes: Performing data cleaning and outlier processing on the historical data to obtain optimized data; The optimized data is standardized to obtain standardized data.

3. The method for controlling local temperature drop in a lane based on deep reinforcement learning according to claim 1, characterized in that: The step of constructing an optimized prediction model of a deep learning temperature field based on the standardized data includes: Based on the standardized data, construct a deep neural network structure; Based on the deep neural network structure, design the input layer of temperature field and wind speed field data; Based on the input layer, design a feature extraction layer and a time series prediction layer; Based on the feature extraction layer and the time series prediction layer, designing a loss function with physical constraints; Based on the loss function with physical constraints, determine model training parameters and optimization methods; By using the model training parameters and the optimization method, an optimization prediction model of the deep learning temperature field is constructed.

4. The method for controlling local temperature drop in a lane based on deep reinforcement learning according to claim 1, characterized in that: The designing of a reinforcement learning control environment based on the optimization prediction model comprises: Based on the optimization prediction model, an environmental state space, an action space and a reward function are designed. The environmental state space includes temperature field distribution, wind speed field distribution, fan speed and wind guide plate angle. The action space includes the adjustment range of the fan speed and the adjustment range of the wind guide plate angle. The reward function includes the target temperature and current energy consumption.

5. The method for controlling local temperature drop in a lane based on deep reinforcement learning according to claim 1, characterized in that: The step of constructing a reinforcement learning strategy network based on the reinforcement learning control environment includes: Based on the reinforcement learning control environment, design the network structure and loss function of deep reinforcement learning; Based on the network structure and loss function of the deep reinforcement learning, a reinforcement learning strategy network is constructed.

6. A tunnel local temperature reduction control method based on deep reinforcement learning according to claim 5, characterized in that: The construction of a reinforcement learning strategy network based on the network structure and loss function of the deep reinforcement learning includes: Based on the network structure and loss function of the deep reinforcement learning, the experience replay pool is used to perform strategy training iterations to obtain iterative results; Based on the iteration results, the strategy performance indicators are evaluated and a reinforcement learning strategy network is constructed. The strategy performance indicators include temperature control accuracy in the target area, temperature field distribution uniformity, wind speed field uniformity, system comprehensive energy consumption and control response time.

7. The method for controlling local temperature drop in a lane based on deep reinforcement learning according to claim 1, characterized in that: The method of realizing real-time optimization control of the local temperature of the tunnel based on the reinforcement learning strategy network includes: Based on the reinforcement learning strategy network, a prediction model considering the dynamic characteristics of the temperature field is constructed, wherein the prediction model sets a prediction time domain and a control time domain; Based on the prediction model, an optimization objective function including temperature control accuracy and energy efficiency is designed; Generate a dynamic optimal control strategy through the optimization objective function and constraint conditions; According to the optimal control strategy, real-time optimization control of the local temperature of the tunnel is achieved.

8. A tunnel local cooling control system based on deep reinforcement learning, the system using a tunnel local cooling control method based on deep reinforcement learning according to any one of claims 1 to 7, characterized in that: The system includes an input device, a processor, an output device, and a memory; The input device includes a data acquisition module; The processor includes a model prediction module, a decision control module and an execution control module; The input device, the processor, the output device and the memory are connected to each other; The memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions.

9. A tunnel local cooling control system based on deep reinforcement learning according to claim 8, characterized in that: The data acquisition module has an adjustable sampling frequency and has data anomaly detection and automatic calibration functions, including a temperature sensor array and a wind speed sensor array. The temperature sensor array is arranged in a gridded form at key positions in the lane, and the wind speed sensor array is arranged at key sections of the airflow channel; The model prediction module has online learning and model updating functions, including a deep learning prediction unit and a reinforcement learning strategy unit, which are used to predict the temperature field distribution and generate control strategies; The decision control module has multi-objective optimization and real-time response capabilities, including a model prediction control unit for optimizing control strategies and generating control instructions; The execution control module has fault diagnosis and automatic protection functions, and includes a dual axial flow fan unit, an adjustable air guide plate group and a cold wall system. The dual axial flow fan unit and the air guide plate are arranged in a coordinated manner. The air guide plate group includes an independently adjustable air guide plate unit. The cold wall system provides a stable cold source output.

10. A tunnel local cooling control system based on deep reinforcement learning according to claim 9, characterized in that: The dual axial flow fan unit includes a main blower, an auxiliary blower, a frequency conversion control unit, an angle adjustment mechanism and a position adjustment mechanism; The main blower is used to provide the main airflow; The auxiliary fan is used to adjust the airflow direction; The frequency conversion control unit is used to adjust the fan speed; The angle adjustment mechanism is used to adjust the angle within the range of 0° to 90°; The position adjustment mechanism is used to adjust the position of the air guide plate.

Images