Method for improving heat transfer efficiency through multi-phase heat transfer reinforcement learning of direct water dispenser

Abstract

The invention belongs to the technical field of heat conduction of direct water dispensers, and particularly relates to a method for improving heat transfer efficiency of multi-phase heat transfer reinforcement learning of a direct water dispenser, which comprises the following steps of: establishing a simulation input set of multi-physical field data, adopting a multi-phase heat transfer theory as a basic model, and constructing a three-dimensional simulation model of a micro-channel heater of the direct water dispenser based on CFD (computational fluid dynamics) software; performing high-precision heat-flow field coupling simulation on different fin structure topologies; setting an environment state as a CFD simulation result, an action space as a fin geometric parameter, and a reward function as a target function of weighted summation of comprehensive heat transfer performance indexes; structural parameters are autonomously explored and updated through circular interaction between the DDPG proxy model and CFD simulation; constructing a corresponding micro-channel heater prototype and performing experimental verification; and an optimization strategy is integrated into the control system, so that the effects of more effectively utilizing steam or bubbles and stably controlling the gas-liquid mixing state in the multiphase heat transfer process are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of heat conduction of direct drinking water machines, and in particular to a method for improving the heat transfer efficiency of direct drinking water machines through multiphase heat transfer reinforcement learning. Background Art

[0002] Improving water heating efficiency is a key technical challenge in the design and use of direct drinking water dispensers. Existing single-phase heat transfer methods, due to limitations in heat conduction and convection, result in slow heating and high energy consumption. Therefore, improving the heat transfer efficiency of direct drinking water dispensers to provide faster and more energy-efficient heating has become a key issue in improving user experience and reducing energy consumption.

[0003] Currently, multiphase heat transfer technology has been introduced into the design of direct drinking water machines because it can effectively improve heat transfer efficiency. By introducing bubbles or vaporization phenomena, such as jet steam or the use of microbubble technology, the heat exchange between the heat transfer medium (usually water) and the heat source can be significantly enhanced. This technology can provide a faster heating rate per unit time, thereby shortening the heating time and reducing energy consumption. In addition, multiphase heat transfer technology can further improve the heat transfer efficiency by improving the flow characteristics of the heat transfer interface, which is of great significance for the effective application of direct drinking water machines.

[0004] Although multiphase heat transfer technology has shown significant advantages, it still faces some challenges in practical application, such as how to more effectively utilize steam or bubbles and how to stably control the gas-liquid mixing state during the multiphase heat transfer process. In addition, further research is needed to optimize heat transfer efficiency by dynamically adjusting heat flow or improving cooling conditions, thereby achieving more efficient direct drinking water machine system design.

[0005] In response to the above technical defects, a solution is proposed to improve the heat transfer efficiency of direct drinking water machines through multiphase heat transfer reinforcement learning. Summary of the Invention

[0006] In order to solve the above problems, the present invention provides the following technical solutions:

[0007] A method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning, comprising:

[0008] A simulation input set of multi-physics field data was established. Using multiphase heat transfer theory as the basic model, a three-dimensional simulation model of a microchannel heater for a direct drinking water dispenser was constructed using CFD software. High-precision thermal-fluid field coupling simulations were performed for different fin structure topologies.

[0009] A reinforcement learning agent model based on DDPG was constructed, with the environment state set as the CFD simulation results, the action space as the fin geometric parameters, and the reward function as the objective function of the weighted sum of comprehensive heat transfer performance indicators.

[0010] Through the cyclic interaction between the DDPG agent model and CFD simulation, the structural parameters are autonomously explored and updated, and the CFD simulation provides feedback on the heat transfer performance indicators, forming a closed-loop learning process until convergence;

[0011] The optimal fin structure topology design obtained by DDPG reinforcement learning was selected, and the corresponding microchannel heater prototype was constructed and experimentally verified.

[0012] The optimization strategy is integrated into the control system and combined with real-time sensor data to predict water flow status and heat load.

[0013] Furthermore, the steps of establishing a simulation input set of multi-physics field data and using multiphase heat transfer theory as the basic model include collecting basic parameters of the direct drinking water machine to ensure the accuracy and consistency of the data, including parameters related to thermal-flow field coupling, selecting suitable CFD software, and becoming familiar with its operating interface and functional modules, and using CAD software or the modeling tools provided by the CFD software to construct a three-dimensional geometric model of the microchannel heater of the direct drinking water machine;

[0014] Mesh the 3D model, select the mesh type, select the flow model, consider the turbulent characteristics of the flow, set the convection heat transfer model, set the fluid inlet velocity, temperature, and pressure parameters, set the outlet pressure or flow, and set the thermal boundary conditions of the heater wall;

[0015] Select the solver, set the discretization method, start the CFD solver, perform thermal-flow field coupling simulation, monitor the residual changes and physical quantity distribution during the calculation process, use the CFD software's post-processing tools to analyze the simulation results, parameterize the fin's geometric parameters to facilitate subsequent optimization, simulate different fin structure topologies, and analyze their impact on heat transfer performance and flow resistance.

[0016] Furthermore, the step of constructing a reinforcement learning agent model with DDPG as the core and setting the environment state as the CFD simulation result includes defining a reinforcement learning environment, where the environment state is determined by the CFD simulation results, including key thermal-flow field parameters, and the action space is defined as the geometric parameters of the fin, which is used to describe the structural topology of the fin;

[0017] The reward function is a weighted sum of comprehensive heat transfer performance indicators, which is used to guide the learning direction of the agent. The reward function is as follows;

[0018] R=w1·E heat +w2·(-F flow)+w3·C compact +w4(-C cost ),

[0019] Among them, w1, w2, w3 and w4 are weight coefficients, E heat is the heat transfer efficiency, F flow is the flow resistance, C compact For compact structure, C cost is the manufacturing cost;

[0020] A deep learning framework was selected to implement the DDPG algorithm. By randomly sampling the fin structure topology, CFD simulation was performed to collect initial experience samples. Based on the current state, actions were generated through the policy network, and noise was added to promote exploration. The generated actions were input into the CFD simulation model for thermal-flow field coupling simulation to obtain the next state and reward. The trend of the reward function during training was monitored to determine whether the agent had converged. After the agent converged, the optimal policy network was used to generate the optimal fin geometric parameters. Based on the optimal fin structure design, a microchannel heater prototype was constructed, and experimental tests were carried out to measure the actual heat transfer efficiency and flow resistance performance indicators.

[0021] Furthermore, the step of achieving autonomous exploration and updating of structural parameters through cyclic interaction between the DDPG agent model and the CFD simulation includes initializing the fin structural parameters of the microchannel heater of the direct drinking water machine, using random initial values ​​or an initial design based on experience, and performing CFD simulation by randomly sampling the fin structural topology to collect initial empirical samples;

[0022] Based on the current state, the policy network generates fin geometry parameters. Noise is added to the generated actions to increase randomness and avoid premature convergence. The generated fin geometry parameters are input into the CFD simulation model to perform a thermal-fluid field coupling simulation. The fluid flow and heat transfer performance of the microchannel heater under different fin structures are simulated. The reward function value is calculated based on the simulation results. The reward function comprehensively considers the heat transfer efficiency and flow resistance performance indicators.

[0023] Randomly sample a batch of experience samples from the experience replay library, use the sampled experience batches to update the value network, and minimize the error between state and action value; during the training process, monitor the changing trend of the reward function to determine whether the agent has converged.

[0024] Furthermore, the steps of selecting the optimal fin structure topology design obtained by DDPG reinforcement learning, constructing the corresponding microchannel heater prototype, and conducting experimental verification include finding the optimal strategy through the collaborative optimization of the policy network and the value network, ensuring a clear understanding of the basic principles and implementation details of the DDPG algorithm, including the network structure, loss function, and optimization method;

[0025] During DDPG training, we continuously monitor the changes in the reward function and record the fin geometry parameters when the reward function reaches its maximum value. We then use CAD software to perform detailed design based on the optimal fin parameters to ensure that the design meets actual manufacturing requirements. Based on the design drawings, we then use the selected manufacturing process to produce a prototype.

[0026] Determine the experimental objectives, such as measuring heat transfer efficiency and flow resistance performance indicators, conduct actual tests according to the experimental plan, record the experimental data, compare the experimental results with the CFD simulation results, and verify the accuracy of the optimization model.

[0027] Furthermore, the step of integrating the optimization strategy into the control system and predicting the water flow state and heat load in combination with real-time sensor data includes determining the main objectives of the control system, including maximizing heat transfer efficiency, minimizing flow resistance, and maintaining stable temperature control, and installing sensors, such as flow sensors, temperature sensors, and pressure sensors;

[0028] Collect sensor data in real time, record water flow velocity, temperature distribution, and pressure change parameters, and pre-process the collected data, including denoising, normalization, and feature extraction, for subsequent analysis and prediction;

[0029] Select an appropriate time series prediction model and train it using historical data to ensure that the model can capture the changing trends of water flow conditions and heat loads. Embed the optimal fin parameters obtained from DDPG reinforcement learning into the control system to ensure that the control system can automatically adjust the fin parameters to maintain optimal heat transfer performance under different water flow and heat load conditions.

[0030] Design a closed-loop control system to monitor sensor data in real time and predict future water flow conditions and heat loads. Based on the prediction results, dynamically adjust the heater's operating parameters, such as the angle and spacing of the fins, to optimize heat transfer performance and flow resistance. Test the control system in an actual operating environment to verify its prediction and control accuracy.

[0031] According to one aspect of the present invention, a system for improving heat transfer efficiency of a direct drinking water dispenser by using multiphase heat transfer reinforcement learning is provided, comprising:

[0032] Reinforcement learning model module, used to generate the action space of fin structure parameters, and realize autonomous exploration and update of fin structure parameters through the coordinated optimization of policy network and value network;

[0033] The CFD simulation module is used to perform thermal-flow field coupling simulation on the fin structure parameters generated by the reinforcement learning model and provide feedback on heat transfer performance indicators;

[0034] The experience replay library module is used to store experience samples obtained during the interaction between the reinforcement learning model and the CFD simulation module, including state, action, reward, and next state;

[0035] The optimization module is used to update the policy network and value network of the reinforcement learning model according to the heat transfer performance indicators fed back by the CFD simulation module until convergence;

[0036] The control system integration module is used to integrate the optimal fin structure parameters obtained by the optimization module into the control system of the direct drinking water machine, combine real-time sensor data to predict the water flow state and heat load, and dynamically adjust the fin structure parameters to maintain the best heat transfer efficiency.

[0037] Furthermore, the system also includes a geometric modeling submodule for constructing a three-dimensional geometric model of the microchannel heater based on the fin structure parameters generated by the reinforcement learning model; a meshing submodule for meshing the geometric model to generate a computational mesh suitable for CFD simulation; a solver submodule for solving the thermal-flow field coupling equations and calculating performance indicators such as heat transfer efficiency and flow resistance; a result output submodule for outputting the result data of the CFD simulation, including temperature distribution, flow velocity distribution, and pressure distribution; and a reward function design submodule for designing a reward function that comprehensively considers heat transfer efficiency and flow resistance.

[0038] The target network update submodule is used to gradually update the target value network and target policy network through a soft update strategy to ensure the stability of the target network; the network training submodule is used to batch update the value network and policy network through samples in the experience replay library to minimize the loss function; the sensor data acquisition submodule is used to collect parameters such as water flow velocity, temperature distribution, and pressure changes of the direct drinking water machine in real time; the state prediction submodule is used to predict future water flow status and heat load based on real-time sensor data; the dynamic adjustment submodule is used to dynamically adjust the fin structure parameters of the microchannel heater according to the prediction results to maintain the optimal heat transfer efficiency.

[0039] According to one aspect of the present invention, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the steps of the above-mentioned method for improving the heat transfer efficiency of a direct drinking water machine multiphase flow reinforcement learning.

[0040] According to one aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for improving the heat transfer efficiency of a direct drinking water machine multiphase flow reinforcement learning are implemented.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] In a method for improving the heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning of the present invention, a simulation input set of multi-physical field data is established, multiphase heat transfer theory is used as a basic model, a three-dimensional simulation model of a microchannel heater of a direct drinking water machine is constructed based on CFD software, and high-precision thermal-flow field coupling simulation is performed on different fin structure topologies; a reinforcement learning agent model with DDPG as the core is constructed, the environmental state is set as the CFD simulation result, the action space is the fin geometric parameters, and the reward function is the objective function of the weighted summation of comprehensive heat transfer performance indicators; through the cyclic interaction between the DDPG agent model and the CFD simulation, autonomous exploration and updating of structural parameters are achieved, and the CFD simulation feeds back the heat transfer performance indicators to form a closed-loop learning process until convergence; the optimal fin structure topology design obtained by DDPG reinforcement learning is selected, and the corresponding microchannel heater prototype is constructed and experimentally verified; the optimization strategy is integrated into the control system, and the water flow state and heat load are predicted in combination with real-time sensor data, which has the effect of more effectively utilizing steam or bubbles and stably controlling the gas-liquid mixing state in the multiphase heat transfer process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0044] Figure 1 This is an overall schematic diagram of a method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to the present invention;

[0045] Figure 2 This is a schematic diagram of a framework of a system for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to the present invention;

[0046] Figure 3 This is a schematic diagram of the computer structure in a system for improving heat transfer efficiency through multiphase heat transfer reinforcement learning in a direct drinking water machine according to the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] like Figure 1-Figure 3 As shown, the present application provides a method for improving heat transfer efficiency of a direct drinking water machine by using multiphase heat transfer reinforcement learning, comprising:

[0049] S1: Establish a simulation input set of multi-physics field data, use multiphase heat transfer theory as the basic model, build a three-dimensional simulation model of the microchannel heater of the direct drinking water machine based on CFD software, and perform high-precision thermal-flow field coupling simulation for different fin structure topologies;

[0050] S2: Build a reinforcement learning agent model based on DDPG, set the environment state as the CFD simulation results, the action space as the fin geometric parameters, and the reward function as the objective function of the weighted sum of the comprehensive heat transfer performance indicators;

[0051] S3: Through the cyclic interaction between the DDPG agent model and the CFD simulation, the structural parameters are autonomously explored and updated, and the CFD simulation provides feedback on the heat transfer performance indicators, forming a closed-loop learning process until convergence;

[0052] S4: Select the optimal fin structure topology design obtained by DDPG reinforcement learning, build the corresponding microchannel heater prototype and conduct experimental verification;

[0053] S5: Integrate the optimization strategy into the control system and combine it with real-time sensor data to predict water flow status and heat load.

[0054] Furthermore, the steps of establishing a simulation input set of multi-physics field data and using multiphase heat transfer theory as the basic model include collecting basic parameters of the direct drinking water machine to ensure the accuracy and consistency of the data, including parameters related to thermal-flow field coupling, selecting suitable CFD software, and becoming familiar with its operating interface and functional modules, and using CAD software or the modeling tools provided by the CFD software to construct a three-dimensional geometric model of the microchannel heater of the direct drinking water machine;

[0055] Mesh the 3D model, select the mesh type, select the flow model, consider the turbulent characteristics of the flow, set the convection heat transfer model, set the fluid inlet velocity, temperature, and pressure parameters, set the outlet pressure or flow, and set the thermal boundary conditions of the heater wall;

[0056] Select the solver, set the discretization method, start the CFD solver, perform thermal-flow field coupling simulation, monitor the residual changes and physical quantity distribution during the calculation process, use the CFD software's post-processing tools to analyze the simulation results, parameterize the fin's geometric parameters to facilitate subsequent optimization, simulate different fin structure topologies, and analyze their impact on heat transfer performance and flow resistance.

[0057] In one embodiment, the basic parameters of the direct drinking water machine are collected, including the geometric dimensions of the microchannel heater (such as channel width, height, length, etc.). The thermal physical properties of the heater material (such as thermal conductivity, specific heat capacity, density, etc.). The physical properties of the working fluid (such as water density, viscosity, specific heat capacity, thermal conductivity, etc., which vary with temperature). The geometric parameters of the fins (such as fin height, fin spacing, fin inclination angle, etc.). The operating conditions (such as water inlet temperature, water outlet temperature, flow rate, etc.) are collected.

[0058] Organize multi-physics data to ensure data accuracy and consistency, including parameters related to thermal-fluid coupling (such as convective heat transfer coefficient, flow resistance, etc.). Format the data into a format readable by CFD software (such as CSV, Excel, or text files).

[0059] Select CFD software, choose suitable CFD software (such as ANSYS Fluent, COMSOL Multiphysics, OpenFOAM, etc.), and familiarize yourself with its operating interface and functional modules.

[0060] Geometric modeling: Use CAD software (such as SolidWorks, AutoCAD) or CFD software's built-in modeling tools to build a 3D geometric model of the microchannel heater for the direct drinking water machine, including details such as the heater body, fin structure, and water inlet and outlet.

[0061] Meshing: Mesh the 3D model and select the appropriate mesh type (such as structured or unstructured). Ensure mesh quality and avoid excessive mesh distortion, especially in critical areas such as fins and flow bends, where mesh refinement is required to improve simulation accuracy.

[0062] Fluid flow model: Select an appropriate flow model (such as the Navier-Stokes equations), consider the turbulent characteristics of the flow (such as using the k-ε turbulence model or the RANS model), and set the physical properties of the fluid (such as density, viscosity, specific heat capacity, etc.).

[0063] Heat transfer model, set the convection heat transfer model (such as local convection heat transfer coefficient calculation). Consider heat conduction (such as the thermal conductivity of the heater material) and radiation heat transfer (such as radiation heat transfer on the fin surface).

[0064] Multiphase flow model (if applicable): If multiphase flow is involved (such as gas-liquid two-phase flow), select an appropriate multiphase flow model (such as VOF model, mixture model, etc.). Set parameters such as interphase force and surface tension.

[0065] Inlet boundary conditions: Set the fluid inlet velocity, temperature, pressure, and other parameters. Ensure that the inlet conditions are consistent with the actual operating conditions. Outlet boundary conditions: Set the outlet pressure (such as free outflow) or flow rate (such as mass flow outlet). Ensure that the outlet conditions do not have an unreasonable impact on the internal flow. Wall boundary conditions: Set the thermal boundary conditions of the heater wall (such as convection heat transfer coefficient, wall temperature, etc.). Ensure that the heat exchange model between the fins and the fluid is accurate.

[0066] Select the appropriate solver (e.g., a pressure-velocity coupled solver, such as the PISO algorithm or SIMPLE algorithm). Set the discretization method (e.g., a second-order upwind scheme or a higher-order scheme). Set the solution parameters and convergence criteria (e.g., residual convergence criteria). Set the time step (e.g., steady-state or transient solution), the number of iterations, and the monitored variables (e.g., pressure, velocity, temperature, etc.).

[0067] Start the simulation and the CFD solver to perform a coupled thermal-flow simulation. Monitor the residuals and physical quantity distributions (such as flow rate, pressure, and temperature) during the calculation. Post-process the results using the CFD software's post-processing tools to analyze the simulation results (such as streamlines, isotherms, and convective heat transfer coefficient distribution). Extract key performance indicators (such as total heat transfer coefficient, flow resistance, and thermal efficiency).

[0068] Fin structure parameterization: Parameterize fin geometric parameters (such as fin height, fin pitch, and fin tilt angle) to facilitate subsequent optimization. Multi-topology simulation: simulate and calculate different fin structure topologies to analyze their impact on heat transfer performance and flow resistance. Record the simulation results for each topology to provide data support for subsequent reinforcement learning.

[0069] Compare simulation results with experimental data to verify the accuracy of the simulation model. If any deviations are found, adjust model parameters (such as turbulence model, wall roughness, etc.) or meshing strategies. Calibrate the simulation model using experimental data to ensure the reliability of the simulation results.

[0070] Through the above steps, a high-precision thermal-flow field coupling simulation model can be established, and simulation analysis of different fin structure topologies can be performed to provide reliable data support for subsequent reinforcement learning optimization.

[0071] Furthermore, the step of constructing a reinforcement learning agent model with DDPG as the core and setting the environment state as the CFD simulation result includes defining a reinforcement learning environment, where the environment state is determined by the CFD simulation results, including key thermal-flow field parameters, and the action space is defined as the geometric parameters of the fin, which is used to describe the structural topology of the fin;

[0072] The reward function is a weighted sum of comprehensive heat transfer performance indicators, which is used to guide the learning direction of the agent. The reward function is as follows;

[0073] R=w1·E heat +w2·(-F flow )+w3·C compact +w4(-C cost ),

[0074] Among them, w1, w2, w3 and w4 are weight coefficients, E heat is the heat transfer efficiency, F flow is the flow resistance, C compact For compact structure, C cost is the manufacturing cost;

[0075] A deep learning framework was selected to implement the DDPG algorithm. By randomly sampling the fin structure topology, CFD simulation was performed to collect initial experience samples. Based on the current state, actions were generated through the policy network, and noise was added to promote exploration. The generated actions were input into the CFD simulation model for thermal-flow field coupling simulation to obtain the next state and reward. The trend of the reward function during training was monitored to determine whether the agent had converged. After the agent converged, the optimal policy network was used to generate the optimal fin geometric parameters. Based on the optimal fin structure design, a microchannel heater prototype was constructed, and experimental tests were carried out to measure the actual heat transfer efficiency and flow resistance performance indicators.

[0076] In one embodiment, a reinforcement learning environment, a state space (StateSpace), is defined, and the state of the environment is determined by CFD simulation results, including key thermal-flow field parameters.

[0077] State variables can include the fluid's temperature distribution (e.g., inlet temperature, outlet temperature, wall temperature, etc.), flow velocity distribution (e.g., primary velocity, return velocity, etc.), pressure distribution (e.g., inlet pressure, outlet pressure, flow resistance, etc.), heat transfer performance indicators (e.g., overall heat transfer coefficient, thermal efficiency, etc.), and flow resistance coefficients (e.g., friction coefficient, pressure loss, etc.).

[0078] Action Space: The action space is defined as the geometric parameters of the fins, describing their structural topology. Action variables can include Fin Height: the height of the fin extending outward from the heater body; Fin Pitch: the distance between adjacent fins; and Fin Angle: the angle at which the fins are tilted relative to the heater body.

[0079] Fin Thickness: The thickness of the fin. Fin Layers: The number of layers or rows of fins.

[0080] Reward function (RewardFunction), the reward function is the weighted sum of comprehensive heat transfer performance indicators, which is used to guide the learning direction of the agent. The components of the reward function may include: Heat transfer efficiency (HeatTransferEfficiency): measures the amount of heat transferred per unit time. Flow resistance (FlowResistance): measures the flow loss when the fluid passes through the heater. Structural compactness (StructuralCompactness): measures the compactness of the fin design to avoid overly complex structures. Manufacturing cost (ManufacturingCost): measures the manufacturing difficulty and cost of the fin structure. Select a deep learning framework: Select a suitable deep learning framework (such as TensorFlow, PyTorch, Keras, etc.) to implement the DDPG algorithm.

[0081] Design the neural network architecture: the Policy Network. Input: state vector (CFD simulation results). Output: action vector (fin geometry parameters). Network Architecture: Either a Multilayer Perceptron (MLP) or a Convolutional Neural Network (CNN) can be used, depending on the complexity of the state space. The Value Network: Input: state vector and action vector. Output: value of the state-action pair (i.e., expected reward). Network Architecture: Again, either an MLP or a CNN can be used.

[0082] Set hyperparameters: Learning Rate (LearningRate): The learning rate of the policy network and value network. Batch Size (BatchSize): The amount of data to train in each batch. Replay Buffer Size (ReplayBufferSize): The number of experience samples to store. Noise Policy (NoisePolicy): The type of noise used for exploration (such as Ornstein-Uhlenbeck noise). Soft Update Coefficient (SoftUpdateCoefficient): The coefficient used to update the target network.

[0083] Initialize the experience replay library. Perform CFD simulations by randomly sampling the fin structure topology to collect initial experience samples (state, action, reward, and next state). Store these initial experience samples in the experience replay library. Sample actions. The agent generates actions (fin geometry parameters) based on the current state through the policy network and adds noise to promote exploration. Execute the action. Input the generated action (fin geometry parameters) into the CFD simulation model for thermal-flow field coupling simulation to obtain the next state and reward. Store the experience. Store the current state, action, reward, and next state in the experience replay library.

[0084] Sample experience batches: Randomly sample a batch of experience samples from the experience replay library. Update the value network: Using the sampled experience batches, update the value network to minimize the error between the state and action values. Update the policy network: Using the gradient of the value network, update the policy network to maximize the expected reward. Soft-update the target network: Using the soft update policy, gradually update the target value network and target policy network. Repeat the training, repeating the above steps until the agent converges (i.e., the reward function no longer changes significantly).

[0085] Convergence determination monitors the reward function's trend during training to determine whether the agent has converged. If the reward function remains stable over multiple training cycles, the agent is considered converged. The optimal fin structure is extracted. Once the agent converges, the optimal policy network is used to generate the optimal fin geometry. The optimal fin structure is then input into a CFD simulation model to verify its heat transfer performance and flow resistance.

[0086] Build a prototype and conduct experimental testing. Based on the optimal fin structure design, construct a microchannel heater prototype. Conduct experimental testing to measure performance indicators such as actual heat transfer efficiency and flow resistance. Compare experimental results with simulation results to verify the accuracy and reliability of the model. Through these steps, a reinforcement learning agent model based on DDPG can be constructed to autonomously optimize the fin structure and improve the heat transfer efficiency and system performance of the direct drinking water dispenser.

[0087] Furthermore, the step of achieving autonomous exploration and updating of structural parameters through cyclic interaction between the DDPG agent model and the CFD simulation includes initializing the fin structural parameters of the microchannel heater of the direct drinking water machine, using random initial values ​​or an initial design based on experience, and performing CFD simulation by randomly sampling the fin structural topology to collect initial empirical samples;

[0088] Based on the current state, the policy network generates fin geometry parameters. Noise is added to the generated actions to increase randomness and avoid premature convergence. The generated fin geometry parameters are input into the CFD simulation model to perform a thermal-fluid field coupling simulation. The fluid flow and heat transfer performance of the microchannel heater under different fin structures are simulated. The reward function value is calculated based on the simulation results. The reward function comprehensively considers the heat transfer efficiency and flow resistance performance indicators.

[0089] Randomly sample a batch of experience samples from the experience replay library, use the sampled experience batches to update the value network, and minimize the error between state and action value; during the training process, monitor the changing trend of the reward function to determine whether the agent has converged.

[0090] Furthermore, the steps of selecting the optimal fin structure topology design obtained by DDPG reinforcement learning, constructing the corresponding microchannel heater prototype, and conducting experimental verification include finding the optimal strategy through the collaborative optimization of the policy network and the value network, ensuring a clear understanding of the basic principles and implementation details of the DDPG algorithm, including the network structure, loss function, and optimization method;

[0091] During DDPG training, we continuously monitor the changes in the reward function and record the fin geometry parameters when the reward function reaches its maximum value. We then use CAD software to perform detailed design based on the optimal fin parameters to ensure that the design meets actual manufacturing requirements. Based on the design drawings, we then use the selected manufacturing process to produce a prototype.

[0092] Determine the experimental objectives, such as measuring heat transfer efficiency and flow resistance performance indicators, conduct actual tests according to the experimental plan, record the experimental data, compare the experimental results with the CFD simulation results, and verify the accuracy of the optimization model.

[0093] Furthermore, the step of integrating the optimization strategy into the control system and predicting the water flow state and heat load in combination with real-time sensor data includes determining the main objectives of the control system, including maximizing heat transfer efficiency, minimizing flow resistance, and maintaining stable temperature control, and installing sensors, such as flow sensors, temperature sensors, and pressure sensors;

[0094] Collect sensor data in real time, record water flow velocity, temperature distribution, and pressure change parameters, and pre-process the collected data, including denoising, normalization, and feature extraction, for subsequent analysis and prediction;

[0095] Select an appropriate time series prediction model and train it using historical data to ensure that the model can capture the changing trends of water flow conditions and heat loads. Embed the optimal fin parameters obtained from DDPG reinforcement learning into the control system to ensure that the control system can automatically adjust the fin parameters to maintain optimal heat transfer performance under different water flow and heat load conditions.

[0096] Design a closed-loop control system to monitor sensor data in real time and predict future water flow conditions and heat loads. Based on the prediction results, dynamically adjust the heater's operating parameters, such as the angle and spacing of the fins, to optimize heat transfer performance and flow resistance. Test the control system in an actual operating environment to verify its prediction and control accuracy.

[0097] According to one aspect of the present invention, a system for improving heat transfer efficiency of a direct drinking water dispenser by using multiphase heat transfer reinforcement learning is provided, comprising:

[0098] Reinforcement learning model module, used to generate the action space of fin structure parameters, and realize autonomous exploration and update of fin structure parameters through the coordinated optimization of policy network and value network;

[0099] The CFD simulation module is used to perform thermal-flow field coupling simulation on the fin structure parameters generated by the reinforcement learning model and provide feedback on heat transfer performance indicators;

[0100] The experience replay library module is used to store experience samples obtained during the interaction between the reinforcement learning model and the CFD simulation module, including state, action, reward, and next state;

[0101] The optimization module is used to update the policy network and value network of the reinforcement learning model according to the heat transfer performance indicators fed back by the CFD simulation module until convergence;

[0102] The control system integration module is used to integrate the optimal fin structure parameters obtained by the optimization module into the control system of the direct drinking water machine, combine real-time sensor data to predict the water flow state and heat load, and dynamically adjust the fin structure parameters to maintain the best heat transfer efficiency.

[0103] Furthermore, the system also includes a geometric modeling submodule for constructing a three-dimensional geometric model of the microchannel heater based on the fin structure parameters generated by the reinforcement learning model; a meshing submodule for meshing the geometric model to generate a computational mesh suitable for CFD simulation; a solver submodule for solving the thermal-flow field coupling equations and calculating performance indicators such as heat transfer efficiency and flow resistance; a result output submodule for outputting the result data of the CFD simulation, including temperature distribution, flow velocity distribution, and pressure distribution; and a reward function design submodule for designing a reward function that comprehensively considers heat transfer efficiency and flow resistance.

[0104] The target network update submodule is used to gradually update the target value network and target policy network through a soft update strategy to ensure the stability of the target network; the network training submodule is used to batch update the value network and policy network through samples in the experience replay library to minimize the loss function; the sensor data acquisition submodule is used to collect parameters such as water flow velocity, temperature distribution, and pressure changes of the direct drinking water machine in real time; the state prediction submodule is used to predict future water flow status and heat load based on real-time sensor data; the dynamic adjustment submodule is used to dynamically adjust the fin structure parameters of the microchannel heater according to the prediction results to maintain the optimal heat transfer efficiency.

[0105] In one embodiment, the reinforcement learning model is initialized to set the initial values ​​of the fin structure parameters of the microchannel heater of the direct drinking water machine, including fin spacing, height, thickness, etc. The experience playback library is initialized to prepare to store the experience samples generated during the reinforcement learning process.

[0106] The action sampling and execution reinforcement learning model generates an action space for fin structural parameters through a policy network, sampling and generating specific fin geometric parameters. The generated fin parameters are input into the CFD simulation module for thermal-flow field coupling simulation, calculating performance indicators such as heat transfer efficiency and flow resistance.

[0107] The experience storage stores the current state (fin parameters), action (optimized fin parameters), reward (comprehensive reward based on heat transfer efficiency and flow resistance), and next state (simulation results) in the experience replay library.

[0108] Network updates randomly sample a batch of experience samples from the experience replay library to update the value network and policy network of the reinforcement learning model. A soft update strategy gradually updates the target value network and target policy network to ensure the stability of the target network. The loss function is calculated and the network parameters are optimized to gradually improve the performance of the reinforcement learning model.

[0109] System Integration and Dynamic Adjustment: The optimal fin structure parameters obtained by the optimization module are integrated into the direct drinking water dispenser's control system. This system collects real-time sensor data, including water flow velocity, temperature distribution, and pressure changes. Based on this real-time sensor data, it predicts future water flow conditions and heat loads. Based on these predictions, it dynamically adjusts the microchannel heater's fin structure parameters to maintain optimal heat transfer efficiency.

[0110] Prototype Manufacturing and Experimental Testing Based on the optimized fin structure parameters, a microchannel heater prototype is manufactured through processes such as 3D printing or CNC machining. The manufactured prototype is experimentally tested to verify its heat transfer performance and flow resistance to ensure the reliability of the optimization results.

[0111] Data visualization and performance evaluation: Visualize CFD simulation results and experimental data for easy analysis and optimization. Comprehensively evaluate the performance of optimized microchannel heaters, including heat transfer efficiency, flow resistance, and cost-effectiveness.

[0112] System initialization sets the fin spacing to 5mm, height to 10mm, and thickness to 0.5mm as initial parameters. The experience replay library is initialized with a capacity of 100,000 samples. During the reinforcement learning training process, the reinforcement learning model generates new fin parameters through the policy network, such as fin spacing of 4.8mm, height of 10.2mm, and thickness of 0.52mm. These parameters are input into the CFD simulation module, which calculates a 15% increase in heat transfer efficiency and a 5% increase in flow resistance. Based on the reward function, the reward value is calculated and the experience samples are stored. Network update and optimization: 100 samples are randomly sampled from the experience replay library to update the value network and policy network. A loss function is calculated, and network parameters are optimized to gradually improve model performance. System integration and dynamic adjustment: The optimized fin parameters are integrated into the control system. Real-time monitoring of the water flow velocity is 0.5m / s, with uniform temperature distribution and stable pressure. Future water flow conditions and heat load are predicted, and fin parameters are dynamically adjusted to maintain optimal heat transfer efficiency. Prototype Fabrication and Experimental Verification: A microchannel heater prototype was fabricated with a fin pitch of 4.8 mm, a height of 10.2 mm, and a thickness of 0.52 mm. Experimental testing showed a 15% improvement in heat transfer efficiency and a 5% increase in flow resistance, consistent with simulation results. Data Visualization and Performance Evaluation: CFD simulation results were visualized to show temperature distribution and flow velocity variations. Comprehensive evaluation showed that the optimized heater achieved a good balance between heat transfer efficiency and flow resistance.

[0113] Through the above-mentioned examples, the present invention successfully implements reinforcement learning optimization of multiphase heat transfer in direct drinking water dispensers, dynamically adjusts fin structural parameters, and significantly improves heat transfer efficiency. The system modules are rationally designed, work together efficiently, and have broad application prospects and commercial value.

[0114] In a method for improving the heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning of the present invention, a simulation input set of multi-physical field data is established, multiphase heat transfer theory is used as a basic model, a three-dimensional simulation model of a microchannel heater of a direct drinking water machine is constructed based on CFD software, and high-precision thermal-flow field coupling simulation is performed on different fin structure topologies; a reinforcement learning agent model with DDPG as the core is constructed, the environmental state is set as the CFD simulation result, the action space is the fin geometric parameters, and the reward function is the objective function of the weighted summation of comprehensive heat transfer performance indicators; through the cyclic interaction between the DDPG agent model and the CFD simulation, autonomous exploration and updating of structural parameters are achieved, and the CFD simulation feeds back the heat transfer performance indicators to form a closed-loop learning process until convergence; the optimal fin structure topology design obtained by DDPG reinforcement learning is selected, and the corresponding microchannel heater prototype is constructed and experimentally verified; the optimization strategy is integrated into the control system, and the water flow state and heat load are predicted in combination with real-time sensor data, which has the effect of more effectively utilizing steam or bubbles and stably controlling the gas-liquid mixing state in the multiphase heat transfer process.

[0115] The present invention also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for improving the heat transfer efficiency of the above-mentioned direct drinking water machine multiphase flow heat reinforcement learning.

[0116] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for improving the heat transfer efficiency of the above-mentioned direct drinking water machine multiphase flow heat reinforcement learning are implemented.

[0117] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0118] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0119] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning, characterized in that: include: A simulation input set of multi-physics field data was established. Using multiphase heat transfer theory as the basic model, a three-dimensional simulation model of a microchannel heater for a direct drinking water dispenser was constructed using CFD software. High-precision thermal-fluid field coupling simulations were performed for different fin structure topologies. A reinforcement learning agent model based on DDPG was constructed, with the environment state set as the CFD simulation results, the action space as the fin geometric parameters, and the reward function as the objective function of the weighted sum of comprehensive heat transfer performance indicators. Through the cyclic interaction between the DDPG agent model and CFD simulation, the structural parameters are autonomously explored and updated, and the CFD simulation provides feedback on the heat transfer performance indicators, forming a closed-loop learning process until convergence; The optimal fin structure topology design obtained by DDPG reinforcement learning was selected, and the corresponding microchannel heater prototype was constructed and experimentally verified. The optimization strategy is integrated into the control system and combined with real-time sensor data to predict water flow status and heat load.

2. The method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to claim 1 is characterized in that: The steps of establishing a simulation input set of multi-physics field data and using multiphase heat transfer theory as the basic model include collecting basic parameters of the direct drinking water machine to ensure the accuracy and consistency of the data, including parameters related to thermal-flow field coupling, selecting appropriate CFD software, and becoming familiar with its operating interface and functional modules, and using CAD software or the modeling tools provided by the CFD software to construct a three-dimensional geometric model of the microchannel heater of the direct drinking water machine; Mesh the 3D model, select the mesh type, select the flow model, consider the turbulent characteristics of the flow, set the convection heat transfer model, set the fluid inlet velocity, temperature, and pressure parameters, set the outlet pressure or flow, and set the thermal boundary conditions of the heater wall; Select the solver, set the discretization method, start the CFD solver, perform thermal-flow field coupling simulation, monitor the residual changes and physical quantity distribution during the calculation process, use the CFD software's post-processing tools to analyze the simulation results, parameterize the fin's geometric parameters to facilitate subsequent optimization, simulate different fin structure topologies, and analyze their impact on heat transfer performance and flow resistance.

3. The method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to claim 1 is characterized in that: The step of constructing a reinforcement learning agent model with DDPG as the core and setting the environment state as the CFD simulation result includes defining a reinforcement learning environment, where the environment state is determined by the CFD simulation results, including key thermal-flow field parameters, and the action space is defined as the geometric parameters of the fin, which are used to describe the structural topology of the fin; The reward function is a weighted sum of comprehensive heat transfer performance indicators, which is used to guide the learning direction of the agent. The reward function is as follows; R=w1·E heat +w2·(-F flow )+w3·C compact +w4(-C cost ), Among them, w1, w2, w3 and w4 are weight coefficients, E heat is the heat transfer efficiency, F flow is the flow resistance, C compact For compact structure, C cost is the manufacturing cost; A deep learning framework was selected to implement the DDPG algorithm. By randomly sampling the fin structure topology, CFD simulation was performed to collect initial experience samples. Based on the current state, actions were generated through the policy network, and noise was added to promote exploration. The generated actions were input into the CFD simulation model for thermal-flow field coupling simulation to obtain the next state and reward. The trend of the reward function during training was monitored to determine whether the agent had converged. After the agent converged, the optimal policy network was used to generate the optimal fin geometric parameters. Based on the optimal fin structure design, a microchannel heater prototype was constructed, and experimental tests were carried out to measure the actual heat transfer efficiency and flow resistance performance indicators.

4. The method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to claim 1 is characterized in that: The steps of realizing autonomous exploration and updating of structural parameters through cyclic interaction between the DDPG agent model and the CFD simulation include initializing the fin structural parameters of the microchannel heater of the direct drinking water machine, using random initial values ​​or an initial design based on experience, performing CFD simulation by randomly sampling the fin structural topology, and collecting initial empirical samples; Based on the current state, the policy network generates fin geometry parameters. Noise is added to the generated actions to increase randomness and avoid premature convergence. The generated fin geometry parameters are input into the CFD simulation model to perform a thermal-fluid field coupling simulation. The fluid flow and heat transfer performance of the microchannel heater under different fin structures are simulated. The reward function value is calculated based on the simulation results. The reward function comprehensively considers the heat transfer efficiency and flow resistance performance indicators. Randomly sample a batch of experience samples from the experience replay library, use the sampled experience batches to update the value network, and minimize the error between state and action value; during the training process, monitor the changing trend of the reward function to determine whether the agent has converged.

5. The method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to claim 1 is characterized in that: The steps of selecting the optimal fin structure topology design obtained by DDPG reinforcement learning, building the corresponding microchannel heater prototype, and conducting experimental verification include finding the optimal strategy through collaborative optimization of the policy network and the value network, ensuring a clear understanding of the basic principles and implementation details of the DDPG algorithm, including network structure, loss function, and optimization method; During DDPG training, we continuously monitor the changes in the reward function and record the fin geometry parameters when the reward function reaches its maximum value. We then use CAD software to perform detailed design based on the optimal fin parameters to ensure that the design meets actual manufacturing requirements. Based on the design drawings, we then use the selected manufacturing process to produce a prototype. Determine the experimental objectives, such as measuring heat transfer efficiency and flow resistance performance indicators, conduct actual tests according to the experimental plan, record the experimental data, compare the experimental results with the CFD simulation results, and verify the accuracy of the optimization model.

6. The method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to claim 5 is characterized in that: The steps of integrating the optimization strategy into the control system and predicting the water flow state and heat load in combination with real-time sensor data include determining the main objectives of the control system, including maximizing heat transfer efficiency, minimizing flow resistance, and maintaining stable temperature control, and installing sensors such as flow sensors, temperature sensors, and pressure sensors; Collect sensor data in real time, record water flow velocity, temperature distribution, and pressure change parameters, and pre-process the collected data, including denoising, normalization, and feature extraction, for subsequent analysis and prediction; Select an appropriate time series prediction model and train it using historical data to ensure that the model can capture the changing trends of water flow conditions and heat loads. Embed the optimal fin parameters obtained from DDPG reinforcement learning into the control system to ensure that the control system can automatically adjust the fin parameters to maintain optimal heat transfer performance under different water flow and heat load conditions. Design a closed-loop control system to monitor sensor data in real time and predict future water flow conditions and heat loads. Based on the prediction results, dynamically adjust the heater's operating parameters, such as the angle and spacing of the fins, to optimize heat transfer performance and flow resistance. Test the control system in an actual operating environment to verify its prediction and control accuracy.

7. A system for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning, characterized in that: include: Reinforcement learning model module, used to generate the action space of fin structure parameters, and realize autonomous exploration and update of fin structure parameters through the coordinated optimization of policy network and value network; The CFD simulation module is used to perform thermal-flow field coupling simulation on the fin structure parameters generated by the reinforcement learning model and provide feedback on heat transfer performance indicators; The experience replay library module is used to store experience samples obtained during the interaction between the reinforcement learning model and the CFD simulation module, including state, action, reward, and next state; The optimization module is used to update the policy network and value network of the reinforcement learning model according to the heat transfer performance indicators fed back by the CFD simulation module until convergence; The control system integration module is used to integrate the optimal fin structure parameters obtained by the optimization module into the control system of the direct drinking water machine, combine real-time sensor data to predict the water flow state and heat load, and dynamically adjust the fin structure parameters to maintain the best heat transfer efficiency.

8. The system for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to claim 7 is characterized in that: The system also includes a geometric modeling submodule for constructing a three-dimensional geometric model of the microchannel heater based on the fin structure parameters generated by the reinforcement learning model; a meshing submodule for meshing the geometric model to generate a computational mesh suitable for CFD simulation; and a solver submodule for solving the thermal-flow field coupling equations to calculate performance indicators such as heat transfer efficiency and flow resistance. The result output submodule is used to output the result data of CFD simulation, including temperature distribution, flow velocity distribution, and pressure distribution; The reward function design submodule is used to design a reward function that comprehensively considers heat transfer efficiency and flow resistance; The target network update submodule is used to gradually update the target value network and target policy network through a soft update strategy to ensure the stability of the target network; The network training submodule is used to batch update the value network and policy network through samples in the experience replay library to minimize the loss function; the sensor data acquisition submodule is used to collect parameters such as water flow rate, temperature distribution, and pressure change of the direct drinking water machine in real time; The state prediction submodule is used to predict future water flow states and heat loads based on real-time sensor data; the dynamic adjustment submodule is used to dynamically adjust the fin structure parameters of the microchannel heater according to the prediction results to maintain optimal heat transfer efficiency.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for improving heat transfer efficiency of a direct drinking water machine through multiphase heat transfer reinforcement learning according to any one of claims 1 to 6 are implemented.

Images