Multi-type heat exchanger forward optimization design method and device based on reinforcement learning agent cluster

Abstract

This invention provides a method and apparatus for forward optimization design of multi-type heat exchangers based on reinforcement learning agent clusters, relating to the field of heat exchanger optimization design technology. The method includes the following steps: generating standardized datasets for each type of heat exchanger and storing them in a multi-type heat exchanger database; constructing reinforcement learning agents and performing offline training to generate type-specific reinforcement learning agent clusters; if an initial structure is included, using the optimized geometric structure parameters output by the reinforcement learning agents as candidate optimization schemes; if no initial structure is included, generating multiple candidate optimization schemes for each type of heat exchanger; evaluating the performance of each candidate optimization scheme, ranking the evaluation results, and outputting the candidate optimization scheme with the highest score as the final recommended design result. This method improves the efficiency of heat exchanger optimization design by using unified modeling of multiple types of heat exchangers and collaborative forward output from reinforcement learning agent clusters.

Description

Technical Field

[0001] This invention relates to the field of heat exchanger optimization design technology, and in particular to a method and apparatus for forward optimization design of multiple types of heat exchangers based on reinforcement learning agent clusters. Background Technology

[0002] Heat exchangers are key components in the thermal management system of aero-engines, and their design directly affects the energy efficiency, structural reliability, weight, volume, and overall aircraft compatibility of the thermal management system. As aero-engines evolve towards higher thrust-to-weight ratios, higher power densities, wider flight envelopes, and compact, lightweight designs, thermal management systems face higher heat dissipation demands and more complex operating environments, placing higher requirements on the comprehensive performance of heat exchangers. Modern heat exchanger design has gradually evolved from traditional single-function heat transfer performance optimization to a multi-objective, multi-constraint, and multi-disciplinary coupled optimization problem that considers factors such as flow heat transfer performance, structural strength and lifespan, manufacturing processes, and assembly constraints. Against this backdrop, developing efficient and intelligent heat exchanger optimization design methods is of great significance for improving the thermal management level of aero-engines.

[0003] Existing heat exchanger optimization designs typically employ iterative optimization methods based on performance evaluation models. First, an initial structural scheme is determined based on design conditions, performance requirements, and boundary constraints. Then, empirical correlations, numerical simulations, experimental data, or surrogate models are used to establish the mapping relationship between structural parameters and performance. Finally, multiple rounds of iterative optimization are performed using manual parameter tuning, gradient optimization, or heuristic optimization algorithms to obtain optimal design parameters. This type of method belongs to a performance evaluation-driven reverse optimization model, continuously evaluating the performance of candidate schemes and adjusting design parameters until the design requirements are met. However, the above design process relies heavily on performance evaluation and iterative optimization, making it difficult to directly generate optimized schemes from design conditions, resulting in low design efficiency. Furthermore, existing design methods are often applicable to specific operating conditions and optimization objectives, exhibiting limited generalization ability.

[0004] In recent years, the development of artificial intelligence technology, especially reinforcement learning, has provided new solutions for complex structural optimization. For heat exchanger optimization design, the comprehensive performance changes after adjusting structural parameters can be used as reinforcement learning reward signals. Historical simulation and experimental data are then used to train a reinforcement learning agent, enabling it to gradually learn the direct mapping relationship between design conditions and optimization decisions, thus forming a positive decision-making capability for heat exchanger optimization design. Compared to traditional iterative optimization methods, the positive optimization design agent built based on reinforcement learning can directly generate optimization schemes based on design conditions, potentially reducing iterative calculations during the optimization process and effectively improving design efficiency.

[0005] However, existing applications of reinforcement learning in heat exchanger design still have the following shortcomings: First, they mostly model single-type heat exchangers separately, lacking a unified parameterized description and cross-type organization method applicable to multiple types of heat exchangers; second, even when multiple agents are trained separately, they mostly exist in parallel, lacking agent matching, scheduling, collaborative application, and unified optimization mechanisms for design tasks; third, the utilization of subsequent design verification results is usually discontinuous, and a closed-loop continuous evolution mechanism of verification result feedback—incremental update or retraining—capability improvement has not yet been formed. Therefore, it is urgent to propose a forward optimization design method for multiple types of heat exchangers based on reinforcement learning agent cluster collaboration and continuous evolution, to achieve unified construction, management, intelligent scheduling, and collaborative application of agents for different types of heat exchangers, thereby improving heat exchanger design efficiency and design level. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a forward optimization design method for multiple types of heat exchangers based on a cluster of reinforcement learning agents, to solve the technical problems in the prior art, such as low efficiency in heat exchanger optimization design, independent intelligent design methods for different types of heat exchangers, lack of unified scheduling, coordination, and continuous evolution mechanisms among multiple agents, and inability to automatically generate design schemes without initial structure. The method includes: Historical design data of multiple types of heat exchangers are acquired and described using a unified parameterization method. Standardized datasets for each type of heat exchanger are generated and stored in a multi-type heat exchanger database. The unified parameterization method includes common description parameters and characteristic description parameters. For each type of heat exchanger, the corresponding standardized dataset is extracted from the multi-type heat exchanger database, reinforcement learning agents are constructed and trained offline, and the trained heat exchanger agents of each type are encapsulated and registered to generate a reinforcement learning agent cluster for multi-type heat exchangers. The system receives a heat exchanger optimization design task, determines whether it includes an initial structure, and if it does, identifies the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, matches and calls the corresponding type of reinforcement learning agent, using the output optimized geometric structure parameters as candidate optimization schemes. If the initial structure is not included, for each type of heat exchanger, it generates candidate initial structure schemes based on the common description parameters and the characteristic description parameters, calls the corresponding type of agent to output the optimized geometric structure parameters, and generates multiple candidate optimization schemes. The performance of candidate optimization schemes is evaluated. When there are multiple candidate optimization schemes, the evaluation results of the multiple candidate optimization schemes are ranked, and the candidate optimization scheme with the highest score is output as the final recommended design result. The design and verification are repeatedly optimized, and the generated verification data is added to the database of the multi-type heat exchangers. The agent is then retrained. When the optimization effect of the retrained agent is better than that of the old agent before training, the new agent is used to replace the old agent in the reinforcement learning agent cluster and the version update information is recorded.

[0007] This invention also provides a forward optimization design device for multiple types of heat exchangers based on a reinforcement learning agent cluster, to solve the technical problems in the prior art such as low efficiency in heat exchanger optimization design, independent intelligent design methods for different types of heat exchangers, and lack of unified scheduling, coordination, and continuous evolution mechanisms among multiple agents. The device includes: The data standardization module is used to acquire historical design data of multiple types of heat exchangers, perform unified parameterization description, generate standardized datasets of each type of heat exchanger, and store them in a multi-type heat exchanger database. The unified parameterization description includes common description parameters and characteristic description parameters. The intelligent agent cluster construction module is used to extract the corresponding standardized dataset from the multi-type heat exchanger database for each type of heat exchanger, construct reinforcement learning agents and perform offline training, encapsulate and register the trained intelligent agents of each type of heat exchanger, and generate a reinforcement learning agent cluster for multi-type heat exchangers. The task decision module is used to receive the heat exchanger optimization design task, determine whether it includes an initial structure, and if it includes the initial structure, identify the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, match and call the corresponding type of reinforcement learning agent, and use the output optimized geometric structure parameters as candidate optimization schemes. If it does not include the initial structure, for each type of heat exchanger, generate candidate initial structure schemes based on the common description parameters and the characteristic description parameters, call the corresponding type of agent to output the optimized geometric structure parameters, and generate multiple candidate optimization schemes. The scheme evaluation module is used to evaluate the performance of candidate optimization schemes. When there are multiple candidate optimization schemes, the evaluation results of multiple candidate optimization schemes are sorted, and the candidate optimization scheme with the highest score is output as the final recommended design result. The version update module is used to repeatedly optimize and verify the design, supplement the generated verification data into the multi-type heat exchanger database, and retrain the agent. When the optimization effect of the retrained agent is better than that of the old agent before training, the new agent is used to replace the old agent in the reinforcement learning agent cluster and the version update information is recorded.

[0008] Compared with the prior art, the beneficial effects that at least one technical solution adopted in the embodiments of this specification can achieve include at least: By establishing a unified parameterized description framework that includes common and characteristic description parameters, a standardized description of multiple types of heat exchangers is achieved. A reinforcement learning agent cluster for multiple heat exchanger types is constructed, enabling unified encapsulation, registration, matching, and collaborative optimization of agents for different types of heat exchangers within a unified framework. It also supports direct invocation of the corresponding agent's output optimization scheme when an initial structure is available, and automatic generation of multiple candidate initial structure schemes without an initial structure, followed by unified performance evaluation and ranking for optimal selection. This achieves rapid positive output from design conditions to optimized structure schemes. A training data supplementation and agent retraining mechanism is established, which helps improve the performance of the agent cluster as design rounds increase. This effectively solves the technical problems of low efficiency in heat exchanger optimization design, independent intelligent design of multiple heat exchanger types, and lack of unified scheduling, collaboration, and continuous evolution mechanisms among multiple agents in existing technologies. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart of a forward optimization design method for multi-type heat exchangers based on a reinforcement learning agent cluster, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the technical route of the forward optimization design method for multiple types of heat exchangers based on reinforcement learning agent clusters provided in the embodiments of the present invention; Figure 3 This is a flowchart of an embodiment of the present invention for implementing the above-described forward optimization design method for multiple types of heat exchangers based on a reinforcement learning agent cluster; Figure 4 This is a structural block diagram of a forward optimization design device for multiple types of heat exchangers based on a reinforcement learning agent cluster, provided in an embodiment of the present invention. Detailed Implementation

[0011] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0012] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] In this embodiment of the invention, a forward optimization design method for multiple types of heat exchangers based on a reinforcement learning agent cluster is provided, such as... Figure 1 As shown, the method includes: Step S101: Obtain historical design data of multiple types of heat exchangers, perform unified parameterization description, generate standardized datasets of each type of heat exchanger and store them in the multi-type heat exchanger database, wherein the unified parameterization description includes common description parameters and characteristic description parameters; Step S102: For each type of heat exchanger, extract the corresponding standardized dataset from the multi-type heat exchanger database, construct reinforcement learning agents and train them offline, encapsulate and register the trained agents for each type of heat exchanger, and generate a reinforcement learning agent cluster for multi-type heat exchangers. Step S103: Receive the heat exchanger optimization design task, determine whether it includes an initial structure, if it includes the initial structure, identify the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, match and call the corresponding type of reinforcement learning agent, and use the output optimized geometric structure parameters as candidate optimization schemes; if it does not include the initial structure, for each type of heat exchanger, generate candidate initial structure schemes based on the common description parameters and the characteristic description parameters, call the corresponding type of agent to output the optimized geometric structure parameters, and generate multiple candidate optimization schemes; Step S104: Evaluate the performance of candidate optimization schemes. When there are multiple candidate optimization schemes, rank the evaluation results of the multiple candidate optimization schemes and output the candidate optimization scheme with the highest score as the final recommended design result. Step S105: Repeatedly perform optimization design and verification, supplement the generated verification data into the multi-type heat exchanger database, and retrain the agent. When the optimization effect of the retrained agent is better than that of the old agent before training, replace the old agent with the new agent in the reinforcement learning agent cluster and record the version update information.

[0014] In practice, the following steps are used to generate standardized datasets for various types of heat exchangers and store them in a multi-type heat exchanger database: Operating condition parameters, material physical property parameters, heat exchanger performance parameters, and optimization objective description parameters are set for each type of heat exchanger and used as common description parameters. The operating condition parameters include the physical properties of the hot and cold fluids, inlet and outlet temperatures, pressure, and flow rate. The material physical property parameters include the density, specific heat capacity, and thermal conductivity of the solid material. The heat exchanger performance parameters include the heat transfer coefficient and flow resistance coefficient. The optimization objective description parameters include the variable hyperparameters in the optimization objective function and constraint function. Specific geometric parameters for each type of heat exchanger are also set as characteristic description parameters. For plate-fin heat exchangers, the characteristic description parameters include fin height, fin pitch, fin thickness, and flow channel dimensions. For tube-fin heat exchangers, the characteristic description parameters include tube diameter, number of tube rows, fin pitch, and fin thickness. The characteristic description parameters for printed circuit board heat exchangers are... This includes channel width, channel depth, and channel angle. For each type of heat exchanger, operating condition parameters and optimization target description parameters are pre-set, and an initial structure and a modified structure are defined. The heat exchanger performance parameters and corresponding optimization objective function values ​​of the initial structure and the modified structure are obtained through numerical simulation or surrogate modeling. A set of quadruple sample data consisting of the state, action, and reward value corresponding to the initial structure and the state corresponding to the modified structure is obtained through data post-processing. The state includes common description parameters and characteristic description parameters, the action includes geometric structural parameters from the initial structure to the modified structure, and the reward value is calculated based on the change in the optimization objective function value before and after the structural change. The quadruple sample data corresponding to each type of heat exchanger are classified to generate a multi-type heat exchanger database.

[0015] In specific implementation, the reward in the quadruple sample data is calculated in the following way through the following steps: When the modified structure does not meet the constraints, the reward value is set to a preset negative value; when the modified structure meets the constraints, the reward value is set to the reduction in the value of the optimization objective function before and after the structural change, wherein the value of the optimization objective function is a function of the heat exchanger performance parameters, and the constraints include geometric parameter range constraints and weight constraints.

[0016] In specific implementation, the following steps are used to extract the corresponding standardized dataset from the multi-type heat exchanger database, construct a reinforcement learning agent, and perform offline training: For each type of heat exchanger, a reinforcement learning agent is constructed. The reinforcement learning agent employs an offline reinforcement learning algorithm. Its input is a state vector corresponding to the initial structure, formed by concatenating operating condition parameters, initial geometric structure parameters, initial heat exchanger performance parameters, and optimization target description parameters. The output is an action vector, which includes the optimized geometric structure parameters. Quadruple sample data corresponding to the current heat exchanger type is extracted from the multi-type heat exchanger database. Using the state corresponding to the initial structure in the quadruple sample data as input and the action as the expected output, the agent is trained using an offline reinforcement learning algorithm, establishing a positive mapping relationship from design conditions to optimized structural parameters. Training is performed separately for each type of heat exchanger, resulting in trained reinforcement learning agents for each type. All trained reinforcement learning agents are uniformly encapsulated and registered to an agent cluster manager. The agent cluster manager maintains the type label, version information, and calling interface of each agent, generating a reinforcement learning agent cluster for multi-type heat exchangers.

[0017] In practice, the following steps are used to perform performance verification and evaluation when the received heat exchanger optimization design task includes an initial structure: When the agent cluster receives a heat exchanger optimization design task containing the initial structure, it identifies the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, matches and calls the corresponding type of reinforcement learning agent, and uses the output optimized geometric structure parameters as candidate optimization schemes; it calls numerical simulation or a surrogate model to verify and evaluate the candidate optimization schemes, and obtains the verified heat exchanger performance parameters of the candidate optimization schemes; it compares the verified heat exchanger performance parameters with the initial heat exchanger performance parameters in the initial structure, calculates the performance improvement before and after optimization, and feeds back the design results.

[0018] In practice, the following steps are used to generate multiple candidate optimization schemes: When the agent cluster receives a heat exchanger optimization design task that does not include the initial structure, for each type of heat exchanger, candidate initial structure schemes are generated based on the common description parameters and the characteristic description parameters. The corresponding agent is then invoked to generate multiple candidate optimization schemes: the allowable variation range of the characteristic description parameters for each type of heat exchanger is read from the multi-type heat exchanger database; for each type of heat exchanger, a set of geometric structure parameters is randomly sampled within the allowable variation range, and these geometric structure parameters are combined with the common description parameters in the current design task to construct candidate initial structure schemes for the current type of heat exchanger; numerical simulation or a surrogate model is invoked to evaluate the performance of each candidate initial structure scheme, obtaining the heat exchanger performance parameters for each candidate initial structure scheme; the operating condition parameters and optimization target description parameters from the common description parameters, the geometric structure parameters of the candidate initial structure schemes, and the heat exchanger performance parameters are concatenated to form the state vector corresponding to each candidate initial structure scheme; the state vectors corresponding to each candidate initial structure scheme are input to the corresponding type of agent, and each agent outputs the optimized geometric structure parameters, generating multiple candidate optimization schemes.

[0019] In specific implementation, the following steps are used to evaluate the performance of each candidate optimization scheme when the intelligent agent cluster generates multiple candidate optimization schemes, rank the evaluation results, and output the candidate optimization scheme with the highest score as the final recommended design result: The performance of each candidate optimization scheme is evaluated by numerical simulation or surrogate model to obtain the heat exchanger performance parameters corresponding to each candidate optimization scheme. The comprehensive performance score of each candidate optimization scheme is calculated based on the heat exchanger performance parameters, wherein the comprehensive performance score is calculated by weighting the heat exchange performance, flow resistance performance and constraint satisfaction. The candidate optimization schemes are sorted from high to low according to the comprehensive performance score, and the candidate optimization scheme with the highest comprehensive performance score is selected as the final recommended design result output.

[0020] In practice, the following steps are used to output the candidate optimization scheme with the highest score as the final recommended design result and then retrain: The final recommended design result is verified by numerical simulation. The verification data is processed according to the unified parameterization description into four-tuple verification sample data consisting of the state, action, reward value, and state corresponding to the changed structure corresponding to the initial structure. The four-tuple verification sample data is added to the dataset of the corresponding type of heat exchanger in the multi-type heat exchanger database. The amount of new sample data for the corresponding type of heat exchanger is counted. When the amount of new sample data reaches a preset threshold, the heat exchanger agent corresponding to each type of heat exchanger is retrained.

[0021] In practice, the following steps are used to retrain the heat exchanger agent corresponding to each type of heat exchanger: Extract the complete dataset of the current type of heat exchanger from the multi-type heat exchanger database. The complete dataset includes historical quadruplet sample data and quadruplet validation sample data. Use the complete dataset as the training set to retrain the agent for the current type of heat exchanger, generating a new version of the agent. Record the training sample range, update time, and validation effect corresponding to the new version of the agent. Compare the performance of the new version with the current old version on a preset validation metric. Only if the new version is better than the old version on the preset validation metric is the new version replaced with the old version and the registration information of the agent cluster updated. If the new version is not better than the old version on the preset validation metric, the current old version is maintained and the new version is discarded. The quadruplet validation sample data is retained in the multi-type heat exchanger database and merged with the subsequently accumulated validation sample data to generate the quadruplet validation sample data used during retraining.

[0022] I. Overall Process Overview like Figure 2 As shown, the forward optimization design method for multiple types of heat exchangers provided by this invention mainly includes four stages: dataset generation stage, multi-agent construction and training stage, multi-agent cluster deployment and collaborative application stage, and multi-agent continuous evolution stage. Specifically, the dataset generation stage is used to form standardized reinforcement learning datasets corresponding to different types of heat exchangers; the multi-agent construction and training stage is used to train agents for each type of heat exchanger separately; the multi-agent cluster deployment and collaborative application stage is used for agent invocation, scheme generation, and optimization for actual design tasks; and the multi-agent continuous evolution stage is used to supplement the dataset with validated new data and drive the continuous improvement of agent capabilities.

[0023] The following is combined Figure 3 The specific embodiments of the present invention will be described in further detail below.

[0024] II. Specific Implementation Methods in the Dataset Generation Phase 2.1 Unified Parameterized Description In this embodiment, a unified parameterized description is first performed for different types of heat exchangers. The unified parameterized description includes at least two parts: common description parameters and characteristic description parameters.

[0025] Common descriptive parameters include: Operating parameters: physical properties of hot and cold fluids (density, specific heat capacity, thermal conductivity, viscosity), inlet and outlet temperatures, pressure, flow rate, etc. Physical properties of solid materials: density, specific heat capacity, thermal conductivity, etc. Heat exchanger performance parameters: heat transfer coefficient, flow resistance coefficient, heat exchanger weight, etc. Optimization target description parameters: Variable hyperparameters in the optimization objective function (such as weight coefficient thresholds, limit values, etc.) are used to characterize different optimization requirements.

[0026] The characteristic description parameters are proprietary geometric parameters for different types of heat exchangers.

[0027] For example, for plate-fin heat exchangers, characteristic description parameters include fin height, fin pitch, fin thickness, flow channel dimensions (width and height of the cold side flow channel, width and height of the hot side flow channel), baffle thickness, etc.

[0028] For tube-fin heat exchangers, characteristic parameters include tube diameter (outer diameter, inner diameter), tube wall thickness, number of tube rows, tube spacing (lateral spacing, longitudinal spacing), fin pitch, fin thickness, etc.

[0029] For printed circuit board heat exchangers (PCHEs), characteristic description parameters include microchannel width, channel depth, channel angle, channel pitch, number of plates, and plate thickness.

[0030] The above examples are only used to illustrate the characteristic parameters of different types of heat exchangers and do not constitute a limitation on the scope of protection of this invention. The method described in this invention is also applicable to other types of heat exchangers with heat transfer units, such as shell-and-tube heat exchangers, spiral plate heat exchangers, and heat pipe heat exchangers.

[0031] 2.2 Optimization Problem Modeling Based on a unified parametric description, a mathematical model is constructed for the heat exchanger optimization problem. The optimization problem is described as follows: under certain operating conditions and constraints, optimize the geometric design parameters of the heat exchanger to achieve a specified optimization objective, typically expressed as minimizing the objective function.

[0032] The objective function J can be expressed as a function of the heat exchanger performance parameters: J = f(Δp, Q, V, m, ...) Where Δp is the flow pressure drop, Q is the heat exchange capacity, V is the heat exchanger volume, m is the heat exchanger weight, and "..." can be other heat exchanger performance parameters.

[0033] The constraints include: Geometric parameter range constraints: All geometric parameters must be within the physically achievable range, such as fin height not less than the minimum feature size of additive manufacturing or machining, and pipe diameter not less than the minimum manufacturable pipe diameter, etc. Weight constraint: The total weight of the heat exchanger shall not exceed the design allowable value; The variable hyperparameters (such as weighting coefficients, thresholds, and limit values) in the objective function and constraints are defined as optimization objective description parameters to characterize different optimization requirements.

[0034] 2.3 Generation of Quadruple Sample Data for Reinforcement Learning In the context of optimizing various types of heat exchangers, we define the state, action, and reward in reinforcement learning.

[0035] The status includes: Common descriptive parameters include operating condition parameters (physical properties of hot and cold fluids, inlet and outlet temperatures, pressures, flow rates, etc.), heat exchanger performance parameters (heat transfer coefficient, flow resistance coefficient, etc.), and optimization target descriptive parameters (variable hyperparameters). Geometric parameters in the characteristic description parameters (property geometric parameters for each type of heat exchanger). The action only includes characteristic description parameters, namely the geometric structural parameters of each type of heat exchanger to be optimized. Specifically, the action is the target geometric parameter value required to change from the current structure to the target structure.

[0036] The reward value is calculated according to the following rules: When the modified structure does not meet the constraints, the reward value is a negative value with a large absolute value (such as -100) to punish the behavior that violates the constraints. When the modified structure satisfies the constraints, the reward value is the reduction in the value of the objective function before and after the structural change, i.e.: R = f(s initial ) - f(s changed ) Where, f(s) initial f(s) represents the optimization objective function value corresponding to the initial structure. changed The value of the objective function corresponding to the changed structure is denoted as . A positive reduction indicates an improvement in the optimization objective, resulting in a positive reward; a negative reduction indicates a deterioration in the optimization objective, resulting in a negative reward.

[0037] For example, if the optimization objective is to minimize the pressure drop / 100, and the initial structural pressure drop is 1000 Pa, changing the structural pressure drop to 800 Pa results in a reduction of 200 Pa and a reward value of +2; if the structural pressure drop is changed to 1100 Pa, the reduction is -100 Pa and the reward value is -1. Those skilled in the art can adjust the sign and calculation method of the reward value according to the nature of the actual optimization objective (minimization or maximization).

[0038] The following methods are used to generate datasets for multiple types of heat exchangers: For a specific type of heat exchanger, pre-defined operating condition parameters and optimization objective description parameters are established. Then, within the allowable parameter variation range, an initial structure and a modified structure are defined—the initial structure serves as the baseline before the change, and the modified structure serves as the comparison after the change. Both represent a set of specific geometric parameter values ​​for this type of heat exchanger. The heat exchanger performance parameters and corresponding optimization objective function values ​​are obtained through numerical simulation (e.g., CFD calculations) or surrogate models (e.g., response surface models based on simulation or experimental data fitting, Kriging models, etc.). Next, data post-processing yields a set of quadruplet sample data consisting of the state, action, reward value corresponding to the initial structure, and the state corresponding to the modified structure.

[0039] Taking a plate-fin heat exchanger as an example, the given operating conditions are: hot fluid inlet temperature 200℃, flow rate 0.5 kg / s; cold fluid inlet temperature 40℃, flow rate 0.8 kg / s; the initial structural geometric parameters are set as follows: fin height 6 mm, fin pitch 3 mm, fin thickness 0.3 mm, and flow channel width 4 mm; the modified structural geometric parameters are set as follows: fin height 7 mm, fin pitch 2.8 mm, fin thickness 0.25 mm, and flow channel width 4.2 mm. Numerical simulation yields a pressure drop of 850 Pa and a heat transfer of 45 kW for the initial structure, and a pressure drop of 720 Pa and a heat transfer of 48 kW for the modified structure. If the optimization objective is a weighted combination of minimizing the pressure drop and maximizing the heat transfer, the corresponding objective function value can be calculated, thus obtaining the reward value for this modification.

[0040] Repeating the above process—that is, continuously generating new "initial structure + changed structure" combinations within the allowed range—can yield a large amount of quadruple sample data. Repeating this process multiple times can obtain datasets corresponding to various types of heat exchangers, which can then be categorized and organized to form a multi-type heat exchanger database for storing these datasets.

[0041] 2.4 Explanation of Dataset Size To ensure the effectiveness of agent training for each type of heat exchanger, the standardized dataset for each type of heat exchanger should contain a sufficient number of quadruplet samples. Preferably, the number of samples for each type of heat exchanger should be no less than 200. For heat exchanger types with insufficient samples, the dataset can be expanded in the following ways: Perform more intensive random sampling within the allowed parameter variation range; Samples are generated in batches using numerical simulation (CFD); Synthetic samples are generated using surrogate models (such as response surface models based on simulation or experimental data fitting, Kriging models, etc.).

[0042] III. Specific Implementation Methods for the Agent Construction and Training Phase 3.1 Intelligent Agent Architecture Design A reinforcement learning agent is built for each type of heat exchanger. The agent employs offline reinforcement learning algorithms, such as Deep Deterministic Policy Gradient (DDPG), Deep Q-Network (DQN), or Soft Actor-Critic (SAC). Preferably, DDPG or SAC is used for continuous action spaces (geometric parameters are continuous values); DQN is used for discrete action spaces (geometric parameters are discrete options).

[0043] The agent's input is the state corresponding to the initial structure, and this state vector is formed by concatenating the following parts: Operating condition parameters (such as physical properties of hot and cold fluids, inlet and outlet temperatures, pressure, flow rate, etc.); Initial geometric parameters (values ​​of proprietary geometric parameters for this type of heat exchanger); Initial heat exchanger performance parameters (heat transfer coefficient, flow resistance coefficient, etc.); Optimize the target description parameters (optimize the variable hyperparameters in the objective function and constraint function).

[0044] The output of the intelligent agent is the action, which is the optimized geometric parameters. Taking a plate-fin heat exchanger as an example, the output is the optimized fin height, fin pitch, fin thickness, and flow channel size.

[0045] 3.2 Network Structure Design For agents employing the DDPG algorithm, both their Actor and Critic networks utilize fully connected neural network structures. For example, the Actor network structure is as follows: Input layer: State vector dimension (taking a plate-fin heat exchanger as an example, approximately 15-20 dimensions); Hidden layer 1: 128 neurons, ReLU activation function; Hidden layer 2: 128 neurons, ReLU activation function; Output layer: Action vector dimension (number of geometric parameters, approximately 4-6 dimensions for a plate-fin heat exchanger), Tanh activation function (scales the output to the range of [-1,1] and then maps it to the allowed parameter range).

[0046] The Critic network structure is as follows: Input layer: concatenation of state vector and action vector (approximately 20-26 dimensions); Hidden layer 1: 128 neurons, ReLU activation function; Hidden layer 2: 128 neurons, ReLU activation function; Output layer: 1-dimensional, outputs Q-value.

[0047] For an agent employing the DQN algorithm, its Q-network structure is as follows: Input layer: state vector dimension; Hidden layer 1: 128 neurons, ReLU activation function; Hidden layer 2: 128 neurons, ReLU activation function; Hidden layer 3: 128 neurons, ReLU activation function; Output layer: Action space size (number of discrete actions), outputs the Q value of each action.

[0048] The hyperparameter settings for the agent are as follows: Learning rate: 1×10 -4 Up to 1×10 -3 (Preferably, the Actor network is 1×10) -4 The Critic network is 1×10 -3 ); Discount factor γ: 0.95 to 0.99 (preferably, γ = 0.99); Mini-batch sampling size: 32 to 128; The target network soft update coefficient τ ranges from 0.001 to 0.01.

[0049] 3.3 Offline Training Process The agent for each type of heat exchanger is trained using the corresponding quadruple sample data for that type of heat exchanger. The training process employs an offline reinforcement learning algorithm. The specific training steps are as follows: Step 3.3.1: Initialize the parameters of the Actor network and Critic network (or Q network) of the agent, and initialize the target network (its parameters are the same as those of the main network).

[0050] Step 3.3.2: Extract the quadruple sample dataset corresponding to the current type of heat exchanger from the multi-type heat exchanger database. For each sample in the dataset (initial structural state, action, reward value, change in structural state), input the initial structural state into the agent to obtain the action output by the agent.

[0051] Step 3.3.3: Calculate the difference between the agent's output action and the actual action in the sample, and update the network parameters based on the reward value. For the DDPG algorithm, the Critic network is updated by minimizing the loss function (mean square Bellman error) of the Critic network, and then the Actor network is updated by the policy gradient.

[0052] Step 3.3.4: Update the target network parameters using the soft update method.

[0053] Step 3.3.5: After completing one round of traversal of all samples, repeat multiple rounds of iterative training until the performance of the agent converges.

[0054] During training, conservative optimization techniques of offline reinforcement learning (such as CQL, BCQ, etc.) are used to avoid the agent overestimating actions outside the dataset, thereby improving training stability and generalization ability.

[0055] After completing the above training for each type of heat exchanger, intelligent optimization design agents corresponding to each type of heat exchanger can be obtained. Each agent learns the positive mapping relationship from the design conditions of that type of heat exchanger to the optimized structural parameters—that is, given a set of operating conditions and optimization objectives, the agent can directly output the optimized geometric structural parameter scheme without going through the iterative trial and error process of traditional optimization methods.

[0056] IV. Specific Implementation Methods for Multi-Agent Cluster Deployment and Collaborative Application Phase 4.1 Construction of Intelligent Agent Cluster The trained intelligent agents of various types of heat exchangers are uniformly packaged, registered, managed, and scheduled to form a multi-agent cluster for intelligent design of multiple types of heat exchangers. The specific implementation method is as follows: Unified encapsulation: The network model, input / output interfaces, and preprocessing logic of each agent are packaged into independent executable units. The encapsulation format employs containerization technology (such as Docker) to ensure consistency in operation across different deployment environments. The encapsulated agents expose standardized RESTful APIs or gRPC interfaces, with a unified interface format: input is a JSON-formatted state vector, and output is a JSON-formatted action vector.

[0057] Unified Registration: Each encapsulated agent is registered to the agent cluster manager. Registration information includes: agent ID (unique identifier), heat exchanger type label, version number, training sample range (the operating conditions covered by the training data), training completion time, and validation performance metrics (average reward value on the validation set, prediction accuracy, etc.). The agent cluster manager uses a service registration and discovery mechanism (such as Consul, etcd, or ZooKeeper) to support the dynamic addition, deletion, and updating of agents.

[0058] Unified Management: The agent cluster manager is responsible for maintaining the lifecycle of each agent, including version management and fault recovery. Version management records the update history of each agent and supports version rollback; fault recovery automatically restarts or switches to a standby instance when an agent anomaly is detected.

[0059] Unified Scheduling: The agent cluster manager incorporates task identification, agent matching, agent invocation and execution, and result collaborative optimization functions. The task identification function analyzes whether the design task contains an initial structure, initial heat exchanger type, operating conditions, and optimization target requirements. The agent matching function searches the registry for the corresponding agent based on the identified heat exchanger type. The agent invocation and execution function converts the design conditions into state vectors and invokes the agent interface. The result collaborative optimization function comprehensively evaluates and ranks multiple candidate solutions.

[0060] 4.2 Forward Design Pattern with Initial Structure When a new heat exchanger optimization design task is received and the input information includes the initial structure, the system directly enters the forward design mode with the initial structure.

[0061] The specific process is as follows: Step 4.2.1: Analyze the design task and identify the target heat exchanger type. For example, if the design task is "optimize plate-fin heat exchanger", then the target type is identified as "plate-fin".

[0062] Step 4.2.2: Extract initial structural information from the input information, including operating condition parameters (physical properties of hot and cold fluids, inlet and outlet temperatures, pressures, flow rates, etc.), initial geometric structural parameters (fin height, fin pitch, etc. of the current scheme), initial heat exchanger performance parameters (heat transfer coefficient, pressure drop, etc. of the current scheme), and optimization target description parameters.

[0063] Step 4.2.3: Organize the above information according to the unified parameterized description framework to construct the state vector. The format of the state vector is consistent with that used during training.

[0064] Step 4.2.4: Based on the identified target heat exchanger type, the agent cluster manager matches the corresponding agent from the registry, calls the inference interface of the agent, and inputs the state vector into the agent.

[0065] Step 4.2.5: The agent performs forward inference and outputs the optimized geometric parameters. This output is the candidate optimization scheme. The candidate optimization scheme is evaluated using numerical simulation or a surrogate model to obtain the corresponding heat exchanger performance parameters (heat transfer coefficient, pressure drop, volume, weight, etc.). This scheme is then output as the final recommended design result.

[0066] 4.3 Multiple Scheme Generation and Optimization Mode Without Initial Structure When the input information does not contain an initial structure, the system enters a forward design mode without an initial structure. This mode can automatically generate candidate initial structures and complete optimization design and scheme selection even without an initial solution.

[0067] The specific process is as follows: Step 4.3.1: Read the allowable variation range of characteristic description parameters for each type of heat exchanger from the multi-type heat exchanger database. For example, the allowable range for fin height of plate-fin heat exchangers is 4~10mm, and the allowable range for fin pitch is 2~4mm; the allowable range for tube diameter of tube-fin heat exchangers is 8~25mm, and the allowable range for the number of tube rows is 2~6 rows; the allowable range for microchannel width of PCHE is 0.5~2mm, and the allowable range for channel depth is 0.5~3mm.

[0068] Step 4.3.2: For each type of heat exchanger, randomly sample a set of geometric structural parameters within the allowable variation range, and combine them with the common descriptive parameters (operating condition parameters and optimization target descriptive parameters) in the current design task to construct candidate initial structural schemes for the current type of heat exchanger.

[0069] Step 4.3.3: Call numerical simulation or surrogate model to evaluate the performance of each candidate initial structure scheme, obtain the heat exchanger performance parameters of each candidate initial structure scheme, and concatenate the working condition parameters, the geometric structure parameters of the candidate initial structure scheme, the heat exchanger performance parameters and the optimization target description parameters to form the state vector corresponding to each candidate initial structure scheme.

[0070] Step 4.3.4: The agent cluster manager matches and calls the corresponding agents based on the labels of each type of heat exchanger. Each state vector is input to the corresponding agent, and each agent outputs the optimized geometric structure parameters, generating multiple candidate optimization schemes (one candidate optimization scheme corresponds to each heat exchanger type).

[0071] Step 4.3.5: Evaluate and select the best candidate optimization schemes. This includes: using numerical simulation or a surrogate model to evaluate each candidate scheme and obtain the corresponding heat exchanger performance parameters (heat transfer coefficient, pressure drop, volume, weight, etc.); calculating the comprehensive performance score based on the evaluation results. When constraints are not met (e.g., whether geometric parameters are within allowable ranges, weight exceeds limits), the comprehensive performance score is a negative value with a large absolute value (e.g., -100); when constraints are met, the comprehensive performance score takes into account both heat transfer performance (e.g., heat transfer coefficient, heat transfer capacity) and flow resistance performance (e.g., pressure drop, flow resistance coefficient), and is obtained through a weighted summation.

[0072] For example, the overall performance score can be expressed as: Score = w1·Q / Q ref + w2·(Δp ref / Δp) Where Q represents the heat exchange, Δp represents the pressure drop, and values ​​with the ref subscript are reference values ​​(such as the optimal value or design target value in each scheme). w1 and w2 are weighting coefficients (satisfying w1+w2=1), set according to the specific optimization objective. For example, if heat exchange is given priority, w1=0.8 and w2=0.2 can be set.

[0073] Candidate optimization schemes are ranked from highest to lowest based on their comprehensive performance scores, and the candidate optimization scheme with the highest score is output as the final recommended design result. The output includes the recommended heat exchanger type, its corresponding optimized geometric parameters, and expected performance indicators.

[0074] V. Specific Implementation Methods for the Multi-Agent Continuous Evolution Stage 5.1 Verification data feedback In the stage of multi-agent cluster deployment and collaborative application for intelligent design of various types of heat exchangers, the optimized design results output by the multi-agent cluster have been verified by numerical simulation. The data obtained from the above verification are processed into four-tuple verification sample data according to a unified parameterization description, specifically including: the state, action, reward value corresponding to the initial structure, and the state corresponding to the changed structure.

[0075] In this context, the initial structure corresponds to the state vector of the structure before verification, while the changed structure corresponds to the state vector of the structure after the action is performed. The action is the change in geometric parameters from the initial structure to the changed structure. The reward value is calculated according to the aforementioned rules: when the structure does not meet the constraints, a negative value with a large absolute value is assigned; when the structure meets the constraints, the reward value is determined based on the change in the objective function value before and after the structure change.

[0076] The above quadruple verification sample data will be added to the corresponding heat exchanger dataset in the multi-type heat exchanger database.

[0077] 5.2 Retraining Triggering Conditions The number of newly added sample data for the corresponding type of heat exchanger is counted. When the number of newly added data reaches a preset threshold (e.g., 50 new samples), the retraining process of the agent for that type of heat exchanger is triggered.

[0078] The retraining trigger condition is set based on the following considerations: if the threshold is set too low, retraining will be too frequent, increasing computational overhead and potentially causing training instability due to insufficient sample size; if the threshold is set too high, the agent will not be able to learn from new data in a timely manner, slowing down the evolution speed. Preferably, the preset threshold is 30-50 sets of samples, or 15%-25% of the original dataset size. Those skilled in the art can flexibly adjust this threshold according to the actual application scenario and the data accumulation speed.

[0079] 5.3 Agent Retraining and Version Management After triggering retraining, the complete dataset of that type of heat exchanger (including historical quadruplet sample data and newly added quadruplet validation sample data) is extracted from the multi-type heat exchanger database. The agent is then retrained using the complete dataset as the training set to generate a new version of the agent.

[0080] After each retraining session, record the following information corresponding to the new version: Training sample range: The range of operating conditions covered by the dataset used for training (such as temperature range, pressure range, flow rate range, etc.). Update time: Timestamp indicating retraining completed; Validation results: Performance metrics on independent validation sets (such as average reward value, optimization success rate, etc.); The version replacement rules are as follows: Compare the performance of the new version with the current old version on the preset verification metrics (such as the average reward value on the verification set). Only when the new version is better than the old version on the preset verification metrics, replace the old version with the new version and update the registration information (version number, verification effect, etc.) in the agent cluster manager.

[0081] If the new version does not outperform the old version in the preset validation metrics, the current old version will be maintained and the new version will be discarded. The four-tuple validation sample data corresponding to the discarded new version will be retained in the multi-type heat exchanger database and merged with the subsequently accumulated validation sample data for use in the next retraining. Although this version was not adopted as the online version, its corresponding validation data can still provide effective information for subsequent training and will not be treated as discarded data.

[0082] This approach enables the performance of the intelligent agent cluster to improve with each round of use, forming a closed-loop continuous evolution mechanism of "design → verification → sample feedback → retraining → version update".

[0083] Based on the same inventive concept, this invention also provides a forward optimization design apparatus for multiple types of heat exchangers based on reinforcement learning agent clusters, as described in the following embodiments. Since the principle of the problem-solving approach of the forward optimization design apparatus for multiple types of heat exchangers based on reinforcement learning agent clusters is similar to that of the forward optimization design method for multiple types of heat exchangers based on reinforcement learning agent clusters, the implementation of the apparatus can refer to the implementation of the forward optimization design method for multiple types of heat exchangers based on reinforcement learning agent clusters, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0084] Figure 4This is a structural block diagram of a multi-type heat exchanger forward optimization design device based on reinforcement learning agent clusters according to an embodiment of the present invention, such as... Figure 4 As shown, it includes: a data standardization module 401, an intelligent agent cluster construction module 402, a task decision module 403, a solution evaluation module 404, and a version update module 405. The structure is described below.

[0085] The data standardization module 401 is used to acquire historical design data of multiple types of heat exchangers, perform unified parameterization description, generate standardized datasets of each type of heat exchanger and store them in a multi-type heat exchanger database. The unified parameterization description includes common description parameters and characteristic description parameters. The agent cluster construction module 402 is used to extract the corresponding standardized dataset from the multi-type heat exchanger database for each type of heat exchanger, construct reinforcement learning agents and perform offline training, encapsulate and register the trained agents of each type of heat exchanger, and generate a reinforcement learning agent cluster for multi-type heat exchangers. The task decision module 403 is used to receive the heat exchanger optimization design task, determine whether it includes an initial structure, and if it includes the initial structure, identify the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, match and call the corresponding type of reinforcement learning agent, and use the output optimized geometric structure parameters as candidate optimization schemes. If it does not include the initial structure, for each type of heat exchanger, generate candidate initial structure schemes based on the common description parameters and the characteristic description parameters, call the corresponding type of agent to output the optimized geometric structure parameters, and generate multiple candidate optimization schemes. The scheme evaluation module 404 is used to evaluate the performance of candidate optimization schemes. When there are multiple candidate optimization schemes, the evaluation results of multiple candidate optimization schemes are sorted, and the candidate optimization scheme with the highest score is output as the final recommended design result. The version update module 405 is used to repeatedly optimize and verify the design, supplement the generated verification data to the multi-type heat exchanger database, and retrain the agent. When the optimization effect of the retrained agent is better than that of the old agent before training, the new agent is used to replace the old agent in the reinforcement learning agent cluster and the version update information is recorded.

[0086] In one embodiment, the data standardization module includes: The common parameter setting unit is used to set the operating condition parameters, material physical property parameters, heat exchanger performance parameters, and optimization target description parameters for various types of heat exchangers as common description parameters. The operating condition parameters include the physical properties of hot and cold fluids, inlet and outlet temperatures, pressures, and flow rates. The material physical property parameters include the density, specific heat capacity, and thermal conductivity of solid materials. The heat exchanger performance parameters include the heat transfer coefficient and flow resistance coefficient. The optimization target description parameters include the variable hyperparameters in the optimization objective function and constraint condition function. The characteristic parameter setting unit is used to set the proprietary geometric parameters of each type of heat exchanger as characteristic description parameters. Among them, the characteristic description parameters of plate-fin heat exchangers include fin height, fin pitch, fin thickness and flow channel size; the characteristic description parameters of tube-fin heat exchangers include tube diameter, number of tube rows, fin pitch and fin thickness; and the characteristic description parameters of printed circuit board heat exchangers include channel width, channel depth and channel angle. The initial structure setting unit is used to pre-set the operating condition parameters and optimization target description parameters for each type of heat exchanger, and to set an initial structure and a modified structure. The quadruplet sample generation unit is used to obtain the heat exchanger performance parameters and corresponding optimization objective function values ​​of the initial structure and the changed structure through numerical simulation or surrogate model, and to obtain a set of quadruplet sample data consisting of the state, action, reward value corresponding to the initial structure and the state corresponding to the changed structure through data post-processing. The state includes common description parameters and characteristic description parameters, the action includes geometric structure parameters from the initial structure to the changed structure, and the reward value is calculated based on the change in the optimization objective function value before and after the structure change. The database generation unit is used to classify the quadruple sample data corresponding to each type of heat exchanger and generate a multi-type heat exchanger database.

[0087] In one embodiment, the quadruplet sample generation unit is further configured to set the reward value to a preset negative value when the modified structure does not meet the constraints; and to set the reward value to the reduction in the optimization objective function value before and after the structural modification when the modified structure meets the constraints, wherein the optimization objective function value is a function of the heat exchanger performance parameters, and the constraints include geometric parameter range constraints and weight constraints.

[0088] In one embodiment, the agent cluster building module includes: The agent construction unit is used to construct a reinforcement learning agent for each type of heat exchanger. The reinforcement learning agent adopts an offline reinforcement learning algorithm. The input of the reinforcement learning agent is a state vector corresponding to the initial structure. The state vector is formed by concatenating the operating condition parameters, the initial geometric structure parameters, the initial heat exchanger performance parameters, and the optimization target description parameters. The output of the reinforcement learning agent is an action vector, which includes the optimized geometric structure parameters. The agent training unit is used to extract the quadruple sample data corresponding to the current type of heat exchanger from the multi-type heat exchanger database, and use the state corresponding to the initial structure in the quadruple sample data as input and the action as expected output to train the agent using an offline reinforcement learning algorithm, so that the reinforcement learning agent establishes a positive mapping relationship from design conditions to optimized structural parameters. The type-by-type training completion unit is used to train each type of heat exchanger separately, and obtain the training completed reinforcement learning agent corresponding to each type of heat exchanger. The agent cluster generation unit is used to uniformly encapsulate each trained reinforcement learning agent and register it to the agent cluster manager. The agent cluster manager maintains the type label, version information and calling interface of each agent to generate a reinforcement learning agent cluster for multiple types of heat exchangers.

[0089] In one embodiment, the task decision module includes: The candidate optimization scheme generation unit is used to identify the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, match and call the corresponding type of reinforcement learning agent when the agent cluster receives a heat exchanger optimization design task containing the initial structure, and use the output optimized geometric structure parameters as candidate optimization schemes. The optimization scheme verification and evaluation unit is used to call numerical simulation or surrogate model to verify and evaluate the candidate optimization scheme, and obtain the heat exchanger performance parameters after the verification of the candidate optimization scheme. The performance improvement calculation unit is used to compare the verified heat exchanger performance parameters with the initial heat exchanger performance parameters in the initial structure, calculate the performance improvement before and after optimization, and provide the design results to the designers.

[0090] In one embodiment, the task decision module further includes: The task determination and triggering unit without initial structure is used to generate candidate initial structure schemes for each type of heat exchanger based on the common description parameters and the characteristic description parameters when the intelligent agent cluster receives a heat exchanger optimization design task that does not contain the initial structure, and to call the corresponding type of intelligent agent to generate multiple candidate optimization schemes. The allowable variation range reading unit is used to read the allowable variation range of the characteristic description parameters of each type of heat exchanger from the multi-type heat exchanger database; The candidate initial structure scheme construction unit is used to randomly sample a set of geometric structure parameters within the allowable variation range for each type of heat exchanger, and combine the geometric structure parameters with the common description parameters in the current design task to construct a candidate initial structure scheme for the current type of heat exchanger. The state vector construction unit is used to call numerical simulation or surrogate model to evaluate the performance of each candidate initial structure scheme, obtain the heat exchanger performance parameters of each candidate initial structure scheme, and concatenate the working condition parameters and optimization target description parameters in the common description parameters, the geometric structure parameters of the candidate initial structure scheme, and the heat exchanger performance parameters to form the state vector corresponding to each candidate initial structure scheme. The multi-candidate optimization scheme generation unit is used to input the state vectors corresponding to each candidate initial structure scheme into the corresponding type of agent, and each agent outputs the optimized geometric structure parameters to generate multiple candidate optimization schemes.

[0091] In one embodiment, the solution evaluation module includes: The performance evaluation unit is used to call numerical simulation or proxy model to evaluate the performance of each candidate optimization scheme and obtain the heat exchanger performance parameters corresponding to each candidate optimization scheme. The comprehensive performance scoring calculation unit is used to calculate the comprehensive performance score of each candidate optimization scheme based on the heat exchanger performance parameters, wherein the comprehensive performance score is calculated by weighting the heat exchange performance, flow resistance performance and constraint satisfaction. The sorting and output unit, when the agent cluster generates multiple candidate optimization schemes, sorts the multiple candidate optimization schemes from high to low according to the comprehensive performance score, and selects the candidate optimization scheme with the highest comprehensive performance score as the final recommended design result output.

[0092] In one embodiment, the above-described apparatus further includes a data return module.

[0093] In one embodiment, the data backflow module includes: The verification sample generation unit is used to perform numerical simulation verification on the final recommended design result, and processes the verification data into a quadruple verification sample data consisting of the state, action, reward value and the state corresponding to the changed structure, according to the unified parameterization description. The data feedback and supplementation unit is used to supplement the quadruple verification sample data into the data set of the corresponding type of heat exchanger in the multi-type heat exchanger database. The retraining trigger unit is used to count the amount of new sample data for the corresponding type of heat exchanger. When the amount of new sample data reaches a preset threshold, the heat exchanger agent corresponding to each type of heat exchanger is retrained.

[0094] In one embodiment, the retraining triggering unit is further configured to extract a complete dataset of the current type of heat exchanger from the multi-type heat exchanger database, the complete dataset including historical quadruplet sample data and quadruplet validation sample data; use the complete dataset as a training set to retrain the agent of the current type of heat exchanger to generate a new version of the agent, and record the training sample range, update time, and validation effect corresponding to the new version of the agent; compare the performance of the new version with the current old version on a preset validation metric, and only if the new version is better than the old version on the preset validation metric, replace the old version with the new version and update the registration information of the agent cluster; if the new version is not better than the old version on the preset validation metric, maintain the current old version and discard the new version, retain the quadruplet validation sample data in the multi-type heat exchanger database, and merge it with the subsequently accumulated validation sample data to generate the quadruplet validation sample data used during retraining.

[0095] The embodiments of the present invention achieve the following technical effects: By establishing a unified parameterization framework for common and characteristic description parameters, the problem of separate modeling and difficult unified management of different types of heat exchangers is solved, enabling collaborative design of multiple types of heat exchangers under a unified data organization and task expression framework. Through offline training of categorized reinforcement learning agents, the agents acquire the ability to positively map design conditions to optimized structural parameters. After a design task is assigned, the agent can directly output an optimized solution without repeated iterative trial and error, significantly shortening the design cycle.

[0096] By generating multiple candidate initial structures and invoking multiple types of agents in parallel, a fully automated solution generation and optimization is achieved under conditions without an initial structure, overcoming the limitation of traditional methods that must rely on existing initial solutions for optimization. Through a verification data feedback-driven retraining and version replacement mechanism, the design capability of the agent cluster continuously improves with each usage round, enabling efficient accumulation and reuse of design knowledge.

[0097] The intelligent agent cluster supports two design modes: with initial structure and without initial structure, to adapt to the needs of different engineering scenarios; clustered deployment supports high-concurrency calls, meeting the multi-task parallel requirements in actual engineering design.

[0098] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A forward optimization design method for multi-type heat exchangers based on reinforcement learning agent clusters, characterized in that, include: Historical design data of multiple types of heat exchangers are acquired and described using a unified parameterization method. Standardized datasets for each type of heat exchanger are generated and stored in a multi-type heat exchanger database. The unified parameterization method includes common description parameters and characteristic description parameters. For each type of heat exchanger, the corresponding standardized dataset is extracted from the multi-type heat exchanger database, reinforcement learning agents are constructed and trained offline, and the trained heat exchanger agents of each type are encapsulated and registered to generate a reinforcement learning agent cluster for multi-type heat exchangers. The system receives a heat exchanger optimization design task, determines whether it includes an initial structure, and if it does, identifies the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, matches and calls the corresponding type of reinforcement learning agent, using the output optimized geometric structure parameters as candidate optimization schemes. If the initial structure is not included, for each type of heat exchanger, it generates candidate initial structure schemes based on the common description parameters and the characteristic description parameters, calls the corresponding type of agent to output the optimized geometric structure parameters, and generates multiple candidate optimization schemes. The performance of candidate optimization schemes is evaluated. When there are multiple candidate optimization schemes, the evaluation results of the multiple candidate optimization schemes are ranked, and the candidate optimization scheme with the highest score is output as the final recommended design result. The design and verification are repeatedly optimized, and the generated verification data is added to the database of the multi-type heat exchangers. The agent is then retrained. When the optimization effect of the retrained agent is better than that of the old agent before training, the new agent is used to replace the old agent in the reinforcement learning agent cluster and the version update information is recorded.

2. The forward optimization design method for multiple types of heat exchangers as described in claim 1, characterized in that, Generate standardized datasets for various types of heat exchangers and store them in a multi-type heat exchanger database, including: The operating condition parameters, material physical property parameters, heat exchanger performance parameters, and optimization target description parameters of various types of heat exchangers are set as common description parameters. The operating condition parameters include the physical properties of hot and cold fluids, inlet and outlet temperatures, pressures, and flow rates. The material physical property parameters include the density, specific heat capacity, and thermal conductivity of solid materials. The heat exchanger performance parameters include the heat transfer coefficient and flow resistance coefficient. The optimization target description parameters include the variable hyperparameters in the optimization objective function and constraint condition function. Each type of heat exchanger has its own unique geometric parameters as characteristic description parameters. For example, the characteristic description parameters of plate-fin heat exchangers include fin height, fin pitch, fin thickness, and flow channel size; the characteristic description parameters of tube-fin heat exchangers include tube diameter, number of tube rows, fin pitch, and fin thickness; and the characteristic description parameters of printed circuit board heat exchangers include channel width, channel depth, and channel angle. For each type of heat exchanger, pre-set operating condition parameters and optimization target description parameters, and set an initial structure and a modified structure; The heat exchanger performance parameters and corresponding optimization objective function values ​​of the initial structure and the changed structure are obtained through numerical simulation or surrogate model. A set of four-tuple sample data consisting of the state, action, reward value corresponding to the initial structure and the state corresponding to the changed structure is obtained through data post-processing. The state includes common description parameters and characteristic description parameters. The action includes the geometric structure parameters of changing from the initial structure to the changed structure. The reward value is calculated based on the change in the optimization objective function value before and after the structure change. The sample data of the quadruples corresponding to each type of heat exchanger are classified to generate a multi-type heat exchanger database.

3. The forward optimization design method for multiple types of heat exchangers as described in claim 2, characterized in that, The reward in the quadruple sample data is calculated in the following way: When the modified structure does not meet the constraints, the reward value is set to a preset negative value; When the modified structure satisfies the constraints, the reward value is set as the reduction in the value of the optimization objective function before and after the structural change. The optimization objective function is a function of the heat exchanger performance parameters, and the constraints include geometric parameter range constraints and weight constraints.

4. The forward optimization design method for multiple types of heat exchangers as described in claim 1, characterized in that, Extracting corresponding standardized datasets from the multi-type heat exchanger database, constructing a reinforcement learning agent and training it offline, including: For each type of heat exchanger, a reinforcement learning agent is constructed. The reinforcement learning agent adopts an offline reinforcement learning algorithm. The input of the reinforcement learning agent is the state vector corresponding to the initial structure. The state vector is formed by concatenating the operating condition parameters, the initial geometric structure parameters, the initial heat exchanger performance parameters, and the optimization target description parameters. The output of the reinforcement learning agent is an action vector, which includes the optimized geometric structure parameters. Extract the quadruple sample data corresponding to the current type of heat exchanger from the multi-type heat exchanger database. Take the state corresponding to the initial structure in the quadruple sample data as input and the action as expected output. Train the agent with an offline reinforcement learning algorithm so that the reinforcement learning agent can establish a positive mapping relationship from design conditions to optimized structural parameters. Each type of heat exchanger is trained separately to obtain a reinforcement learning agent corresponding to each type of heat exchanger. Each trained reinforcement learning agent is uniformly packaged and registered to the agent cluster manager. The agent cluster manager maintains the type label, version information and calling interface of each agent, generating a reinforcement learning agent cluster for multiple types of heat exchangers.

5. The forward optimization design method for multiple types of heat exchangers as described in claim 1, characterized in that, Also includes: When the agent cluster receives a heat exchanger optimization design task containing the initial structure, it identifies the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, matches and calls the corresponding type of reinforcement learning agent, and uses the output optimized geometric structure parameters as candidate optimization schemes. The candidate optimization scheme is verified and evaluated by calling numerical simulation or surrogate model to obtain the verified heat exchanger performance parameters of the candidate optimization scheme; The verified heat exchanger performance parameters are compared with the initial heat exchanger performance parameters in the initial structure. The performance improvement before and after optimization is calculated, and the design results are fed back.

6. The forward optimization design method for multiple types of heat exchangers as described in claim 1, characterized in that, Also includes: When the agent cluster receives a heat exchanger optimization design task that does not include the initial structure, for each type of heat exchanger, it generates candidate initial structure schemes based on the common description parameters and the characteristic description parameters, and calls the corresponding type of agent to generate multiple candidate optimization schemes: Read the allowable variation range of the characteristic description parameters of each type of heat exchanger from the multi-type heat exchanger database; For each type of heat exchanger, a set of geometric parameters are randomly sampled within the allowable variation range. These geometric parameters are then combined with common descriptive parameters in the current design task to construct a candidate initial structure scheme for the current type of heat exchanger. The performance of each candidate initial structure scheme is evaluated by calling numerical simulation or surrogate model to obtain the heat exchanger performance parameters of each candidate initial structure scheme. The working condition parameters and optimization target description parameters in the common description parameters, the geometric structure parameters of the candidate initial structure scheme, and the heat exchanger performance parameters are concatenated to form the state vector corresponding to each candidate initial structure scheme. The state vectors corresponding to each candidate initial structure scheme are input into the corresponding type of agent, and each agent outputs the optimized geometric structure parameters to generate multiple candidate optimization schemes.

7. The forward optimization design method for multiple types of heat exchangers as described in claim 1, characterized in that, The performance of candidate optimization schemes is evaluated. When multiple candidate optimization schemes exist, the evaluation results of the multiple candidate optimization schemes are ranked, and the candidate optimization scheme with the highest score is output as the final recommended design result, including: The performance of each candidate optimization scheme is evaluated by calling numerical simulation or surrogate model to obtain the heat exchanger performance parameters corresponding to each candidate optimization scheme. The comprehensive performance score of each candidate optimization scheme is calculated based on the heat exchanger performance parameters, wherein the comprehensive performance score is calculated by weighting the heat exchange performance, flow resistance performance and constraint satisfaction. The candidate optimization schemes are sorted from high to low according to the comprehensive performance score, and the candidate optimization scheme with the highest comprehensive performance score is selected as the final recommended design result.

8. The forward optimization design method for multiple types of heat exchangers as described in any one of claims 1 to 7, characterized in that, After outputting the candidate optimization solution with the highest score as the final recommended design result, the following is also included: The final recommended design result is verified by numerical simulation. The verification data is processed into four-tuple verification sample data consisting of the state, action, reward value, and state corresponding to the changed structure, according to the unified parameterization description. The quadruple verification sample data is added to the corresponding type of heat exchanger dataset in the multi-type heat exchanger database. The number of new sample data for the corresponding type of heat exchanger is counted. When the number of new sample data reaches a preset threshold, the heat exchanger agent corresponding to each type of heat exchanger is retrained.

9. The forward optimization design method for multiple types of heat exchangers as described in claim 8, characterized in that, Retrain the heat exchanger agent corresponding to each type of heat exchanger, including: Extract the complete dataset of the current type of heat exchanger from the multi-type heat exchanger database. The complete dataset includes historical quadruple sample data and quadruple verification sample data. Using the complete dataset as the training set, the agent for the current type of heat exchanger is retrained to generate a new version of the agent, and the training sample range, update time, and verification effect corresponding to the new version of the agent are recorded. The performance of the new version is compared with that of the current old version on a preset verification metric. Only when the new version is better than the old version on the preset verification metric, the old version is replaced by the new version and the registration information of the intelligent agent cluster is updated. If the new version is not superior to the old version in the preset verification metrics, the current old version is maintained and the new version is discarded. The quadruple verification sample data is retained in the multi-type heat exchanger database and merged with the subsequently accumulated verification sample data to generate the quadruple verification sample data used during retraining.

10. A device for forward optimization design of multi-type heat exchangers based on reinforcement learning agent clusters, characterized in that, include: The data standardization module is used to acquire historical design data of multiple types of heat exchangers, perform unified parameterization description, generate standardized datasets of each type of heat exchanger, and store them in a multi-type heat exchanger database. The unified parameterization description includes common description parameters and characteristic description parameters. The intelligent agent cluster construction module is used to extract the corresponding standardized dataset from the multi-type heat exchanger database for each type of heat exchanger, construct reinforcement learning agents and perform offline training, encapsulate and register the trained intelligent agents of each type of heat exchanger, and generate a reinforcement learning agent cluster for multi-type heat exchangers. The task decision module is used to receive the heat exchanger optimization design task, determine whether it includes an initial structure, and if it includes the initial structure, identify the target heat exchanger type and, based on the construction state of the common description parameters and the characteristic description parameters, match and call the corresponding type of reinforcement learning agent, and use the output optimized geometric structure parameters as candidate optimization schemes. If it does not include the initial structure, for each type of heat exchanger, generate candidate initial structure schemes based on the common description parameters and the characteristic description parameters, call the corresponding type of agent to output the optimized geometric structure parameters, and generate multiple candidate optimization schemes. The scheme evaluation module is used to evaluate the performance of candidate optimization schemes. When there are multiple candidate optimization schemes, the evaluation results of multiple candidate optimization schemes are sorted, and the candidate optimization scheme with the highest score is output as the final recommended design result. The version update module is used to repeatedly optimize and verify the design, supplement the generated verification data into the multi-type heat exchanger database, and retrain the agent. When the optimization effect of the retrained agent is better than that of the old agent before training, the new agent is used to replace the old agent in the reinforcement learning agent cluster and the version update information is recorded.

Images