A heterogeneous reinforcement learning and finite element co-simulation system and method

Abstract

The application belongs to the technical field of computer simulation, and particularly relates to a heterogeneous reinforcement learning and finite element collaborative simulation system and method. The system comprises a finite element simulation server and an RL calculation server. The finite element simulation server comprises a server-side program layer, an FEA calculation task layer and a first hardware resource layer. The RL calculation server comprises a client-side program layer, an RL calculation task layer and a second hardware resource layer. A bidirectional data interaction channel is established between the server-side program layer of the finite element simulation server and the client-side program layer of the RL calculation server, and the first hardware resource layer and the second hardware resource layer are independent of each other. The FEA calculation task and the RL calculation task are distributed to corresponding resource platforms through the heterogeneous computing architecture, so that the problem of competition for computing resources when two tasks are simultaneously run on a single device is avoided, and the hardware resources can be flexibly combined, thereby effectively improving the utilization efficiency of resources.

Description

Technical Field

[0001] This invention belongs to the field of computational simulation technology, specifically relating to a heterogeneous reinforcement learning and finite element co-simulation system and method. Background Technology

[0002] With the deep integration of artificial intelligence and engineering simulation, optimization based on the synergy of reinforcement learning (RL) and finite element analysis (FEA) has become an important method for the design of complex systems (such as aerospace structures and new energy vehicle battery packs), which can effectively improve product design efficiency and enable rapid product iteration.

[0003] Both FEA and RL tasks have high demands on CPU and memory resources. Currently, FEA and RL tasks are typically deployed on hardware devices sharing computing resources, such as CPU-GPU hybrid servers. This results in high hardware configuration requirements and high power consumption for individual devices, and easily leads to competition for CPU and GPU resources between the two tasks. Therefore, in scenarios based on RL and FEA collaborative optimization, how to optimize hardware resource utilization, avoid CPU and memory resource contention, reduce hardware power consumption, and improve overall computing efficiency has become a pressing technical challenge. Summary of the Invention

[0004] The purpose of this invention is to provide a heterogeneous reinforcement learning and finite element co-simulation system and method to optimize hardware resource utilization, avoid CPU and memory resource contention, reduce hardware power consumption and improve overall computing efficiency.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A heterogeneous reinforcement learning and finite element co-simulation system includes a finite element simulation server and an RL computation server;

[0007] The finite element simulation server includes: a server-side program layer, a FEA calculation task layer, and a first hardware resource layer. The first hardware resource layer includes a first CPU and a first memory. The server-side program layer is interconnected with the FEA calculation task layer, and the FEA calculation task layer is interconnected with the first hardware resource layer.

[0008] The RL computing server includes: a client program layer, an RL computing task layer, and a second hardware resource layer. The second hardware resource layer includes a second CPU, a GPU, and a second memory. The client program layer is interconnected with the RL computing task layer, and the RL computing task layer is interconnected with the second hardware resource layer.

[0009] A bidirectional data interaction channel is established between the server-side program layer of the finite element simulation server and the client-side program layer of the RL calculation server, and the first hardware resource layer and the second hardware resource layer are independent of each other.

[0010] Furthermore, the FEA server-side program layer receives the first control instruction sent by the RL client-side program layer, and controls the FEA computing task layer to call the first hardware resources to execute the FEA computing task in order to obtain simulation result data. The RL computing server client sends a second control instruction to the RL computing task layer according to user needs. Based on the obtained simulation result data, the client controls the RL computing task layer to call the second hardware resources to execute the RL computing task in order to obtain feedback optimization parameters and send them to the FEA server-side program layer.

[0011] Furthermore, a bidirectional data interaction channel is established between the FEA server-side program layer and the RL client-side program layer through a network transmission protocol. Specifically, the FEA server-side program layer uses the TCP protocol to send simulation result data to the RL client-side program layer, and the RL client-side program layer uses the TCP protocol to send feedback optimization parameters to the FEA server-side program layer. Heartbeat packets between the FEA computation task and the RL computation task are transmitted using the UDP protocol.

[0012] A heterogeneous reinforcement learning and finite element co-simulation method is proposed. This method is implemented using a heterogeneous reinforcement learning and finite element co-simulation system and includes the following steps: Step 1, configure initial parameters: Configure the initial simulation parameters of the finite element simulation server, as well as the data processing parameters, model parameters and training process parameters of the RL server according to the requirements.

[0013] Step 2: The RL client program layer generates a first control instruction or a second control instruction according to user requirements. The first control instruction includes starting / stopping the execution of the FEA calculation task; the second control instruction includes starting / stopping the execution of the RL calculation task.

[0014] Step 3: The FEA server-side program layer receives the first control instruction sent by the RL client program layer, and controls the FEA calculation task layer to call the first hardware resource layer to execute the FEA task and generate the initial simulation result data according to the first control instruction.

[0015] Step 4: The RL client program layer reads the simulation result data through the TCP protocol. Using the simulation result as input, under the control of the second control instruction, it calls the second hardware resource layer to execute the RL task and generate feedback optimization parameters.

[0016] Step 5: The FEA server-side program layer reads the feedback optimization parameters through the TCP protocol, dynamically adjusts the simulation parameter configuration of the FEA calculation task layer based on the feedback optimization parameters, and re-executes the FEA calculation; then steps 3 to 5 are repeated until the error between the simulation result data and the design target value converges to the preset threshold, the RL client sends a stop command, and the process ends.

[0017] Furthermore, step 3 also includes a process of compressing the simulation result data obtained by the FEA computation task layer calling the first hardware resource layer to execute the FEA task using an adaptive block parallel post-processing and data compression algorithm, including the following steps:

[0018] 1) Obtain the current number of processes n (n≥2) on the first CPU;

[0019] 2) The FEA simulation results data includes k grid nodes, each node has m physical field variables; the FEA simulation results data is divided into n-1 blocks according to the nodes numbered consecutively from 1 to k; each block is processed by a separate thread to achieve parallel processing, while an additional thread is reserved to avoid exhausting CPU resources;

[0020] 3) Within each block, determine whether the target node is located in that block; if so, extract the physical field variables for the corresponding node.

[0021] 4) After the physical field variables of the target nodes in all the block threads have been extracted, the physical field variables of all the extracted target nodes are spliced ​​together according to the consecutive node numbers.

[0022] 5) Based on the accuracy requirements of the RL server for the simulation results, select a data compression algorithm with lossy or lossless compression modes to compress the extracted data after splicing, so as to reduce data storage space and network transmission bandwidth.

[0023] Furthermore, the data compression algorithm with lossy or lossless compression modes is the ZFP data compression algorithm.

[0024] By adopting the above technical solution, the present invention has the following beneficial effects:

[0025] 1. This invention adopts a heterogeneous computing architecture, which allocates FEA computing tasks and RL computing tasks to corresponding hardware resource platforms, avoiding the problem of computing resource competition when a single device runs two tasks at the same time; and realizes long-distance interconnection of heterogeneous devices through the network, thereby achieving flexible combination of hardware resources and improving resource utilization efficiency.

[0026] 2. This invention transmits only the physical field variables corresponding to the necessary simulation result nodes between the FEA calculation task and the RL calculation task, rather than the high-dimensional full data. This not only reduces the network bandwidth pressure, but also ensures the security of the finite element model data.

[0027] 3. This invention designs a hybrid transmission mechanism between the finite element simulation server and the RL server, combining high real-time small data transmission based on the UDP protocol with high-reliability large data transmission based on the TCP protocol. A heartbeat mechanism ensures a stable connection between the devices over a long period, balancing the real-time performance and reliability of data transmission. This mechanism is not limited to UDP and TCP protocols; other network protocols with both high real-time and high-reliability transmission characteristics can also be used.

[0028] 4. In this invention, within a finite element simulation server, adaptive block-based parallel post-processing and data compression algorithms are combined to process the FEA simulation results data in blocks based on the number of CPU threads. Only the physical field variables of the target nodes are extracted, thus accelerating the post-processing speed of the finite element simulation results data. Depending on the accuracy requirements of the simulation results, a data compression algorithm with lossy or lossless compression modes is selected as needed, saving data network transmission bandwidth and shortening the waiting time for RL tasks. Attached Figure Description

[0029] Figure 1 This is the system architecture of the heterogeneous reinforcement learning and finite element co-simulation method of the present invention;

[0030] Figure 2 This is the dual-channel hybrid transmission mechanism data interaction channel used in the embodiment;

[0031] Figure 3 The flowchart of the adaptive block parallel post-processing and data compression algorithm is shown in the example. Detailed Implementation

[0032] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0033] like Figure 1 As shown in the figure, this embodiment provides a heterogeneous reinforcement learning and finite element co-simulation system, including a finite element simulation server and an RL calculation server.

[0034] The finite element simulation server includes: a server-side program layer, a FEA calculation task layer, and a first hardware resource layer. The first hardware resource layer includes a first CPU and a first memory. The server-side program layer is interconnected with the FEA calculation task layer, and the FEA calculation task layer is interconnected with the first hardware resource layer.

[0035] The RL computing server includes: a client program layer, an RL computing task layer, and a second hardware resource layer. The second hardware resource layer includes a second CPU, a GPU, and a second memory. The client program layer is interconnected with the RL computing task layer, and the RL computing task layer is interconnected with the second hardware resource layer.

[0036] To enable collaborative processing of long-distance FEA and RL computing tasks and effectively improve the utilization rate of idle resources on older equipment, this embodiment establishes a bidirectional data interaction channel between the server-side program layer of the finite element simulation server and the client-side program layer of the RL computing server via a network transmission protocol. The first and second hardware resource layers are independent of each other, achieving decoupling of hardware resources between FEA and RL computing tasks, avoiding CPU and GPU resource contention, preventing overload of a single device, and thus avoiding expensive hardware upgrade costs.

[0037] During implementation, the FEA server-side program layer receives the first control instruction sent by the RL client-side program layer. Based on the first control instruction, it controls the FEA computing task layer to call the first hardware resources to execute the FEA computing task in order to obtain simulation result data. The RL computing server client sends a second control instruction to the RL computing task layer according to user requirements. Based on the obtained simulation result data, it controls the RL computing task layer to call the second hardware resources to execute the RL computing task in order to obtain feedback optimization parameters and send them to the FEA server-side program layer.

[0038] To balance high real-time small data transmission and high-reliability large data transmission, this embodiment designs a dual-channel hybrid transmission mechanism based on UDP and TCP, effectively isolating small data and large data transmission channels while ensuring both real-time performance and reliability. Specifically, as follows... Figure 2 As shown:

[0039] The FEA simulation results obtained from the finite element simulation server typically include multiple physical field variables from multiple mesh nodes, such as the coordinates, equivalent values, and component values ​​of physical fields like stress, strain, and displacement, resulting in a large amount of data. Therefore, this embodiment designs data transmission based on the TCP protocol to ensure the integrity of the data received by the RL computation task layer.

[0040] The feedback optimization parameters calculated by the RL server are typically large in volume, and the accuracy of the transmitted feedback optimization parameters must be ensured. In this embodiment, the feedback optimization parameters are sent to the FEA calculation task based on the TCP protocol. In addition, heartbeat packets between the FEA calculation task and the RL calculation task are sent through a UDP-based channel to ensure a stable connection between the two over a long period of time and to guarantee the continuity of the system's collaborative simulation.

[0041] Based on the above-mentioned heterogeneous reinforcement learning and finite element co-simulation system, this embodiment also provides a heterogeneous reinforcement learning and finite element co-simulation method, which includes the following steps:

[0042] Step 1, Configure Initial Parameters: Configure the initial simulation parameters of the finite element simulation server, as well as the data processing parameters, model parameters, and training process parameters of the RL server, according to requirements. In this embodiment, the initial simulation parameters include geometric model parameters, material properties, boundary conditions, loads, analysis type, constraint settings, initial conditions, and load case tables; data processing parameters include data cleaning, dataset partitioning, and data normalization; model structure-related parameters include model structure, model hyperparameters, and initialization methods; and training process parameters include learning rate, optimization method, batch size, loss function, and number of training epochs.

[0043] Step 2: The RL client program layer generates a first control instruction or a second control instruction according to user requirements. The first control instruction includes starting / stopping the execution of the FEA calculation task; the second control instruction includes starting / stopping the execution of the RL calculation task.

[0044] Step 3: The FEA server-side program layer receives the first control instruction sent by the RL client program layer, and controls the FEA calculation task layer to call the first hardware resource layer to execute the FEA task and generate the initial simulation result data according to the first control instruction.

[0045] Step 4: The RL client program layer reads the simulation result data through the TCP protocol. Using the simulation result as input, under the control of the second control instruction, it calls the second hardware resource layer to execute the RL task and generate feedback optimization parameters.

[0046] Step 5: The FEA server-side program layer reads the feedback optimization parameters through the TCP protocol, dynamically adjusts the simulation parameter configuration of the FEA calculation task layer based on the feedback optimization parameters, and re-executes the FEA calculation; then steps 3 to 5 are repeated until the error between the simulation result data and the design target value converges to the preset threshold, the RL client sends a stop command, and the process ends.

[0047] To further shorten the waiting time at the RL computing end, save network bandwidth, and improve the overall system operating efficiency, this embodiment also employs an adaptive block-parallel post-processing and data compression algorithm in step 3 to compress the simulation result data obtained by the FEA computing task layer calling the first hardware resource layer to execute the FEA task. For example... Figure 3 As shown, it includes the following steps:

[0048] 1) Obtain the current number of processes n (n≥2) on the first CPU.

[0049] 2) The FEA simulation results data includes k grid nodes, and each node corresponds to m physical field variables. To achieve parallel processing, these nodes are divided into n-1 blocks by consecutive numbering from 1 to k; each block is processed by a separate thread, while an additional thread is reserved to avoid exhausting CPU resources; the blocks are numbered sequentially from 1 to n−1, and since the total number of nodes is k, each block is evenly distributed. There are 10 nodes, where the first block contains node numbers 1 to 10. The second block contains node numbers. to Subsequent blocks are divided in the same way. If a node cannot be evenly distributed to each block, the remaining nodes are allocated to the last block, ensuring that each node is uniquely assigned to a specific block.

[0050] 3) Within each block, determine whether the target node is located in that block; if so, extract the physical field variables for the corresponding node; if not, skip that block and search from other blocks. 4) After the physical field variable extraction of the target node in all block threads is completed, concatenate all the extracted target node physical field variables according to the consecutive node numbers;

[0051] 5) FEA simulation results typically have a large number of decimal places. Based on the accuracy requirements of the RL server for the simulation results, a data compression algorithm with lossy or lossless compression modes is selected to compress the extracted data after splicing, thereby reducing data storage space and network transmission bandwidth. In this embodiment, the ZFP data compression algorithm with lossy or lossless compression modes is selected.

[0052] In summary, the heterogeneous reinforcement learning and finite element co-simulation system and method provided in this embodiment solves the problems caused by the fact that FEA and RL tasks are usually deployed on hardware devices with shared computing resources (such as CPU-GPU hybrid servers) in the prior art: the hardware configuration requirements of a single device are high, the power consumption is high, and the two tasks are prone to competing for CPU and GPU resources. At the same time, it optimizes the use of hardware resources, avoids competition for CPU and memory resources, reduces the power consumption of hardware devices, and improves the overall computing efficiency.

[0053] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A heterogeneous reinforcement learning and finite element collaborative simulation system, comprising a finite element simulation server and an RL computing server; characterized in that: the finite element simulation server comprises a server-side program layer, an FEA computing task layer, and a first hardware resource layer comprising a first CPU and a first memory; the server-side program layer is interconnected with the FEA computing task layer, and the FEA computing task layer is interconnected with the first hardware resource layer; the RL computing server comprises a client-side program layer, an RL computing task layer, and a second hardware resource layer comprising a second CPU, a GPU, and a second memory; the client-side program layer is interconnected with the RL computing task layer, and the RL computing task layer is interconnected with the second hardware resource layer; a bidirectional data interaction channel is established between the server-side program layer of the finite element simulation server and the client-side program layer of the RL computing server, and the first hardware resource layer and the second hardware resource layer are independent of each other; the FEA server-side program layer receives a first control instruction sent by the RL client-side program layer, controls the FEA computing task layer to call the first hardware resource to execute an FEA computing task according to the first control instruction, to obtain simulation result data; the RL computing server client sends a second control instruction to the RL computing task layer according to user requirements, controls the RL computing task layer to call the second hardware resource to execute an RL computing task according to the second control instruction based on the obtained simulation result data, to obtain feedback optimization parameters and send them to the FEA server-side program layer.

2. The heterogeneous reinforcement learning and finite element co-simulation system of claim 1, wherein, A bidirectional data interaction channel is established between the FEA server-side program layer and the RL client-side program layer through a network transmission protocol; wherein: the FEA server-side program layer sends simulation result data to the RL client-side program layer using a TCP protocol, the RL client-side program layer sends feedback optimization parameters to the FEA server-side program layer using a TCP protocol, and heartbeat packets between the FEA computing task and the RL computing task are transmitted using a UDP protocol.

3. A heterogeneous reinforcement learning and finite element co-simulation method, implemented by the heterogeneous reinforcement learning and finite element co-simulation system of claim 1 or 2, characterized in that, The method comprises the following steps: Step 1: configuring initial parameters; configuring simulation initial parameters of the finite element simulation server and data processing parameters, model parameters, and training process parameters of the RL server according to requirements; Step 2: the RL client-side program layer generates a first control instruction or a second control instruction according to user requirements; the first control instruction includes starting / stopping execution of the FEA computing task; the second control instruction includes starting / stopping execution of the RL computing task; Step 3: the FEA server-side program layer receives the first control instruction sent by the RL client-side program layer, controls the FEA computing task layer to call the first hardware resource layer to execute the FEA task according to the first control instruction, and generates initial simulation result data; Step 4: the RL client-side program layer reads the simulation result data through a TCP protocol, takes the simulation result as input, calls the second hardware resource layer to execute the RL task under the control of the second control instruction, and generates feedback optimization parameters. Step 5, the FEA server program layer reads the feedback optimization parameters through the TCP protocol, dynamically adjusts the simulation parameter configuration of the FEA calculation task layer based on the feedback optimization parameters, and re-executes the FEA calculation; then steps 3 to 5 are repeated until the error of the simulation result data and the design target value converges to a preset threshold, the RL client sends a stop instruction, and the process ends.

4. The method of claim 3, wherein, The step 3 further includes using an adaptive block parallel post-processing and data compression algorithm to compress the simulation result data obtained by calling the first hardware resource layer to execute the FEA task of the FEA calculation task layer, and the implementation process includes the following steps: 1) obtaining the current process number n of the first CPU, and n≥2; 2) the FEA simulation result data includes k grid nodes, each node has m physical field variables; the FEA simulation result data is divided into n-1 blocks according to the continuous node number 1 to k; each block is processed by a single thread to realize parallel processing, and an additional thread is reserved to avoid CPU resource exhaustion; 3) in each block, it is judged whether the target node is located in the block; if yes, the physical field variable of the corresponding node is extracted; 4) after the physical field variable extraction work of the target node in all block threads is completed, the extracted target node physical field variables are spliced according to the continuous node number; 5) according to the accuracy requirement of the RL server on the simulation result, a data compression algorithm with lossy or lossless compression mode is selected according to the requirement to compress the spliced extracted data, so as to reduce the data storage space and network transmission bandwidth.

5. The method of claim 4, wherein, The data compression algorithm with lossy or lossless compression mode is ZFP data compression algorithm.

Images