A reinforcement learning-based method and system for designing integrated energy absorption and heat dissipation structures.

Abstract

This invention relates to the field of integrated energy absorption and heat dissipation structure design technology, addressing the problem that existing battery pack base plate structures struggle to simultaneously achieve low temperature, high energy absorption, and large design space, leading to optimization difficulties. It provides a reinforcement learning-based method and system for designing integrated energy absorption and heat dissipation structures. The method includes: discretizing the design area of ​​the battery pack base plate into a grid space; initializing a parallel policy network; within the grid space, each parallel agent selects an action and state, generates a walking path, obtains the corresponding grid structure, and performs collision and heat dissipation simulations; combining the reward function, the value estimate of the current state is calculated based on the Critic network, and an advantage function is constructed accordingly; updating the Critic and Actor networks; after training, the optimal integrated energy absorption and heat dissipation structure and its grid structure design parameters are obtained, achieving multi-objective collaborative optimization of the structure under collision and heat dissipation conditions.

Description

Technical Field

[0001] This invention relates to the field of integrated energy absorption and heat dissipation structure design technology, and in particular to an integrated energy absorption and heat dissipation structure design method and system based on reinforcement learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The heat dissipation and safety of the battery pack have become key factors restricting the overall performance of new energy vehicles. The battery pack base plate not only needs to have good heat dissipation capabilities to maintain the battery operating within a suitable temperature range, but also needs to withstand bottom impacts such as flying stones from the road to protect the internal batteries from impact. Traditional battery pack base plates mostly use single-function metal plates or foam sandwich structures, which are difficult to meet the dual requirements of low temperature and high specific energy absorption at the same time. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a reinforcement learning-based integrated energy absorption and heat dissipation structural design method and system, which enables precise control of structural performance and multi-objective collaborative optimization of the structure under collision and heat dissipation conditions.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a method for designing an integrated energy absorption and heat dissipation structure based on reinforcement learning.

[0006] In one or more embodiments, a reinforcement learning-based method for designing an integrated energy absorption and heat dissipation structure is provided, comprising: Discretize the design area of ​​the battery pack base plate into a grid space and establish a unified coordinate system; Initialize the parallel policy network, which includes setting the number of parallel agents, constructing a centralized Critic network and an Actor network for each agent, and initializing the shared experience replay buffer for the agents. Within the grid space, after each parallel agent selects an action and state, the agent generates a walking path, obtains the corresponding grid structure, and performs collision simulation and heat dissipation simulation. Based on the collision simulation results, heat dissipation simulation results, and the reward function that comprehensively considers energy absorption performance and heat dissipation performance, the reward value obtained by each parallel agent is calculated. Then, each agent puts its local observation, action, reward value and next observation as experience samples into the experience replay buffer. The value estimate of the current state is calculated based on the Critic network and the advantage function is constructed accordingly. Based on empirical samples randomly sampled from the empirical replay buffer, the loss function calculated by the advantage function is used to update the Critic network and the Actor network until the parallel agent exploration stops. After training, the optimal energy absorption-heat dissipation integrated structure and its mesh structure design parameters are obtained.

[0007] As one implementation method, the expression for the reward function is:

[0008] in, As a reward value; The specific energy absorption of the grid structure; The specific energy absorption of a solid structure with the same size as the mesh structure; The maximum temperature of the grid structure; The maximum temperature of the solid structure with the same size as the mesh structure.

[0009] As one implementation method, the Critic network approximates the true reward by minimizing the mean squared error loss, and its loss function is set as follows: for: ; ; in, The average reward shared by multiple agents; This represents the reward value for the i-th agent. The number of agents; To calculate the value estimate of the current state based on the Critic network; The state of the agent.

[0010] As one implementation method, the Actor network uses the policy gradient method for updating, and the policy loss function of the i-th agent is set as follows: : ; ; in, For the action of the i-th agent, Design a path strategy for the i-th agent; The dominant function; This represents the current global state of the agent. The average reward shared by multiple agents.

[0011] As one implementation method, for each agent i, the current global state is obtained. Each agent receives an action based on its Actor network. Among them, actions Two of the probabilities from the four directions (up, down, left, right) are randomly selected for adjustment, and the sampling process follows a policy distribution: .

[0012] In one implementation, a grid is randomly selected from the top row of the grid space as the "starting point". Then, a path is generated with probability according to the direction of up, down, left or right. As the path continues to extend, a virtual row is added below the bottom row of the grid. The path stops generating when it reaches any grid in the virtual row. That grid is the "end point".

[0013] As one implementation method, the grid structure is a solid skeleton formed by the reserved area of ​​discrete grid units in the design area of ​​the battery pack base plate. After stretching the solid skeleton area, it is assembled with the upper and lower plates to form a three-dimensional heat dissipation-energy absorption structure that can absorb energy through collision and transfer heat.

[0014] A second aspect of the present invention provides an integrated energy absorption and heat dissipation structure design system based on reinforcement learning.

[0015] In one or more embodiments, a reinforcement learning-based integrated energy absorption and heat dissipation structural design system includes: The design area discretization module is used to discretize the design area of ​​the battery pack base plate into a grid space and establish a unified coordinate system. The network initialization module is used to initialize the parallel policy network. It includes setting the number of parallel agents, building a centralized Critic network and an Actor network for each agent, and initializing the shared experience replay buffer for the agents. The mesh structure generation module is used to generate a walking path for each parallel agent in the mesh space after the agent selects an action and state, obtains the corresponding mesh structure, and performs collision simulation and heat dissipation simulation. The reward calculation module is used to calculate the reward value obtained by each parallel agent based on the collision simulation results, heat dissipation simulation results, and a reward function that comprehensively considers energy absorption performance and heat dissipation performance. Then, each agent puts its local observations, actions, reward values, and next observations into the experience replay buffer as experience samples. The value estimate of the current state is calculated based on the Critic network and the advantage function is constructed accordingly. The network update module is used to update the Critic network and Actor network based on empirical samples randomly sampled from the empirical replay buffer, combined with the loss function calculated by the advantage function, until the parallel agent exploration stopping condition is reached. The optimal determination module is designed to obtain the optimal energy absorption-heat dissipation integrated structure and its mesh structure design parameters after training is completed.

[0016] A third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the reinforcement learning-based integrated energy absorption and heat dissipation structure design method described above.

[0018] A fourth aspect of the present invention provides an electronic device.

[0019] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the reinforcement learning-based integrated energy absorption and heat dissipation structure design method described above.

[0020] Compared with the prior art, the beneficial effects of the present invention are: This invention discretizes the design area of ​​the battery pack base plate into a grid space, transforms the internal structural design of the base plate into a controllable configuration generation process of cell retention and deletion, integrates multi-agent parallel reinforcement learning and simulation, and adopts multiple parallel agents to explore different design strategies simultaneously. Compared with the single-agent method, this significantly improves the exploration efficiency and solves the problems of large design space and low efficiency of traditional optimization methods faced by new energy vehicle battery pack base plates under the dual performance requirements of heat dissipation and collision. It achieves precise control of structural performance and realizes multi-objective collaborative optimization of the structure under collision and heat dissipation conditions. It provides an efficient and automated design method for complex structural components such as new energy battery pack base plates that need to meet the requirements of impact safety and thermal management at the same time. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of the energy absorption-heat dissipation integrated structure design method based on reinforcement learning according to an embodiment of the present invention; Figure 2 This is a detailed illustration of the mesh structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the actual application of the mesh structure according to an embodiment of the present invention; Figure 4 This is a reinforcement learning-based mesh structure optimization design process according to an embodiment of the present invention; Figure 5This is a schematic diagram of a mesh structure heat dissipation simulation model according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a mesh structure collision simulation model according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the heat dissipation simulation results of the mesh structure according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the collision simulation results of the mesh structure according to an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] Figure 1 A schematic diagram of the energy absorption-heat dissipation integrated structure design method based on reinforcement learning according to an embodiment of the present invention is provided. Figure 1 The energy absorption-heat dissipation integrated structure design method based on reinforcement learning in this embodiment may include the following steps S101 to S105.

[0027] The specific implementation process of steps S101 to S105 is as follows: Step S101: Discretize the design area of ​​the battery pack base plate into a grid space and establish a unified coordinate system.

[0028] The design area of ​​the battery pack base plate is discretized into several square grid cells of the same size, with each grid cell having a side length of 0.5 × 0.5 mm. For example... Figure 3 As shown, the mesh structure generation area has a total of 40×58 meshes. The dimensions of the upper and lower flat plates are 20mm×20mm×0.5mm. Therefore, the dimensions of the assembled mesh structure are 20mm×20mm×30mm. The density of the aluminum alloy A205 material used in the mesh structure is 2680 kg / m³. 3 It has a Young's modulus of 75 GPa, a Poisson's ratio of 0.33, a yield strength of 297 MPa, and a thermal conductivity of λ 200 W / (m K).

[0029] To facilitate the subsequent generation of structural paths, all grids are arranged in rows and columns, and a unified coordinate system is established with the top-left grid as the origin. A grid is randomly selected from the top row of the grid space as the "starting point," and then a path is generated with probability based on the direction of up, down, left, or right.

[0030] As the path approaches the boundary, its selectable directions are restricted to ensure it remains within the design area. For example, when the path reaches the leftmost edge of the grid, it can only move right, up, or down. As the path extends, a virtual row is added below the bottom row of the grid. Generation stops when the path reaches any grid cell in this virtual row; this cell is the "end point." Considering that only one point in the virtual row appears in the path, limiting the freedom of grid structure design, this row of grid cells does not participate in the grid structure design. At this point, the entire path can be viewed as a connected trajectory "walking out" of the grid space. The grid cells traversed by the path are retained, while those not traversed are deleted, resulting in a two-dimensional structural sketch composed of discrete grid cells, as shown below. Figure 2 As shown.

[0031] Step S102: Initialize the parallel policy network, which includes setting the number of parallel agents, constructing a centralized Critic network and an Actor network corresponding to each agent, and initializing the shared experience replay buffer for the agents.

[0032] For example, setting the number of parallel agents to K =3, each agent is responsible for exploring and generating different path design strategies. A Critic network and an Actor network are constructed, where the Critic network is used to evaluate the global state value function. V ( s t Each Actor network corresponds to one agent, which is used to output the probability distribution of actions. An experience replay buffer (shared by agents) is initialized to store the interaction trajectories of each agent.

[0033] For each agent i, obtain the current global state. Each agent receives an action based on its Actor network. Among them, actions Two of the probabilities from the four directions (up, down, left, right) are randomly selected for adjustment, and the sampling process follows a policy distribution: .

[0034] Step S103: In the grid space, after each parallel agent selects an action and state, the agent generates a walking path, obtains the corresponding grid structure, and performs collision simulation and heat dissipation simulation.

[0035] Select Action a t and global state Then, the agent generates a walking path based on the adjusted directional probability and generates the corresponding mesh structure. The mesh structure is not a simple random graphic, but a solid skeleton formed by reserved areas of sub-millimeter-level units; after stretching the solid areas, it is assembled with the upper and lower plates to form a three-dimensional heat dissipation-energy absorption structure that can absorb energy through collisions and transfer heat.

[0036] Collision Simulation: In the structural collision energy absorption simulation, a rigid circular plate with a diameter of 90 mm and a mass of 2.95 kg is applied to the upper and lower surfaces of the mesh structure. One end of the rigid circular plate is fixed, and the initial impact velocity of the other end is 15 m / s. A global self-contact with a friction coefficient of 0.15 is set between the structures. Explicit dynamic analysis is performed, and the specific energy absorption during the collision process is calculated when the rigid body collision velocity drops to 0 or the collision deformation reaches 70% of the structural height. SEA Its expression is SEA = E / m ,in E The total energy absorbed by structural deformation. m For structural quality.

[0037] Heat dissipation simulation: The heat source is simplified to an 8×8×5mm cuboid, which is in contact with the top center of the mesh structure. Heat conduction occurs directly between the heat source and the mesh unit structure, while convection heat exchange occurs around the structure with the air. The thermal conductivity of the heat source is set to 20 W / (m²). K), the heat flux density at the interface between the heat source and the grid structure is 6.25 W / m. 2 The surface heat transfer coefficient between the mesh structure and the surrounding environment is 8 W / (m²). 2 K), ambient temperature T The temperature was set to 25℃. A steady-state simulation was performed to obtain the temperature distribution, and the maximum temperature in the structure at this point was extracted from the stabilized temperature field. T .

[0038] Step S104: Based on the collision simulation results, heat dissipation simulation results, and the reward function that comprehensively considers energy absorption performance and heat dissipation performance, calculate the reward value obtained by each parallel agent. Then, each agent puts its local observation, action, reward value, and next observation as experience samples into the experience replay buffer. Then, calculate the value estimate of the current state based on the Critic network and construct the advantage function accordingly.

[0039] The reward function is constructed by comprehensively considering both energy absorption and heat dissipation performance. For specific energy absorption optimization, the objective is to maximize the SEA (Sequential Energy Absorption Area). Therefore, when the SEA of the structure is greater than the initial SEA, the reward value should be positive, and vice versa. For temperature optimization, the objective is to minimize the temperature. Therefore, the lower the maximum temperature T of the structure, the higher the reward, and the higher the T, the lower the reward. The design of the reward function not only considers the optimization of these two indicators but also uses normalization to make the two objectives comparable, avoiding one objective dominating the optimization process. The expression of the reward function is:

[0040] in, As a reward value; The specific energy absorption of the grid structure; The specific energy absorption of a solid structure with the same size as the mesh structure; The maximum temperature of the grid structure; The maximum temperature of the solid structure with the same size as the mesh structure. Calculate the reward value obtained by the agent at this time according to the reward function. .

[0041] Step S105: Based on the empirical samples randomly sampled from the empirical replay buffer, update the Critic network and Actor network using the loss function calculated by the advantage function, until the parallel agent exploration stopping condition is met.

[0042] After the action is performed a t , change the design state from s t Updated to s t+1 Each agent records its local observations, actions, rewards, and next observation, i.e., { s t , a t , r t , s t+1 The experience is stored in a shared experience replay buffer. Every fixed step (e.g., every 10 environment steps), a batch of experiences is randomly sampled from the buffer, the loss function is calculated, and the Critic network and Actor network are updated.

[0043] The Critic network approximates the true reward by minimizing the mean squared error loss, and its loss function is set as follows: for: ; ; in, The average reward shared by multiple agents; This represents the reward value for the i-th agent. The number of agents; To calculate the value estimate of the current state based on the Critic network; The state of the agent.

[0044] The Actor network updates using the policy gradient method, with the policy loss function of the i-th agent set as follows: : ; ; in, For the action of the i-th agent, Design a path strategy for the i-th agent; The dominant function; This represents the current global state of the agent. The average reward shared by multiple agents.

[0045] In the specific implementation process, every 10 iterations, the Actor network parameters of multiple agents are synchronized to maintain policy diversity and improve training stability. As the number of training iterations increases, action sampling noise is gradually reduced, thereby gradually making the policy more deterministic.

[0046] Step S106: After training is completed, the optimal energy absorption-heat dissipation integrated structure and its mesh structure design parameters are obtained.

[0047] like Figure 4 As shown, in the design process of the energy absorption-heat dissipation integrated structure based on reinforcement learning, a[0] direction probability array is defined, containing four direction probabilities: up, down, left, and right, all with an initial value of 0.25. The action space of the agent is set to randomly select two of the four direction probabilities, incrementing one by 0.05 and decrementing the other by 0.05. Then, the agent randomly generates a complete path based on the current direction probabilities. Collision energy absorption and steady-state heat dissipation simulations are performed on a 20mm×20mm×30mm solid structure to obtain the SEA. ref = 9354.633 J / kg, T = 40.752℃.

[0048] Initialize the environment and reinforcement learning parameters, randomly select actions, and input the Python modeling script after taking the actions into the ABAQUS software for modeling. Perform collision and heat dissipation simulations, as detailed below. Figures 5 to 8 As shown, the .obd file is obtained, and then the result is post-processed to obtain the reward value.

[0049] The agent selects an action based on the reward value to update the state parameters, takes another action to change the direction probability, generates a random path, and then obtains the reward value for the next action based on collision and heat simulation, repeating the above steps.

[0050] Using a reinforcement learning algorithm, the parameter array 'a' was optimized after a set number of training iterations (e.g., 2000) to obtain a mesh structure that simultaneously possesses energy absorption and heat dissipation properties. Experimental results show that the energy absorption effect of the mesh cell structure optimized by the reinforcement learning agent was effectively improved, while the maximum temperature in the stabilized temperature field was significantly reduced.

[0051] This invention presents a structure based on sub-millimeter-level grid cell data. This structure uses 0.5mm × 0.5mm sub-millimeter-level cells as the smallest design object, forming the structure through the retention, deletion, and stretching of grid cells, achieving integrated energy absorption and heat dissipation. However, the design parameter space for such structures is extremely large. Using traditional experimental design methods is not only costly and time-consuming, but also difficult to converge to the global optimum. Reinforcement learning, through agent-environment interaction, can efficiently explore a vast design space. However, conventional single-agent reinforcement learning suffers from low sample efficiency. Multi-agent parallel reinforcement learning, by utilizing multiple agents to simultaneously explore different design regions, can significantly improve exploration efficiency.

[0052] In one or more embodiments, a reinforcement learning-based integrated energy absorption and heat dissipation structure design system is also provided, which can be implemented in software. The reinforcement learning-based integrated energy absorption and heat dissipation structure design system includes the following software modules: The design area discretization module is used to discretize the design area of ​​the battery pack base plate into a grid space and establish a unified coordinate system. The network initialization module is used to initialize the parallel policy network. It includes setting the number of parallel agents, building a centralized Critic network and an Actor network for each agent, and initializing the shared experience replay buffer for the agents. The mesh structure generation module is used to generate a walking path for each parallel agent in the mesh space after the agent selects an action and state, obtains the corresponding mesh structure, and performs collision simulation and heat dissipation simulation. The reward calculation module is used to calculate the reward value obtained by each parallel agent based on the collision simulation results, heat dissipation simulation results, and a reward function that comprehensively considers energy absorption performance and heat dissipation performance. Then, each agent puts its local observations, actions, reward values, and next observations into the experience replay buffer as experience samples. The value estimate of the current state is calculated based on the Critic network and the advantage function is constructed accordingly. The network update module is used to update the Critic network and Actor network based on empirical samples randomly sampled from the empirical replay buffer, combined with the loss function calculated by the advantage function, until the parallel agent exploration stopping condition is reached. The optimal determination module is designed to obtain the optimal energy absorption-heat dissipation integrated structure and its mesh structure design parameters after training is completed.

[0053] It should be noted that each module in the reinforcement learning-based integrated energy absorption and heat dissipation structure design system of the present invention corresponds one-to-one with each step in the reinforcement learning-based integrated energy absorption and heat dissipation structure design method in the above embodiments, and their specific implementation processes are the same, so they will not be repeated here.

[0054] The structure of the electronic device according to embodiments of the present invention will be described in detail below. The electronic device provided in the embodiments of the present invention includes: at least one processor, a memory, a user interface, and at least one network interface. The various components in the reinforcement learning-based energy absorption-heat dissipation integrated structure design system are coupled together through a bus system. It can be understood that the bus system is used to realize the connection and communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus. The user interface may include a display, keyboard, mouse, trackball, click wheel, buttons, a touchpad, or a touch screen, etc.

[0055] It is understood that the memory can be volatile memory or non-volatile memory, or both. The memory in this embodiment of the invention is capable of storing data to support the operation of the terminal. Examples of this data include any computer programs used to operate on the terminal, such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0056] In some embodiments, the reinforcement learning-based integrated energy absorption and heat dissipation structure design system provided in this invention can be implemented using a combination of hardware and software. For example, the reinforcement learning-based integrated energy absorption and heat dissipation structure design system provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the reinforcement learning-based integrated energy absorption and heat dissipation structure design method provided in this invention. For instance, the hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0057] As an example, a processor can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where a general-purpose processor can be a microprocessor or any conventional processor, etc.

[0058] As an example of the hardware implementation of the reinforcement learning-based integrated energy absorption and heat dissipation structure design system provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor in the form of a hardware decoding processor. For example, it can be executed by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the reinforcement learning-based integrated energy absorption and heat dissipation structure design method provided in this embodiment of the invention.

[0059] The memory in this embodiment of the invention is used to store various types of data to support the operation of the reinforcement learning-based energy absorption-heat dissipation integrated structure design system, or to store data for execution. Figure 1The program code for the method shown. Examples of this data include: any executable instructions for operation on a reinforcement learning-based integrated energy absorption and heat dissipation structure design system, such as executable instructions that can be included in the executable instructions to implement the reinforcement learning-based integrated energy absorption and heat dissipation structure design method of the embodiments of the present invention.

[0060] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including functions for executing... Figure 1 The program code for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by the central processing unit, it performs the various functions defined in the apparatus of this application.

[0061] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for designing an integrated energy absorption and heat dissipation structure based on reinforcement learning, characterized in that, include: Discretize the design area of ​​the battery pack base plate into a grid space and establish a unified coordinate system; Initialize the parallel policy network, which includes setting the number of parallel agents, constructing a centralized Critic network and an Actor network for each agent, and initializing the shared experience replay buffer for the agents. Within the grid space, after each parallel agent selects an action and state, the agent generates a walking path, obtains the corresponding grid structure, and performs collision simulation and heat dissipation simulation. Based on the collision simulation results, heat dissipation simulation results, and the reward function that comprehensively considers energy absorption performance and heat dissipation performance, the reward value obtained by each parallel agent is calculated. Then, each agent puts its local observation, action, reward value and next observation as experience samples into the experience replay buffer. The value estimate of the current state is calculated based on the Critic network and the advantage function is constructed accordingly. Based on empirical samples randomly sampled from the empirical replay buffer, the loss function calculated by the advantage function is used to update the Critic network and the Actor network until the parallel agent exploration stops. After training, the optimal energy absorption-heat dissipation integrated structure and its mesh structure design parameters are obtained.

2. The energy absorption-heat dissipation integrated structure design method based on reinforcement learning as described in claim 1, characterized in that, The expression for the reward function is: in, As a reward value; The specific energy absorption of the grid structure; The specific energy absorption of a solid structure with the same size as the mesh structure; The maximum temperature of the grid structure; The maximum temperature of the solid structure with the same size as the mesh structure.

3. The energy absorption-heat dissipation integrated structure design method based on reinforcement learning as described in claim 1, characterized in that, The Critic network approximates the true reward by minimizing the mean squared error loss, and its loss function is set as follows: for: ； ； in, The average reward shared by multiple agents; This represents the reward value for the i-th agent. The number of agents; To calculate the value estimate of the current state based on the Critic network; The state of the agent.

4. The energy absorption-heat dissipation integrated structure design method based on reinforcement learning as described in claim 1, characterized in that, The Actor network updates using the policy gradient method, with the policy loss function of the i-th agent set as follows: : ； ； in, For the action of the i-th agent, Design a path strategy for the i-th agent; The dominant function; This represents the current global state of the agent. The average reward shared by multiple agents.

5. The energy absorption-heat dissipation integrated structure design method based on reinforcement learning as described in claim 1, characterized in that, For each agent i, obtain the current global state. Each agent receives an action based on its Actor network. Among them, actions Two of the probabilities from the four directions (up, down, left, right) are randomly selected for adjustment, and the sampling process follows a policy distribution: .

6. The energy absorption-heat dissipation integrated structure design method based on reinforcement learning as described in claim 1, characterized in that, In the grid space, a grid cell is randomly selected from the top row as the "starting point". Then, a path is generated with probability based on the direction of up, down, left, or right. As the path extends, a virtual row is added below the bottom row of the grid. The path stops generating when it reaches any grid cell in the virtual row, and that grid cell is the "ending point".

7. The energy absorption-heat dissipation integrated structure design method based on reinforcement learning as described in claim 1, characterized in that, The grid structure is a solid skeleton formed by the reserved area of ​​discrete grid units in the design area of ​​the battery pack base plate. After stretching the solid skeleton area, it is assembled with the upper and lower plates to form a three-dimensional heat dissipation-energy absorption structure that can absorb energy through collision and transfer heat.

8. A reinforcement learning-based integrated energy absorption and heat dissipation structural design system, characterized in that, The reinforcement learning-based integrated energy absorption and heat dissipation structure design method according to any one of claims 1-7 includes: The design area discretization module is used to discretize the design area of ​​the battery pack base plate into a grid space and establish a unified coordinate system. The network initialization module is used to initialize the parallel policy network. It includes setting the number of parallel agents, building a centralized Critic network and an Actor network for each agent, and initializing the shared experience replay buffer for the agents. The mesh structure generation module is used to generate a walking path for each parallel agent in the mesh space after the agent selects an action and state, obtains the corresponding mesh structure, and performs collision simulation and heat dissipation simulation. The reward calculation module is used to calculate the reward value obtained by each parallel agent based on the collision simulation results, heat dissipation simulation results, and a reward function that comprehensively considers energy absorption performance and heat dissipation performance. Then, each agent puts its local observations, actions, reward values, and next observations into the experience replay buffer as experience samples. The value estimate of the current state is calculated based on the Critic network and the advantage function is constructed accordingly. The network update module is used to update the Critic network and Actor network based on empirical samples randomly sampled from the empirical replay buffer, combined with the loss function calculated by the advantage function, until the parallel agent exploration stopping condition is reached. The optimal determination module is designed to obtain the optimal energy absorption-heat dissipation integrated structure and its mesh structure design parameters after training is completed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the reinforcement learning-based integrated energy absorption and heat dissipation structure design method as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the reinforcement learning-based integrated energy absorption and heat dissipation structure design method as described in any one of claims 1-7.

Images