Battery pack thermal management model training method and device, and computer device

Abstract

This application provides a training method, apparatus, and computer equipment for a battery pack thermal management model. The method includes: acquiring sample data of the battery pack, the sample data including at least action sample data and state sample data of the battery pack; establishing a reward function for an initial thermal management model based on the sample data, the initial thermal management model being a reinforcement learning model; and training the initial thermal management model based on the reward function. By acquiring sample data of the battery pack, establishing a reward function for the initial thermal management model based on the sample data, and then training the initial thermal management model based on the reward function, the resulting battery pack thermal management model solves the technical problems of poor environmental adaptability and inability to adaptively adjust battery pack energy consumption in related technologies for managing battery pack temperature. It improves the environmental adaptability of battery pack temperature management and enables adaptive adjustment of battery pack energy consumption.

Description

Technical Field

[0001] This application relates to the field of battery technology, and in particular to a training method, apparatus, and computer equipment for a battery pack thermal management model. Background Technology

[0002] With the continuous development of electric vehicles, the application value of battery technology is becoming increasingly important. As the sole energy source for electric vehicles, the battery pack needs to operate within a suitable temperature range to maintain its normal health. Excessively high or low temperatures can damage the battery cells.

[0003] In related technologies, to ensure that the battery operates within a suitable temperature range, a battery thermal management system is typically activated to heat or cool the battery pack, thereby regulating its temperature. However, the thermal management system itself consumes energy from the battery pack, and its operation relies on pre-defined logical rules. In practical applications, due to the diversity of usage environments and vehicle operating conditions, while operating the thermal management system according to established logical rules can maintain a normal battery pack temperature, it is not applicable to all scenarios and cannot guarantee that energy consumption will be minimized in all usage environments. Therefore, the technical solutions for managing battery pack temperature in related technologies have poor environmental adaptability and cannot adaptively adjust battery pack energy consumption.

[0004] There are currently no effective solutions to the technical problems of poor environmental adaptability and inability to adaptively adjust battery pack energy consumption in the relevant technologies for managing battery pack temperature. Summary of the Invention

[0005] Therefore, it is necessary to provide a training method, apparatus, and computer equipment for a battery pack thermal management model to address the aforementioned technical problems, thereby solving the problems that existing technical solutions for managing the temperature of battery packs have poor environmental adaptability and cannot adaptively adjust the energy consumption of battery packs.

[0006] Firstly, this application provides a training method for a battery pack thermal management model, including:

[0007] Obtain sample data of the battery pack, wherein the sample data includes at least action sample data and status sample data of the battery pack;

[0008] Based on the sample data, a reward function for an initial thermal management model is established, wherein the initial thermal management model is a reinforcement learning model;

[0009] The initial thermal management model is trained based on the reward function.

[0010] In one embodiment, the reward function includes at least a temperature term, a water pump energy consumption term, and a temperature regulation energy consumption term. The temperature term is determined based on the highest sampling temperature, the lowest sampling temperature, the upper temperature threshold, and the lower temperature threshold of the battery pack. The water pump energy consumption term is determined based on the water pump speed. The temperature regulation energy consumption term is determined based on the actual inlet water temperature and the preset inlet water temperature.

[0011] In one embodiment, the sample data is collected at multiple time points, and training the initial thermal management model based on the reward function includes:

[0012] Based on the reward function and the sample data at each time point, calculate the reward value at each time point;

[0013] Based on the reward value at each time point, calculate the total future reward value at each time point, where the total future reward value is the sum of the reward values ​​at all time points after the current time point;

[0014] The state sample data, action sample data, and future reward total value at each time point are input into the initial thermal management model, and the initial thermal management model is trained.

[0015] In one embodiment, the iterative process of the forward path of the initial thermal management model includes:

[0016] Using the current projection state value at the current time point, the total future reward value of the current projection, and the historical projection state values, total future rewards of the historical projections, and historical projection action values ​​of all historical time points as inputs, the current projection action value at the current time point is predicted;

[0017] A deduction sequence is established based on the deduction status value, deduction action value, and total deduction future reward value at each time point.

[0018] In one embodiment, training the initial thermal management model includes:

[0019] Based on the inferred action values ​​and the action sample data, a loss function is established, and the parameters of the initial thermal management model are adjusted based on the loss function.

[0020] In one embodiment, training the initial thermal management model further includes:

[0021] The battery pack state data sequence is input into the trained target thermal management model;

[0022] Based on the target thermal management model, the state data sequence is extrapolated to obtain the action data sequence.

[0023] In one embodiment, after obtaining the sample data of the battery pack, the method further includes:

[0024] The sample data is cleaned, and the objects of data cleaning include at least missing values ​​and outliers in the sample data.

[0025] Secondly, this application also provides a training device for a battery pack thermal management model, comprising:

[0026] The acquisition module is used to acquire sample data of the battery pack, wherein the sample data includes at least action sample data and status sample data of the battery pack;

[0027] A module is established to establish a reward function for an initial thermal management model based on the sample data, wherein the initial thermal management model is a reinforcement learning model;

[0028] The training module is used to train the initial thermal management model based on the reward function.

[0029] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0030] Obtain sample data of the battery pack, wherein the sample data includes at least action sample data and status sample data of the battery pack;

[0031] Based on the sample data, a reward function for an initial thermal management model is established, wherein the initial thermal management model is a reinforcement learning model;

[0032] The initial thermal management model is trained based on the reward function.

[0033] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0034] Obtain sample data of the battery pack, wherein the sample data includes at least action sample data and status sample data of the battery pack;

[0035] Based on the sample data, a reward function for an initial thermal management model is established, wherein the initial thermal management model is a reinforcement learning model;

[0036] The initial thermal management model is trained based on the reward function.

[0037] This application provides a training method, apparatus, and computer device for a battery pack thermal management model. The method includes: acquiring sample data of the battery pack, the sample data including at least action sample data and state sample data of the battery pack; establishing a reward function for an initial thermal management model based on the sample data, the initial thermal management model being a reinforcement learning model; and training the initial thermal management model based on the reward function. By acquiring sample data of the battery pack, establishing a reward function for the initial thermal management model based on the sample data, and then training the initial thermal management model based on the reward function, the resulting battery pack thermal management model solves the technical problems of poor environmental adaptability and inability to adaptively adjust battery pack energy consumption in related technologies for managing battery pack temperature. It improves the environmental adaptability of battery pack temperature management and allows for adaptive adjustment of battery pack energy consumption. Attached Figure Description

[0038] Figure 1 This is an application environment diagram of the training method for the battery pack thermal management model in one embodiment of this application;

[0039] Figure 2 This is a flowchart illustrating the training method of a battery pack thermal management model in one embodiment of this application;

[0040] Figure 3 This is a schematic diagram of a reinforcement learning model in a training method for a battery pack thermal management model according to an embodiment of this application;

[0041] Figure 4 This is a flowchart illustrating a training method for a battery pack thermal management model according to another embodiment of this application.

[0042] Figure 5 This is a structural block diagram of a training device for a battery pack thermal management model according to an embodiment of this application;

[0043] Figure 6 This is an internal structural diagram of a computer device according to an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] The training method for the battery pack thermal management model provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, Figure 1This is an application environment diagram of the training method for a battery pack thermal management model in one embodiment of this application. The terminal 102 communicates with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104 or placed on a cloud or other network server. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0046] With the continuous development of electric vehicles, the application value of battery technology is becoming increasingly important. The battery pack is the sole energy source for pure electric vehicles, and it needs to be used within a verified temperature range to maintain its normal health. Excessively high or low temperatures can damage the battery cells.

[0047] Therefore, a battery thermal management system is needed in daily practical use to heat or cool the battery pack to maintain it within a suitable normal temperature range. However, existing technologies for managing battery pack temperature have poor environmental adaptability and cannot adaptively adjust battery pack energy consumption.

[0048] To address this, a new type of optimized control technology is needed that can adaptively adjust the cooling or heating inlet temperature and water pump motor speed set by the electric vehicle battery pack thermal management system according to the real-time driving status of the vehicle, so as to heat or cool the battery pack with better energy efficiency.

[0049] Please see Figure 2 , Figure 2 This is a flowchart illustrating the training method of a battery pack thermal management model in one embodiment of this application.

[0050] In one embodiment, such as Figure 2 As shown, the training methods for the battery pack thermal management model include:

[0051] S202: Obtain sample data of the battery pack, including at least action sample data and status sample data of the battery pack.

[0052] Specifically, the battery pack operation sample data includes the operation data of the water inlet when cooling or heating the battery pack. In this embodiment, the operation sample data includes the water inlet set temperature T. set And the water pump speed (rpm). It's understandable that the inlet's set temperature (T) is... setIt can intuitively display the current set temperature of the water inlet. When it is higher than the current battery temperature, the battery pack is heated; when it is lower than the current battery temperature, the battery pack is cooled. It also displays the current water pump speed (rpm). As you can understand, the water pump speed (rpm) is positively correlated with the water inlet speed. The higher the water pump speed (rpm), the faster the water inlet speed and the faster the water flow, resulting in higher efficiency in temperature management of the battery pack.

[0053] Specifically, the state sample data of the battery pack includes battery current I, voltage U, state of charge (SOC), and the highest temperature T at the battery pack temperature sampling point. max The lowest temperature T at the battery pack temperature sampling point min The actual inlet water temperature is T_E. This is understandable; it includes battery current I, voltage U, state of charge (SOC), and the highest temperature at the battery pack temperature sampling point T. max Lowest temperature T at battery pack temperature sampling point min Ambient temperature T a Furthermore, vehicle speed V can provide a complete picture of the battery pack's current state sample data, allowing us to understand its current condition. Understandably, the broader the data dimensions, the higher the accuracy of the judgment regarding the battery's current state. Therefore, training the battery pack thermal management model with state sample data composed of multiple dimensions will yield more accurate results and stronger environmental adaptability.

[0054] S204: Based on the sample data, establish the reward function for the initial thermal management model, which is a reinforcement learning model.

[0055] Specifically, reinforcement learning models include Decision Transformer models, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of a reinforcement learning model in the training method of a battery pack thermal management model according to an embodiment of this application. In this embodiment, the initial thermal management model established based on sample data is a reinforcement learning model. Unlike traditional reinforcement learning methods that fit value functions or calculate policy gradients, the Decision Transformer outputs the optimal action by utilizing a casually masked Transformer. By specifying the expected reward, past states, and actions to the autoregressive (i.e., using all current and previous outputs as inputs for the next iteration, generating an output sequence) model, the Transformer model can generate corresponding actions to achieve the expected reward.

[0056] Understandably, Decision Transformer is an offline reinforcement learning method that can be trained using massive amounts of historical data in a cloud platform big data environment. It is not affected by poor-quality samples in the dataset, so it performs very well in the sparse reward problem in reinforcement learning.

[0057] S206: Train the initial thermal management model based on the reward function.

[0058] Specifically, after establishing the reward function for the initial thermal management model using sample data, the initial thermal management model can be trained using the corresponding reward function. Understandably, the high-dimensional data samples obtained through action and state sample data form a highly accurate training set for the reward function established based on them. Training the initial thermal management model (which is a reinforcement learning model) using this reward function results in a battery pack thermal management model with high environmental adaptability and the ability to adaptively adjust the battery pack's energy consumption.

[0059] In another embodiment, the reward function includes at least a temperature term, a water pump energy consumption term, and a temperature regulation energy consumption term. The temperature term is determined based on the highest sampling temperature, the lowest sampling temperature, the upper temperature threshold, and the lower temperature threshold of the battery pack. The water pump energy consumption term is determined based on the water pump speed. The temperature regulation energy consumption term is determined based on the actual inlet water temperature and the preset inlet water temperature.

[0060] Specifically, obtaining the reward function requires action sample data and state sample data. The action sample data includes the inlet set temperature T. e And water pump speed rpm; state sample data includes battery current I, voltage U, state of charge (SOC), and highest temperature T at the battery pack temperature sampling point. max The lowest temperature T at the battery pack temperature sampling point min Ambient temperature T E Therefore, the established reward function should have three functional terms: temperature, water pump energy consumption, and temperature regulation energy consumption.

[0061] The objective of the reward function is to maintain both the maximum and minimum temperatures within the normal temperature range while simultaneously calculating the energy consumption for temperature regulation and the energy consumption for pump speed.

[0062] The temperature term is determined as follows:

[0063] T = m(T) min -T lower (T) upper -T min )+n(T upper -T max (T) max -Tlower ),

[0064] Where m>0, n>0, are adjustable weighting coefficients. It can be understood that the temperature term T is determined by the highest sampled temperature T of the battery pack. max Minimum sampling temperature T min and the upper limit threshold temperature T upper and the lower limit threshold T lower Determined. Among them, the upper temperature threshold T... upper and the lower limit threshold T lower As can be understood from the battery pack itself, different battery packs have different upper and lower temperature thresholds.

[0065] The method for determining the energy consumption of water pumps is as follows:

[0066] P = k·rpm 2 / SOC

[0067] k is an adjustable weighting coefficient. As can be understood, the pump energy consumption term P is determined by the pump speed. The energy consumption term can be determined by comparing the square of the pump speed (rpm) with the current state of charge (SOC), and then multiplying by the adjustable weighting coefficient k. The energy consumption of the pump varies depending on its speed at different SOC states.

[0068] The method for determining the energy consumption item for temperature regulation is as follows:

[0069] W = C * |T E –T set | / SOC

[0070] Among them, T E T represents the actual inlet water temperature. set The preset inlet water temperature is represented by 'c', which is an adjustable weighting coefficient. It is understood that in some embodiments, since the current ambient temperature and the actual inlet water temperature may be the same, T... E It can also be the current ambient temperature. By calculating the absolute values ​​of the actual inlet water temperature and the preset inlet water temperature, then comparing them with the current state of charge (SBC), and finally adjusting using weighting coefficients, the change in SBC corresponding to each temperature adjustment can be determined. The temperature regulation energy consumption determined in this way is accurate and reliable.

[0071] After obtaining the temperature term T, the water pump energy consumption term P, and the temperature regulation energy consumption term W, the reward function R can be determined based on these. Specifically, the reward function R = TPW. The reward function R obtained here can be used for initial thermal management model training.

[0072] In one embodiment, sample data was collected at multiple time points; please refer to [link / reference]. Figure 4 , Figure 4This is a flowchart illustrating a training method for a battery pack thermal management model according to another embodiment of this application. Training the initial thermal management model based on the reward function includes:

[0073] S402: Calculate the reward value at each time point based on the reward function and the sample data at each time point;

[0074] S404: Based on the reward value at each time point, calculate the total future reward value at each time point. The total future reward value is the sum of the reward values ​​at all time points after the current time point.

[0075] S406: Input the state sample data, action sample data and future reward total value at each time point into the initial thermal management model, and train the initial thermal management model.

[0076] Specifically, first, after obtaining the reward function, the reward value can be obtained using the reward function and the current data. Different sample data correspond to different reward values, and different time points correspond to different sample data. Therefore, it is necessary to obtain the sample data for each time point and then calculate the reward value corresponding to the sample data for each time point.

[0077] Next, based on the reward value at each time point, the total future reward value at each time point can be determined, where R is the sum of the reward values ​​from the current time point to the end of the sequence, calculated as follows:

[0078] R = r t +r t+1 +r t+2 +…+r T

[0079] After acquiring the state sample data, action sample data, and future reward total value at each time point, the initial thermal management model is trained based on the above data to obtain the battery pack thermal management model of this embodiment.

[0080] Understandably, by using the state value, action value, and reward value at each time point as the training set to train a reinforcement training model, future actions can be predicted based on current and past actions. The model trained on future actions can then predict the future actions of electric vehicles and plan the battery pack layout to optimize the total reward value. Furthermore, the trained model obtained in this way has strong environmental adaptability, and the battery pack adjustment actions based on future action planning can effectively reduce battery pack energy consumption.

[0081] In another embodiment, the iterative process of the forward channel of the initial thermal management model includes: taking the current deduced state value at the current time point, the current deduced future reward total value, and the historical deduced state values, historical deduced future reward total values, and historical deduced action values ​​at all historical time points as inputs, predicting the current deduced action value at the current time point; and establishing a deduced sequence based on the deduced state value, deduced action value, and deduced future reward total value at each time point.

[0082] like Figure 3 As shown, the Casual Transformer, based on the GPT model, differs from traditional reinforcement learning methods that fit value functions or calculate policy gradients. The Decision Transformer masks the optimal action through causality. By using autoregression to take its current and previous outputs as inputs for the next iteration, it iteratively generates an output sequence. Based on past states and actions, the model can generate corresponding actions to achieve the desired reward.

[0083] Specifically, the current projected state value at the current time point, the total projected future reward value at the current time point, and the historical projected state values, the total projected future reward values, and the historical projected action values ​​at all historical time points are used as inputs to the initial thermal management model in the current iteration process to predict the current projected action value at the current time point.

[0084] More specifically, when the initial thermal management model begins its forward propagation iteration, an initial total reward value R0 is set, and an initial state value s0 is input. Based on the initial total reward value R0 and the initial state value s0, the initial action value a0 is predicted, where a0 = DT(s0, R0) (DT() refers to the derivation result of the Decision Transformer), and a real-time reward value R'0 is fed back. Then, at the next time point, the difference between the total reward value R0 and the reward value R'0 is taken as the total reward value R1 at the current time point, and a state transition is performed to obtain the state value s1 at the current time point. Based on the total reward value R1, the state value s1, the total reward value R0, the state value s0, and the action value a0 at the previous time point, the action value a1 at the current time point is predicted, where a1 = DT(s0, R0, a0, R1, s1), and a real-time reward value R'1 is fed back. The above process is repeated until the initial thermal management model completes its iteration.

[0085] In addition, in some embodiments, the input also needs to include embedded information of time points, i.e., position encoding. It can be understood that the embedded information of time points can effectively correspond to the current state value, current inferred state value, current action value, and current inferred action value at the same time point.

[0086] Next, based on the projected action value, projected state value, and projected total future reward value at each time point, a corresponding projection sequence is established to complete the iterative process of the forward path of the initial thermal management model. The projection sequence is the final input sequence of the thermal management model.

[0087] τ=(R1,s1,a1,R2,s2,a2,...,R T s T a T )

[0088] Where T represents the final time point. This input sequence can improve the training accuracy of the model and result in more accurate computational results when outputting the model.

[0089] In one embodiment, training the initial thermal management model includes: establishing a loss function based on the inferred action values ​​and action sample data, and adjusting the parameters of the initial thermal management model based on the loss function.

[0090] During model training, a loss function needs to be established. A loss function maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. In applications, the loss function is often used as a learning criterion in relation to the optimization problem; that is, the model is solved and evaluated by minimizing the loss function.

[0091] Specifically, by extrapolating action values ​​and action sample data, random event data that the model has not extrapolated can be obtained. The loss function obtained based on the above data can be used as random "risk" data to train the model. That is, by adjusting the parameters of the thermal management model through the loss function, the model can be made more adaptable to the environment.

[0092] Specifically, when establishing the loss function, MSE (mean squared error) is used as the loss function for continuous inferred action values ​​and action sample data.

[0093] In another embodiment, after training the initial thermal management model, the method further includes: inputting the state data sequence of the battery pack into the trained target thermal management model; and deducing the action data sequence based on the target thermal management model.

[0094] Specifically, after training the initial thermal management model using a reward function, a trained thermal management model can be obtained. Understandably, by inputting the current state data sequence of the battery pack into the trained target thermal management model, and then having the target thermal management model perform inference calculations based on the state data sequence, an action data sequence based on the inference can be obtained.

[0095] Understandably, during the testing and prediction process, by setting a relatively high total future reward value and a battery state data sequence within a reasonable range, the optimized action value can be obtained, i.e., the deduced action data sequence, namely the pump speed and inlet set temperature. Based on the optimized pump speed and inlet set temperature obtained through deduction, the optimal energy management strategy for the current state value can be used.

[0096] In one embodiment, after obtaining the sample data of the battery pack, the method further includes: cleaning the sample data, wherein the object of data cleaning includes at least missing values ​​and outliers in the sample data.

[0097] Understandably, in this embodiment, a large amount of actual driving data is collected through a cloud platform. Therefore, there is a certain amount of noise in the sample data. Based on this, after obtaining the sample data, it is necessary to perform noise reduction processing on the sample data. In this embodiment, noise reduction processing includes data cleaning of the sample data.

[0098] In this embodiment, after acquiring a large amount of driving process data, the data is first divided into layers based on region and ambient temperature. The data in each layer needs to be as balanced as possible. Then, the driving process data after being divided into layers, i.e., the sample data, is cleaned to remove missing values ​​and outliers. The sample data sequence is also divided into segments according to the driving speed of each vehicle.

[0099] In some embodiments, considering that the action space of the thermal management strategy implemented in actual vehicles only uses a few fixed gears of water pump speed and inlet temperature setpoints, in order to enhance the action space of the training dataset, it is necessary to adopt a real vehicle experiment design, drive and test under standard test conditions or arbitrary industrial control, and collect battery thermal management status data under more speed gears and set temperature conditions.

[0100] The battery pack thermal management model obtained based on the embodiments of this application does not require a complex physical modeling process, making it easy to implement. The modeling process for adaptive control problems is simple, and this algorithm can avoid the traditional problems in traditional reinforcement learning. At the same time, by assigning rewards through a self-attention structure, it also avoids the use of discounted rewards that may lead to undesirable short-sighted behavior, and can effectively solve the problem of sparse rewards.

[0101] The battery pack thermal management model obtained based on the embodiments of this application can be retrained on a small amount of test data to achieve good results, and may even exceed the best sample trajectory already available in the training samples.

[0102] The battery pack thermal management model obtained based on the embodiments of this application takes into account both cooling and heating scenarios, and the Transformer has strong generalization ability, which makes the model have a certain degree of transferability in different scenarios.

[0103] In one embodiment, such as Figure 5 As shown, a training device for a battery pack thermal management model is provided, comprising:

[0104] The acquisition module 10 is used to acquire sample data of the battery pack. The sample data includes at least action sample data and status sample data of the battery pack.

[0105] Module 20 is established to build the reward function of the initial thermal management model based on the sample data. The initial thermal management model is a reinforcement learning model.

[0106] Training module 30 is used to train the initial thermal management model based on the reward function.

[0107] The training module 30 is also used to calculate the reward value at each time point based on the reward function and the sample data at each time point; to calculate the total future reward value at each time point based on the reward value at each time point, the total future reward value being the sum of the reward values ​​at all time points after the current time point; to input the state sample data, action sample data, and total future reward value at each time point into the initial thermal management model, and to train the initial thermal management model.

[0108] The training module 30 is also used to predict the current deduction action value at the current time point by taking the current deduction state value, the current deduction future reward total value, and the historical deduction state values, historical deduction future reward total values, and historical deduction action values ​​at all historical time points as inputs; and to establish a deduction sequence based on the deduction state value, deduction action value, and deduction future reward total value at each time point.

[0109] The training module 30 is also used to establish a loss function based on the inferred action values ​​and action sample data, and to adjust the parameters of the initial thermal management model based on the loss function.

[0110] The training device for the battery pack thermal management model also includes a cleaning module;

[0111] The cleaning module is used to clean the sample data. The objects of data cleaning include at least missing values ​​and outliers in the sample data.

[0112] Each module in the aforementioned battery pack thermal management model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0113] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a training method for a battery pack thermal management model. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0114] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0116] Obtain sample data of the battery pack, which includes at least action sample data and status sample data of the battery pack;

[0117] Based on the sample data, a reward function for the initial thermal management model is established. The initial thermal management model is a reinforcement learning model.

[0118] The initial thermal management model is trained based on the reward function.

[0119] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0120] Calculate the reward value at each time point based on the reward function and the sample data at each time point;

[0121] Based on the reward value at each time point, calculate the total future reward value at each time point. The total future reward value is the sum of the reward values ​​of all time points after the current time point.

[0122] The state sample data, action sample data, and future total reward value at each time point are input into the initial thermal management model, and the initial thermal management model is trained.

[0123] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0124] Using the current projection state value at the current time point, the total future reward value of the current projection, and the historical projection state values, total future rewards of the historical projections, and historical projection action values ​​of all historical time points as inputs, the current projection action value at the current time point is predicted;

[0125] A deduction sequence is established based on the deduction status value, deduction action value, and total deduction future reward value at each time point.

[0126] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0127] Based on the inferred action values ​​and action sample data, a loss function is established, and the parameters of the initial thermal management model are adjusted based on the loss function.

[0128] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0129] The battery pack state data sequence is input into the trained target thermal management model;

[0130] Based on the target thermal management model, the state data sequence is extrapolated to obtain the action data sequence.

[0131] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0132] Data cleaning is performed on the sample data. The objects of data cleaning include at least missing values ​​and outliers in the sample data.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0134] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a battery pack thermal management model, the method comprising: include: Obtain sample data of the battery pack, wherein the sample data includes at least action sample data and status sample data of the battery pack; The sample data was collected at multiple time points; Based on the sample data, a reward function for an initial thermal management model is established. The initial thermal management model is a reinforcement learning model, which includes a Decision Transformer model. The reward function is determined based on the temperature term T, the water pump energy consumption term P, and the temperature regulation energy consumption term W, wherein the temperature term is based on the highest sampling temperature T of the battery pack. max Minimum sampling temperature T min Temperature upper limit threshold T upper and the lower limit threshold T lower It is determined that the water pump energy consumption item is determined by comparing the square of the water pump speed (rpm) with the current state of charge (SOC) and then multiplying it by an adjustable weighting coefficient k. The temperature regulation energy consumption item W is based on the actual inlet water temperature T at the inlet. E and preset inlet water temperature T set Determined, the reward function is expressed as , , ,in, , where c is the adjustable weight coefficient; Based on the reward function and the sample data at each time point, the reward value at each time point is calculated; the objective determined by the reward function is to maintain the highest and lowest temperatures within the normal temperature range while simultaneously calculating the energy consumption for temperature regulation and the energy consumption for water pump speed. Based on the reward value at each time point, calculate the total future reward value at each time point, where the total future reward value is the sum of the reward values ​​at all time points after the current time point; The state sample data, action sample data, and future reward total value at each time point are input into the initial thermal management model, and the initial thermal management model is trained; the position code of the corresponding time point is added to the input of the initial thermal management model; The iterative process of the forward channel of the initial thermal management model includes: taking the current deduced state value, the current deduced future reward total value, and the historical deduced state values, historical deduced future reward total values, and historical deduced action values ​​of all historical time points as inputs, predicting the current deduced action value of the current time point; and establishing a deduced sequence based on the deduced state value, deduced action value, and deduced future reward total value of each time point.

2. The training method for the battery pack thermal management model according to claim 1, characterized in that, The training of the initial thermal management model includes: Based on the inferred action values ​​and the action sample data, a loss function is established, and the parameters of the initial thermal management model are adjusted based on the loss function.

3. The training method for the battery pack thermal management model according to claim 1, characterized in that, The process of training the initial thermal management model further includes: The battery pack state data sequence is input into the trained target thermal management model; Based on the target thermal management model, the state data sequence is extrapolated to obtain the action data sequence.

4. The training method for the battery pack thermal management model according to claim 1, characterized in that, After obtaining the sample data of the battery pack, the process also includes: The sample data is cleaned, and the objects of data cleaning include at least missing values ​​and outliers in the sample data.

5. A training device for a battery pack thermal management model, characterized in that, The device includes: The acquisition module is used to acquire sample data of the battery pack, the sample data including at least action sample data and status sample data of the battery pack; the sample data is collected at multiple time points; A module is established to build a reward function for an initial thermal management model based on the sample data. The initial thermal management model is a reinforcement learning model, including a Decision Transformer model. The reward function is determined based on a temperature term T, a water pump energy consumption term P, and a temperature regulation energy consumption term W. The temperature term is based on the highest sampled temperature T of the battery pack. max Minimum sampling temperature T min Temperature upper limit threshold T upper and the lower limit threshold T lower It is determined that the water pump energy consumption item is determined by comparing the square of the water pump speed (rpm) with the current state of charge (SOC) and then multiplying it by an adjustable weighting coefficient k. The temperature regulation energy consumption item is determined based on the actual inlet water temperature and the preset inlet water temperature. The reward function is expressed as follows: , , ,in, , where c is the adjustable weight coefficient; The training module is used to calculate the reward value at each time point based on the reward function and the sample data at each time point; the reward function determines the objective of maintaining both the highest and lowest temperatures within the normal temperature range, while simultaneously calculating the energy consumption for temperature regulation and water pump speed; based on the reward value at each time point, the future total reward value at each time point is calculated, which is the sum of the reward values ​​at all time points after the current time point; the state sample data, the action sample data, and the future total reward value at each time point are input into the initial thermal management model, and the initial thermal management model is trained; The training module is also used to take the current deduction state value, the total future reward value of the current time point, the historical deduction state value, the total future reward value of the historical time points, and the historical deduction action value as inputs to predict the current deduction action value of the current time point; and to establish a deduction sequence based on the deduction state value, deduction action value, and total future reward value of each time point.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images