Heavy truck battery system intelligent decision-making method based on DQN algorithm and related equipment

Abstract

The invention discloses a heavy truck battery system intelligent decision-making method based on a DQN algorithm and related equipment, and the method comprises the steps: firstly analyzing the composition structure and operation energy flow relation of a heavy truck battery system, and constructing a detailed battery energy consumption calculation model; thirdly, abstracting a constraint condition and an optimization target from the energy consumption calculation model, and converting a decision problem into a Markov decision process; by designing the state, action and reward function of deep reinforcement learning, a DQN model is constructed, historical data is utilized for training, and an intelligent model capable of selecting the optimal action according to the current state is generated; the system obtains the state of the battery in real time, calculates the optimal action through the trained DQN model, and dynamically adjusts the energy distribution and temperature control of the battery, thereby achieving the minimization of the energy consumption and the optimization of the system performance. The energy efficiency and system performance of the heavy truck can be remarkably improved, and the overall operation efficiency and safety of the truck are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent decision-making method for a heavy truck battery system based on a DQN algorithm and related equipment. Background Art

[0002] As the global climate change problem becomes increasingly serious, the impact of traditional fuel vehicles on the environment has aroused widespread concern. Countries have promoted green and low-carbon transformation, and electric vehicles (EVs) have been rapidly promoted as an important means to achieve sustainable transportation. Among them, electric heavy-duty trucks are an important development direction for electric vehicles. However, the performance and range of electric heavy-duty trucks are still limited by the development of battery technology. Batteries are not only the core components of electric vehicles, but their energy density, charge and discharge efficiency, and life directly determine the vehicle's mileage and economy. In addition, the thermal effect of batteries is also a key factor restricting the development of electric vehicles, especially under high load conditions, the battery is prone to overheating, affecting its performance and safety. Therefore, how to optimize the battery energy management system of heavy-duty trucks has become the focus of current research.

[0003] The traditional battery management system (BMS) mainly relies on static models and empirical rules for management and cannot adapt to the complex and changeable actual driving environment. Summary of the invention

[0004] The embodiment of the present invention provides a heavy-duty truck battery system intelligent decision-making method and related equipment based on the DQN algorithm. Through deep reinforcement learning of the DQN algorithm, dynamic optimization in a complex and changeable driving environment is achieved, which significantly improves the energy efficiency and system performance of heavy-duty trucks, extends the battery life, and improves the overall operating efficiency and safety of the vehicle.

[0005] An embodiment of the present invention provides an intelligent decision-making method for a heavy truck battery system based on a DQN algorithm, comprising:

[0006] Analyze the composition structure of the battery system of heavy trucks and the energy flow relationship of heavy truck operation;

[0007] Based on the composition structure of the battery system and the energy flow relationship of the heavy truck operation, a battery energy consumption calculation model of the heavy truck in operation is constructed;

[0008] Abstracting the constraint conditions and optimization objectives of the battery energy consumption calculation model from the battery energy consumption calculation model;

[0009] The constraints of the battery energy consumption calculation model and the decision of the optimization target are converted into Markov decisions, and the deep reinforcement learning state, action and reward function of the Markov decision model are designed. The state is represented by the battery temperature and the drive system power, the action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck, and the reward function is related to the energy consumption;

[0010] Constructing a deep reinforcement learning DQN model for enabling an intelligent agent of a Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward, and training with historical data to obtain a trained deep reinforcement learning DQN model; the historical data includes the historical data of the state and the historical data of the action;

[0011] Acquiring the status of the battery of the heavy truck in real time;

[0012] Based on the state, the optimal action is calculated through the trained deep reinforcement learning DQN model to dynamically adjust the battery energy distribution of the heavy truck.

[0013] As an improvement of the above solution, the analysis of the composition structure of the battery system of the heavy truck and the relationship between the energy flow of the heavy truck operation includes:

[0014] a. The battery system of the heavy truck is divided into four main modules: a charging module, a drive module, a temperature control module and a low voltage module;

[0015] The charging module includes a voltage conversion device and a charging system;

[0016] The drive module includes a motor and a drive system;

[0017] The temperature control module includes the air conditioning compressor and thermal management system;

[0018] The low-voltage module includes low-voltage devices such as in-vehicle instruments, speakers, and lights.

[0019] b. Analyze the energy flow relationship between the four modules, focusing on the impact of energy flow between modules on vehicle energy consumption;

[0020] c. Simplify the overall energy consumption of low-voltage module equipment, assume that its energy consumption is low, the fluctuation range is small, and the energy supply cannot be delayed, and set its energy consumption to a fixed value;

[0021] d. Ignore the loss in the energy transfer process of the battery system and simplify the energy flow model;

[0022] e. Construct a calculation formula for the instantaneous power of the battery.

[0023] As an improvement of the above solution, the battery energy consumption calculation model is:

[0024]

[0025] is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance.

[0026] As an improvement of the above scheme, the construction is used to enable the intelligent agent of the Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward. The deep reinforcement learning DQN model is trained using historical data to obtain a trained deep reinforcement learning DQN model, including:

[0027] Step 4.1: Build a DQN network and initialize neural network parameters: Build a fully connected neural network with 4 layers, the dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, the input layer and the output layer are the dimensions of state features and the dimensions of action respectively;

[0028] Step 4.2: Set the learning rate α, decay factor γ, and total number of training times T z ;

[0029] Step 4.3: Determine whether the historical data of all time periods have been trained. If it has been completed, the training ends, otherwise, go to step 4.4;

[0030] Step 4.4: Obtain the speed change of the heavy truck during this period and calculate the driving power P at different speeds t D and battery temperature τ t ;

[0031] Step 4.5: Determine whether the total number of training times T is met z , if satisfied, go to step 4.10, otherwise go to step 4.6;

[0032] Step 4.6: Evaluate the current network and the target network to obtain Q * (s,a) and Q t (s,a);

[0033] Step 4.7: Based on Q * (s,a) uses the ε-greedy strategy to select action a t , and perform a toperation; Among them, the ε-greedy strategy is introduced to help the agent choose actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1; if rand>ε, the agent chooses the action with the highest Q value in the current state, otherwise it randomly selects the action; the value of ε changes with time, t x is the number of trainings from the beginning to now, T z is the total number of training times for the model;

[0034] Step 4.8: Calculate the total energy consumption E t and reward r t ;

[0035] Step 4.9: Update the evaluation network parameters.

[0036] Another embodiment of the present invention provides a heavy truck battery system intelligent decision-making device based on a DQN algorithm, including:

[0037] An analysis module is used to analyze the composition structure of the battery system of a heavy truck and the energy flow relationship of the heavy truck operation;

[0038] A construction module, for constructing a battery energy consumption calculation model of the heavy truck in operation based on the composition structure of the battery system and the energy flow relationship of the heavy truck operation;

[0039] An extraction module, used to abstract the constraints and optimization objectives of the battery energy consumption calculation model from the battery energy consumption calculation model;

[0040] A conversion module is used to convert the constraints of the battery energy consumption calculation model and the decision of the optimization target into Markov decision, and design the deep reinforcement learning state, action and reward function of the Markov decision model, where the state is represented by the battery temperature and the drive system power, the action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck, and the reward function is related to the energy consumption;

[0041] A training module is used to construct a deep reinforcement learning DQN model for enabling an intelligent agent of a Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward, and to train with historical data to obtain a trained deep reinforcement learning DQN model; the historical data includes the historical data of the state and the historical data of the action;

[0042] An acquisition module, used for acquiring the status of the battery of the heavy truck in real time;

[0043] The decision module is used to dynamically adjust the battery energy distribution of the heavy truck according to the state and calculate the optimal action through the trained deep reinforcement learning DQN model.

[0044] As an improvement of the above solution, the analysis module is specifically used for:

[0045] a. The battery system of the heavy truck is divided into four main modules: a charging module, a drive module, a temperature control module and a low voltage module;

[0046] The charging module includes a voltage conversion device and a charging system;

[0047] The drive module includes a motor and a drive system;

[0048] The temperature control module includes the air conditioning compressor and thermal management system;

[0049] The low-voltage module includes low-voltage devices such as in-vehicle instruments, speakers, and lights.

[0050] b. Analyze the energy flow relationship between the four modules, focusing on the impact of energy flow between modules on vehicle energy consumption;

[0051] c. Simplify the overall energy consumption of low-voltage module equipment, assume that its energy consumption is low, the fluctuation range is small, and the energy supply cannot be delayed, and set its energy consumption to a fixed value;

[0052] d. Ignore the loss in the energy transfer process of the battery system and simplify the energy flow model;

[0053] e. Construct a calculation formula for the instantaneous power of the battery.

[0054] As an improvement of the above solution, the battery energy consumption calculation model is:

[0055]

[0056] is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance.

[0057] As an improvement of the above solution, the training module is specifically used for:

[0058] Build a DQN network and initialize neural network parameters: Build a fully connected neural network with 4 layers. The dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, and the input layer and output layer are the dimensions of state features and action respectively;

[0059] Set the learning rate α, decay factor γ, and total number of training times T z ;

[0060] Determine whether the historical data of all time periods have been trained. If so, the training ends; otherwise, continue to execute;

[0061] Obtain the speed change of heavy trucks during this period and calculate the driving power P at different speeds t D and battery temperature τ t ;

[0062] Determine whether the total number of training times T is met z , if satisfied, go to step 4.10, otherwise go to step 4.6;

[0063] Evaluate the current network and the target network to obtain Q * (s,a) and Q t (s,a);

[0064] Based on Q * (s,a) uses the ε-greedy strategy to select action a t , and perform a t operation; Among them, the ε-greedy strategy is introduced to help the agent choose actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1; if rand>ε, the agent chooses the action with the highest Q value in the current state, otherwise it randomly selects the action; the value of ε changes with time, t x is the number of trainings from the beginning to now, T z is the total number of training times for the model;

[0065] Calculate the total energy consumption E t and reward r t ;

[0066] Update evaluation network parameters.

[0067] Another embodiment of the present invention provides a heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the heavy-duty truck battery system intelligent decision-making method based on the DQN algorithm described in the above-mentioned embodiment of the invention.

[0068] Another embodiment of the present invention provides a storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the heavy-duty truck battery system intelligent decision-making method based on the DQN algorithm described in the above-mentioned invention embodiment.

[0069] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0070] The embodiment of the present invention first analyzes the composition structure and operating energy flow relationship of the heavy truck battery system, and constructs a detailed battery energy consumption calculation model; then, it abstracts the constraints and optimization goals from the energy consumption calculation model, and transforms the decision problem into a Markov decision process (MDP); by designing the state, action and reward function of deep reinforcement learning (DQN), a DQN model is constructed, and historical data is used for training to generate an intelligent model that can select the optimal action according to the current state; in actual operation, the system obtains the state of the battery in real time, and calculates the optimal action through the trained DQN model, dynamically adjusts the energy distribution and temperature control of the battery, thereby minimizing energy consumption and optimizing system performance. The embodiment of the present invention realizes dynamic optimization in complex and changeable driving environments through the deep reinforcement learning DQN algorithm, significantly improves the energy efficiency and system performance of heavy trucks, extends the battery's cruising range, and improves the overall operating efficiency and safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a flowchart of an intelligent decision-making method for a heavy truck battery system based on a DQN algorithm provided by an embodiment of the present invention;

[0072] Figure 2 It is a structural schematic diagram of an intelligent decision-making device for a heavy truck battery system based on a DQN algorithm provided by an embodiment of the present invention;

[0073] Figure 3 It is a structural schematic diagram of a heavy truck battery system intelligent decision-making device based on a DQN algorithm provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] See also Figure 1 , is a flowchart of a heavy truck battery system intelligent decision-making method based on a DQN algorithm provided by an embodiment of the present invention. The heavy truck battery system intelligent decision-making method based on a DQN algorithm comprises steps S10 to S16:

[0076] S10, analyzing the composition structure of the battery system of heavy trucks and the relationship between the energy flow of heavy truck operation;

[0077] S11, constructing a battery energy consumption calculation model of the heavy truck in operation based on the composition structure of the battery system and the energy flow relationship of the heavy truck operation;

[0078] S12, abstracting the constraint conditions and optimization objectives of the battery energy consumption calculation model from the battery energy consumption calculation model;

[0079] S13, converting the constraints of the battery energy consumption calculation model and the decision of the optimization target into Markov decision, designing the deep reinforcement learning state, action and reward function of the Markov decision model, the state is represented by the battery temperature and the drive system power, the action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck, and the reward function is related to the energy consumption;

[0080] S14, constructing a deep reinforcement learning DQN model for enabling the intelligent agent of the Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward, and training with historical data to obtain a trained deep reinforcement learning DQN model; the historical data includes the historical data of the state and the historical data of the action;

[0081] S15, acquiring the status of the battery of the heavy truck in real time;

[0082] S16, based on the state, the optimal action is calculated through the trained deep reinforcement learning DQN model to dynamically adjust the battery energy distribution of the heavy truck.

[0083] The embodiment of the present invention first analyzes the composition structure and operating energy flow relationship of the heavy truck battery system, and constructs a detailed battery energy consumption calculation model; then, it abstracts the constraints and optimization goals from the energy consumption calculation model, and transforms the decision problem into a Markov decision process (MDP); by designing the state, action and reward function of deep reinforcement learning (DQN), a DQN model is constructed, and historical data is used for training to generate an intelligent model that can select the optimal action according to the current state; in actual operation, the system obtains the state of the battery in real time, and calculates the optimal action through the trained DQN model, dynamically adjusts the energy distribution and temperature control of the battery, thereby minimizing energy consumption and optimizing system performance. The embodiment of the present invention realizes dynamic optimization in complex and changeable driving environments through the deep reinforcement learning DQN algorithm, significantly improves the energy efficiency and system performance of heavy trucks, extends the battery's cruising range, and improves the overall operating efficiency and safety of the vehicle.

[0084] It should be noted that by studying the system structure and the relationship between energy flow, a basis is provided for the energy consumption calculation model in the next step. Understanding the composition of the system and the energy flow is the premise for establishing the energy consumption model. Then a detailed energy consumption calculation model and optimization target are established, which provides a mathematical basis for the Markov decision process (MDP) in the next step. The state variables and optimization targets in the energy consumption model are the key to designing the MDP. Then the MDP and reinforcement learning framework are designed, which provides a theoretical basis for the DQN model training in the next step. The training of the DQN model requires clear states, actions, and reward functions. Then the DQN model is trained to generate an intelligent model that can select the optimal action based on the current state. The last step uses this trained model to generate intelligent decision-making plans in actual operation, and execute these plans to achieve dynamic adjustment.

[0085] As an improvement of the above solution, the analysis of the composition structure of the battery system of the heavy truck and the relationship between the energy flow of the heavy truck operation includes:

[0086] a. The battery system of the heavy truck is divided into four main modules: a charging module, a drive module, a temperature control module and a low voltage module;

[0087] The charging module includes a voltage conversion device and a charging system;

[0088] The drive module includes a motor and a drive system;

[0089] The temperature control module includes the air conditioning compressor and thermal management system;

[0090] The low-voltage module includes low-voltage devices such as in-vehicle instruments, speakers, and lights.

[0091] b. Analyze the energy flow relationship between the four modules, focusing on the impact of energy flow between modules on vehicle energy consumption;

[0092] c. Simplify the overall energy consumption of low-voltage module equipment, assume that its energy consumption is low, the fluctuation range is small, and the energy supply cannot be delayed, and set its energy consumption to a fixed value;

[0093] d. Ignore the loss in the energy transfer process of the battery system and simplify the energy flow model;

[0094] e. Construct a calculation formula for the instantaneous power of the battery.

[0095] In this embodiment, specifically, the heavy-duty truck battery energy system is first divided into four parts: a charging module, a drive module, a temperature control module, and a low-voltage module. The charging module refers to the voltage conversion device and the charging system; the drive module is mainly the motor and the drive system; the temperature control module is mainly the air-conditioning compressor and the thermal management system; the low-voltage module mainly refers to the low-voltage devices such as the in-vehicle instruments, speakers, and lights. During the operation of heavy-duty trucks, the energy flow within each module has little effect on the energy consumption of the entire vehicle, while the energy flow between modules is the main source of energy consumption. Appendix Figure 2 A schematic diagram showing the energy flow between modules when the vehicle is in operation. Since the vehicle is in a non-charging state by default, there is no need to consider the charging module.

[0096] Among them, P B Indicates battery power, positive value indicates battery discharge, negative value indicates battery charging; P L Represents the total power of the low voltage unit, P H Indicates the power of the battery thermal management system; P D Indicates driving power, P D ≥0 means the drive system is outputting power, P D <0 means the drive system is braking and recovering kinetic energy, |P D |Indicates the output power or braking regenerative energy.

[0097] In the embodiment of the present invention, in order to simplify the energy flow relationship, the following provisions are made: the overall energy consumption of the low-voltage module equipment is low, the fluctuation range is small, and the energy supply cannot be delayed, that is, P L is set to a fixed value. H All of it is used for battery temperature control, and the energy transfer loss in the energy system is ignored. Therefore, the instantaneous power of the battery Among them, P L is a fixed value, so heavy-duty trucks mainly need to develop appropriate strategies to reduce the energy consumption of the drive system and thermal management system during operation.

[0098] In this embodiment, the composition structure and operating energy flow relationship of the heavy truck battery system are analyzed in detail, and it is divided into a charging module, a driving module, a temperature control module and a low voltage module. By analyzing the energy flow relationship between the modules, the focus is on the impact of the energy flow between the modules on the energy consumption of the whole vehicle, and the energy consumption of the low-voltage module equipment is simplified, and its energy consumption is set to be low and fixed. At the same time, the loss in the energy transfer process is ignored, the energy flow model is simplified, and the calculation formula of the instantaneous power of the battery is constructed. This process provides a solid foundation for the subsequent energy consumption calculation model and optimization strategy. This embodiment clarifies the energy flow relationship between the modules of the battery system through detailed analysis and simplified assumptions, and provides accurate data support for the establishment of the energy consumption calculation model. By simplifying the model, the calculation complexity is reduced, and the practicality and accuracy of the model are improved. This helps to more effectively optimize the battery energy distribution and temperature control in practical applications, thereby significantly reducing energy consumption and improving the operating efficiency and system performance of heavy trucks.

[0099] For ease of understanding, the above embodiment is described below by way of example:

[0100] First, the heavy-duty truck battery energy system is divided into four main modules:

[0101] Charging module: includes voltage conversion equipment and charging system.

[0102] Drive module: includes motor and drive system.

[0103] Temperature control module: includes air conditioning compressor and thermal management system.

[0104] Low voltage module: including low voltage devices such as in-vehicle instruments, speakers, and lights.

[0105] During the operation of a heavy truck, the energy flow within each module has little impact on the energy consumption of the entire vehicle, while the energy flow between modules is the main source of energy consumption. For example, since the vehicle is in a non-charging state by default during operation, there is no need to consider the charging module. Specifically, the instantaneous power P of the battery b It can be expressed as: b =P d +P t +P lv , P d Indicates driving power, positive value indicates battery discharge, negative value indicates charging. t Indicates the battery thermal management system power. lv Indicates the total power of the low voltage unit, which is a fixed value.

[0106] In order to simplify the energy flow relationship, this example makes the following provisions:

[0107] Low voltage module: The overall energy consumption of low voltage module equipment is low, the fluctuation range is small, and the energy supply cannot be delayed, so its energy consumption is set to a fixed value.

[0108] Energy transfer loss: Ignore the loss of energy transfer in the energy system to simplify the model calculation.

[0109] With these simplifying assumptions, the instantaneous power of the battery is mainly determined by the energy consumption of the drive system and thermal management system. Therefore, heavy trucks need to develop appropriate strategies to reduce the energy consumption of the drive system and thermal management system during operation, thereby minimizing the overall energy consumption.

[0110] Assume that a heavy truck is traveling on a highway at a speed of 80 km / h, with a vehicle mass of 20 tons, a ground resistance coefficient of 0.015, a wind resistance coefficient of 0.6, a frontal area of ​​7 square meters, and an air density of 1.225 kg / m 3 According to the above model, the instantaneous power of the drive system is calculated as follows:

[0111] Assuming the vehicle speed is constant and the acceleration is 0, the acceleration power P accel : Ground resistance power P resistance : Air resistance power P air : Therefore, the instantaneous power of the drive system is: P d =6533.33+12247.36=18780.69W Assuming the total power of the low voltage unit P lv 1000W, battery thermal management system power P t is 2000W, then the instantaneous power of the battery P b For: P b =18780.69+2000+1000=21780.69W.

[0112] As an improvement of the above solution, the battery energy consumption calculation model is:

[0113]

[0114] is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance.

[0115] In this embodiment, specifically, a battery energy consumption calculation model for a heavy truck in operation is constructed, and constraints and optimization objectives are abstracted and expressed as mathematical functions.

[0116] Heavy trucks in a period of time (t 1 , t 2 The total battery energy consumption E during the period is mainly composed of low-voltage unit energy consumption, driving energy consumption, thermal management energy consumption and the actual discharge amount that changes with temperature during this period, which can be expressed as:

[0117]

[0118] is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance. In addition, during deceleration, the braking energy recovery and mechanical braking systems will intervene and convert the remaining kinetic energy into electrical energy to charge the battery as much as possible.

[0119]

[0120] v is the instantaneous speed, M is the vehicle mass, ω is the vehicle mass conversion coefficient, g represents the acceleration of gravity, β is the ground resistance coefficient, represents the drag coefficient, and Δ is the frontal windward area of ​​the vehicle.

[0121] is the actual discharge capacity of the battery as the temperature τ changes.

[0122] and For the battery at t 1 and t 2 The discharge capacity at the moment, E B is the standard discharge capacity of the battery. The discharge capacity of the battery at different temperatures is different and the discharge capacity that changes with temperature is recoverable. Therefore, a thermal management system should be used to maintain the battery temperature in the efficient discharge range as much as possible, that is, when the battery temperature rises, the thermal management system cools the battery. When the battery temperature is too low, the thermal management system heats the battery. If the energy consumption required for the thermal management system to do work is less than the discharge capacity recovered by the battery under its action, it means that the battery thermal management system has played a role in reducing energy consumption.

[0123] The battery heat dissipation power is η is the cooling efficiency coefficient of the thermal management system. The battery heat generation power is i t is the instantaneous current, Ω is the resistance, τ is the battery temperature, and ξ is the heat generation coefficient of the battery. Then, the instantaneous thermal power of the battery is When it is positive, the battery temperature increases, and vice versa. is the instantaneous power of the thermal management system, is the maximum power of the thermal management system. λ∈[0,1] is the thermal management system control coefficient, λ=1 means the thermal management system works at the maximum power, λ=0 means the thermal management system does not work.

[0124] In reality, it is always hoped that the energy consumed by vehicle operation is as little as possible. Therefore, the optimization goal of the present invention is to minimize the heavy truck battery energy system in a period of driving time (t 1 ,t 2 ) total energy consumption E(t 1 ,t 2 ),Right now

[0125] In this embodiment, by constructing a battery energy consumption calculation model under the operating state of a heavy truck, the influence of the actual discharge amount of the drive system, low-voltage unit, thermal management system and battery with temperature changes on the total energy consumption is analyzed in detail. Specifically, the instantaneous power of the drive system consists of acceleration power, power required to overcome ground resistance and power required to overcome air resistance. These powers are represented by mathematical functions, and a battery energy consumption calculation model is established in combination with the energy consumption of the thermal management system and the temperature change of the battery. The optimization goal is to achieve efficient use of energy by minimizing the total energy consumption of the heavy truck battery energy system over a period of driving time. This embodiment can accurately evaluate and control the energy consumption of heavy trucks under various working conditions through a detailed energy consumption calculation model and optimization goals. By dynamically adjusting the working state of the battery energy distribution and the thermal management system, the energy consumption of the entire vehicle is significantly reduced, the battery life and the overall performance of the system are improved, thereby improving the operating efficiency and economy of heavy trucks.

[0126] As an improvement to your above solution, the constraints of the battery energy consumption calculation model and the decision of the optimization target are converted into Markov decisions, and the deep reinforcement learning state, action and reward function of the Markov decision model are designed. The state is represented by the battery temperature and the drive system power. The action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck. The reward function is related to the energy consumption, specifically:

[0127] Markov Decision Process (MDP) is a mathematical framework for decision-making problems, which is widely used in fields such as artificial intelligence, economics, and operations research. It describes how an agent chooses actions based on the current state in an uncertain environment to maximize long-term returns by defining elements such as state, action, transition probability, and reward. In MDP, the impact of each decision depends not only on the current action, but also on the possible future states and corresponding rewards.

[0128] In the optimization scheduling of heavy truck battery energy systems, the present invention converts the battery optimization decision problem into a Markov decision process, represented by a 4-tuple (S, A, P, R). At each decision point, the agent obtains the current battery state s t ∈S, select the currently executable action a according to the strategy π(S→A) t ∈A, through the state transition probability p(s t+1 |s t ,a t ) Enter the new state s t+1 And calculate the immediate reward r t ∈R. Here, define Q π (s,a) is the action value function for selecting action a according to a given strategy π in state s.

[0129]

[0130] Among them, α is the learning rate, which is used to control the update speed of the Q value. γ is the decay coefficient, which indicates the influence of future rewards on the current solution. The goal of deep reinforcement learning DQN is to allow the agent to obtain an optimal strategy π * Maximize the expected cumulative reward.

[0131]

[0132] State design: In heavy truck battery energy systems, the battery state and energy allocation strategy mainly depend on the current battery temperature τ t and drive system power Therefore, the state can be represented by a 2-tuple, Action design: In the calculation model of step 2, λ is the thermal management system control coefficient, which mainly controls the operating power of the thermal management system to regulate the battery energy system. Therefore, the action is defined as λ∈A, A is the set of all actions, and the value of λ is between 0-1, indicating that the thermal management system is working at a specific power between shutdown and full power.

[0133] Reward function: The design of the reward function should be related to the objective function. The optimization target of this invention is energy consumption, so the reward function is set as is the average reward obtained at the current state at the current moment, E t is the energy consumption at the current moment.

[0134] In this embodiment, an intelligent decision-making framework is constructed by converting the constraints of the battery energy consumption calculation model and the decision of the optimization target into a Markov decision process (MDP), and by designing the state, action and reward function of deep reinforcement learning (DQN). Specifically, the state is represented by the battery temperature and the drive system power, the action is defined as the control coefficient of the thermal management system, and the reward function is related to energy consumption. Through the deep reinforcement learning DQN model, the agent can select the optimal action according to the current state at each decision point to maximize the long-term cumulative reward, thereby minimizing energy consumption and optimizing system performance.

[0135] It is understandable that the construction process and application process of Markov decision making can refer to the existing technology and will not be elaborated here.

[0136] As an improvement of the above scheme, the construction is used to enable the intelligent agent of the Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward. The deep reinforcement learning DQN model is trained using historical data to obtain a trained deep reinforcement learning DQN model, including:

[0137] Step 4.1: Build a DQN network and initialize neural network parameters: Build a fully connected neural network with 4 layers, the dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, the input layer and the output layer are the dimensions of state features and the dimensions of action respectively;

[0138] Step 4.2: Set the learning rate α, decay factor γ, and total number of training times T z ;

[0139] Step 4.3: Determine whether the historical data of all time periods have been trained. If it has been completed, the training ends, otherwise, go to step 4.4;

[0140] Step 4.4: Obtain the speed change of the heavy truck during this period and calculate the driving power at different speeds and battery temperature τ t ;

[0141] Step 4.5: Determine whether the total number of training times T is met z , if satisfied, go to step 4.10, otherwise go to step 4.6;

[0142] Step 4.6: Evaluate the current network and the target network to obtain Q * (s,a) and Q t (s,a);

[0143] Step 4.7: Based on Q * (s,a) uses the ε-greedy strategy to select action a t, and perform a t operation; Among them, the ε-greedy strategy is introduced to help the agent choose actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1; if rand>ε, the agent chooses the action with the highest Q value in the current state, otherwise it randomly selects the action; the value of ε changes with time, y x is the number of trainings from the beginning to now, T z is the total number of training times for the model;

[0144] Step 4.8: Calculate the total energy consumption E t and reward r t ;

[0145] Step 4.9: Update the evaluation network parameters.

[0146] In this embodiment, a fully connected neural network with 4 layers is constructed. The dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, and the input layer and the output layer are the dimensions of state features and the dimensions of actionable actions, respectively. In order to promote the balance between exploration and utilization in method training, the ε-greedy strategy is introduced to help the agent select actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1. If rand>ε, the agent selects the actionable action with the highest Q value in the current state, otherwise it randomly selects the actionable action. The value of ε changes over time, t x is the number of trainings from the beginning to now, T z is the total number of model training times.

[0147] The model is trained on the behavior of the vehicle in selecting actions as the state changes over a period of time. Therefore, before model training, the historical data must first be preprocessed, that is, the historical data must be divided into several continuous time periods. The detailed training process is as above.

[0148] In this embodiment, by constructing a deep reinforcement learning DQN model, the agent can train and obtain an optimal strategy based on historical data to maximize the expected cumulative reward. Specifically, a 4-layer fully connected neural network is constructed, and the learning rate, decay factor and total number of training times are set. Through the iterative training process, the agent selects the optimal action according to the current state at each decision point, calculates the total energy consumption and reward after executing the action, and updates the network parameters. This process ensures that the agent can gradually optimize its decision-making strategy, thereby minimizing energy consumption and optimizing system performance.

[0149] It should be noted that the model training process and application process of building a deep reinforcement learning DQN model can refer to the existing technology.

[0150] See also Figure 2 , is a structural diagram of a heavy truck battery system intelligent decision-making device based on a DQN algorithm provided by an embodiment of the present invention. The heavy truck battery system intelligent decision-making device based on a DQN algorithm comprises:

[0151] An analysis module 10 is used to analyze the composition structure of the battery system of the heavy truck and the energy flow relationship of the heavy truck operation;

[0152] A construction module 11 is used to construct a battery energy consumption calculation model of the heavy truck in operation based on the composition structure of the battery system and the energy flow relationship of the heavy truck operation;

[0153] An extraction module 12, used to abstract the constraints and optimization objectives of the battery energy consumption calculation model from the battery energy consumption calculation model;

[0154] A conversion module 13 is used to convert the constraints of the battery energy consumption calculation model and the decision of the optimization target into a Markov decision, and design the deep reinforcement learning state, action and reward function of the Markov decision model, where the state is represented by the battery temperature and the drive system power, the action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck, and the reward function is related to the energy consumption;

[0155] A training module 14 is used to construct a deep reinforcement learning DQN model for enabling an intelligent agent of the Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward, and to train with historical data to obtain a trained deep reinforcement learning DQN model; the historical data includes the historical data of the state and the historical data of the action;

[0156] An acquisition module 15, used for acquiring the status of the battery of the heavy truck in real time;

[0157] The decision module 16 is used to dynamically adjust the battery energy distribution of the heavy truck according to the state and calculate the optimal action through the trained deep reinforcement learning DQN model.

[0158] The embodiment of the present invention first analyzes the composition structure and operating energy flow relationship of the heavy truck battery system, and constructs a detailed battery energy consumption calculation model; then, it abstracts the constraints and optimization goals from the energy consumption calculation model, and transforms the decision problem into a Markov decision process (MDP); by designing the state, action and reward function of deep reinforcement learning (DQN), a DQN model is constructed, and historical data is used for training to generate an intelligent model that can select the optimal action according to the current state; in actual operation, the system obtains the state of the battery in real time, and calculates the optimal action through the trained DQN model, dynamically adjusts the energy distribution and temperature control of the battery, thereby minimizing energy consumption and optimizing system performance. The embodiment of the present invention realizes dynamic optimization in complex and changeable driving environments through the deep reinforcement learning DQN algorithm, significantly improves the energy efficiency and system performance of heavy trucks, extends the battery's cruising range, and improves the overall operating efficiency and safety of the vehicle.

[0159] As an improvement of the above solution, the analysis module is specifically used for:

[0160] a. The battery system of the heavy truck is divided into four main modules: a charging module, a drive module, a temperature control module and a low voltage module;

[0161] The charging module includes a voltage conversion device and a charging system;

[0162] The drive module includes a motor and a drive system;

[0163] The temperature control module includes the air conditioning compressor and thermal management system;

[0164] The low-voltage module includes low-voltage devices such as in-vehicle instruments, speakers, and lights.

[0165] b. Analyze the energy flow relationship between the four modules, focusing on the impact of energy flow between modules on vehicle energy consumption;

[0166] c. Simplify the overall energy consumption of low-voltage module equipment, assume that its energy consumption is low, the fluctuation range is small, and the energy supply cannot be delayed, and set its energy consumption to a fixed value;

[0167] d. Ignore the loss in the energy transfer process of the battery system and simplify the energy flow model;

[0168] e. Construct a calculation formula for the instantaneous power of the battery.

[0169] As an improvement of the above solution, the battery energy consumption calculation model is:

[0170]

[0171] is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance.

[0172] As an improvement of the above solution, the training module is specifically used for:

[0173] Build a DQN network and initialize neural network parameters: Build a fully connected neural network with 4 layers. The dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, and the input layer and output layer are the dimensions of state features and action respectively;

[0174] Set the learning rate α, decay factor γ, and total number of training times T z ;

[0175] Determine whether the historical data of all time periods have been trained. If so, the training ends; otherwise, continue to execute;

[0176] Obtain the speed change of heavy trucks during this period and calculate the driving power at different speeds and battery temperature τ t ;

[0177] Determine whether the total number of training times T is met z , if satisfied, go to step 4.10, otherwise go to step 4.6;

[0178] Evaluate the current network and the target network to obtain Q * (s,a) and Q t (s,a);

[0179] Based on Q * (s,a) uses the ε-greedy strategy to select action a t , and perform a t operation; Among them, the ε-greedy strategy is introduced to help the agent choose actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1; if rand>ε, the agent chooses the action with the highest Q value in the current state, otherwise it randomly selects the action; the value of ε changes with time, t x is the number of trainings from the beginning to now, T z is the total number of training times for the model;

[0180] Calculate the total energy consumption E t and reward r t ;

[0181] Update evaluation network parameters.

[0182] See also Figure 3 , is a schematic diagram of a heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm provided in one embodiment of the present invention. The heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm in this embodiment includes: a processor 100, a memory 101, and a computer program stored in the memory 101 and executable on the processor 100, such as a heavy-duty truck battery system intelligent decision-making program based on the DQN algorithm. When the processor 100 executes the computer program, the steps in the above-mentioned embodiments of the heavy-duty truck battery system intelligent decision-making method based on the DQN algorithm are implemented. Alternatively, when the processor 100 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0183] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the intelligent decision-making device for the heavy truck battery system based on the DQN algorithm.

[0184] The heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm may be a control device such as a main control module of a truck. The heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the schematic diagram is only an example of a heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm, and does not constitute a limitation on the heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the heavy-duty truck battery system intelligent decision-making device based on the DQN algorithm may also include input and output devices, network access devices, buses, etc.

[0185] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the intelligent decision-making device for the heavy-duty truck battery system based on the DQN algorithm, and uses various interfaces and lines to connect the various parts of the intelligent decision-making device for the heavy-duty truck battery system based on the DQN algorithm.

[0186] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the heavy truck battery system intelligent decision-making device based on the DQN algorithm by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0187] Wherein, if the module / unit integrated in the intelligent decision-making device of the heavy truck battery system based on the DQN algorithm is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0188] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0189] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An intelligent decision-making method for a heavy truck battery system based on a DQN algorithm, characterized in that: include: Analyze the composition structure of the battery system of heavy trucks and the energy flow relationship of heavy truck operation; Based on the composition structure of the battery system and the energy flow relationship of the heavy truck operation, a battery energy consumption calculation model of the heavy truck in operation is constructed; Abstracting the constraint conditions and optimization objectives of the battery energy consumption calculation model from the battery energy consumption calculation model; The constraints of the battery energy consumption calculation model and the decision of the optimization target are converted into Markov decisions, and the deep reinforcement learning state, action and reward function of the Markov decision model are designed. The state is represented by the battery temperature and the drive system power, the action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck, and the reward function is related to the energy consumption; Constructing a deep reinforcement learning DQN model for enabling an intelligent agent of a Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward, and training with historical data to obtain a trained deep reinforcement learning DQN model; the historical data includes the historical data of the state and the historical data of the action; Acquiring the status of the battery of the heavy truck in real time; Based on the state, the optimal action is calculated through the trained deep reinforcement learning DQN model to dynamically adjust the battery energy distribution of the heavy truck.

2. The intelligent decision-making method for a heavy truck battery system based on a DQN algorithm as claimed in claim 1, characterized in that: The analysis of the composition structure of the battery system of a heavy truck and the relationship between the energy flow of the heavy truck operation includes: a. The battery system of the heavy truck is divided into four main modules: a charging module, a drive module, a temperature control module and a low voltage module; The charging module includes a voltage conversion device and a charging system; The drive module includes a motor and a drive system; The temperature control module includes the air conditioning compressor and thermal management system; The low-voltage module includes low-voltage devices such as in-vehicle instruments, speakers, and lights. b. Analyze the energy flow relationship between the four modules, focusing on the impact of energy flow between modules on vehicle energy consumption; c. Simplify the overall energy consumption of low-voltage module equipment, assume that its energy consumption is low, the fluctuation range is small, and the energy supply cannot be delayed, and set its energy consumption to a fixed value; d. Ignore the loss in the energy transfer process of the battery system and simplify the energy flow model; e. Construct a calculation formula for the instantaneous power of the battery.

3. The intelligent decision-making method for a heavy truck battery system based on a DQN algorithm as claimed in claim 1, characterized in that: The battery energy consumption calculation model is: is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance.

4. The intelligent decision-making method for a heavy truck battery system based on a DQN algorithm as claimed in claim 1, characterized in that: The construction is used to enable the intelligent agent of the Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward of the deep reinforcement learning DQN model, and use historical data training to obtain a trained deep reinforcement learning DQN model, including: Step 4.1: Build a DQN network and initialize neural network parameters: Build a fully connected neural network with 4 layers, the dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, the input layer and the output layer are the dimensions of state features and the dimensions of action respectively; Step 4.2: Set the learning rate α, decay factor γ, and total number of training times T z ; Step 4.3: Determine whether the historical data of all time periods have been trained. If it has been completed, the training ends, otherwise, go to step 4.4; Step 4.4: Obtain the speed change of the heavy truck during this period and calculate the driving power at different speeds and battery temperature τ t ; Step 4.5: Determine whether the total number of training times T is met z , if satisfied, go to step 4.10, otherwise go to step 4.6; Step 4.6: Evaluate the current network and the target network to obtain Q * (s,a) and Q t (s,a); Step 4.7: Based on Q * (s,a) uses the ε-greedy strategy to select action a t , and perform a t operation; Among them, the ε-greedy strategy is introduced to help the agent choose actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1; if rand>ε, the agent chooses the action with the highest Q value in the current state, otherwise it randomly selects the action; the value of ε changes with time, t x is the number of trainings from the beginning to now, T z is the total number of training times for the model; Step 4.8: Calculate the total energy consumption E t and reward r t ; Step 4.9: Update the evaluation network parameters.

5. A heavy truck battery system intelligent decision-making device based on DQN algorithm, characterized in that: include: An analysis module is used to analyze the composition structure of the battery system of a heavy truck and the energy flow relationship of the heavy truck operation; A construction module, for constructing a battery energy consumption calculation model of the heavy truck in operation based on the composition structure of the battery system and the energy flow relationship of the heavy truck operation; An extraction module, used to abstract the constraints and optimization objectives of the battery energy consumption calculation model from the battery energy consumption calculation model; A conversion module is used to convert the constraints of the battery energy consumption calculation model and the decision of the optimization target into Markov decision, and design the deep reinforcement learning state, action and reward function of the Markov decision model, where the state is represented by the battery temperature and the drive system power, the action is defined as the value range of the battery energy adjustment parameter of the battery management system of the heavy truck, and the reward function is related to the energy consumption; A training module is used to construct a deep reinforcement learning DQN model for enabling an intelligent agent of a Markov decision model to obtain an optimal strategy to maximize the expected cumulative reward, and to train with historical data to obtain a trained deep reinforcement learning DQN model; the historical data includes the historical data of the state and the historical data of the action; An acquisition module, used for acquiring the status of the battery of the heavy truck in real time; The decision module is used to dynamically adjust the battery energy distribution of the heavy truck according to the state and calculate the optimal action through the trained deep reinforcement learning DQN model.

6. The heavy truck battery system intelligent decision-making device based on the DQN algorithm as claimed in claim 5, characterized in that: The analysis module is specifically used for: a. The battery system of the heavy truck is divided into four main modules: a charging module, a drive module, a temperature control module and a low voltage module; The charging module includes a voltage conversion device and a charging system; The drive module includes a motor and a drive system; The temperature control module includes the air conditioning compressor and thermal management system; The low-voltage module includes low-voltage devices such as in-vehicle instruments, speakers, and lights. b. Analyze the energy flow relationship between the four modules, focusing on the impact of energy flow between modules on vehicle energy consumption; c. Simplify the overall energy consumption of low-voltage module equipment, assume that its energy consumption is low, the fluctuation range is small, and the energy supply cannot be delayed, and set its energy consumption to a fixed value; d. Ignore the loss in the energy transfer process of the battery system and simplify the energy flow model; e. Construct a calculation formula for the instantaneous power of the battery.

7. The heavy truck battery system intelligent decision-making device based on the DQN algorithm as claimed in claim 5, characterized in that: The battery energy consumption calculation model is: is the instantaneous power of the drive system, in, is the acceleration power, The power required to overcome ground resistance is The power required to overcome air resistance.

8. The heavy truck battery system intelligent decision-making device based on the DQN algorithm as claimed in claim 5, characterized in that: The training module is specifically used for: Build a DQN network and initialize neural network parameters: Build a fully connected neural network with 4 layers. The dimension of the first hidden layer is 256, the dimension of the second hidden layer is 128, and the input layer and output layer are the dimensions of state features and action respectively; Set the learning rate α, decay factor γ, and total number of training times T z ; Determine whether the historical data of all time periods have been trained. If so, the training ends; otherwise, continue to execute; Obtain the speed change of heavy trucks during this period and calculate the driving power at different speeds and battery temperature τ t ; Determine whether the total number of training times T is met z , if satisfied, go to step 4.10, otherwise go to step 4.6; Evaluate the current network and the target network to obtain Q * (s,a) and Q t (s,a); Based on Q * (s,a) uses the ε-greedy strategy to select action a t , and perform a t operation; Among them, the ε-greedy strategy is introduced to help the agent choose actions, that is, at each scheduling point, a random number rand is randomly generated, and its value range is 0-1; if rand>ε, the agent chooses the action with the highest Q value in the current state, otherwise it randomly selects the action; the value of ε changes with time, t x is the number of trainings from the beginning to now, T z is the total number of training times for the model; Calculate the total energy consumption E t and reward r t ; Update evaluation network parameters.

9. A heavy truck battery system intelligent decision-making device based on DQN algorithm, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the intelligent decision-making method for the heavy truck battery system based on the DQN algorithm as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the intelligent decision-making method for a heavy truck battery system based on the DQN algorithm as described in any one of claims 1 to 4.

Images