Q-learning based thermal management control method for pure electric vehicle

Abstract

The application discloses a kind of pure electric vehicle thermal management control methods based on Q-learning, comprising: S1. current state information of vehicle is collected;S2. determine vehicle demand power;S3. based on current state information of vehicle and vehicle demand power, optimal battery heating power and optimal passenger compartment heating power are solved using Q-learning algorithm;S4. battery and passenger compartment are heated by optimal battery heating power and optimal passenger compartment heating power respectively.The application can adjust thermal management control in real time in actual application process, and control the temperature of integrated thermal management system within the optimum working temperature range.

Description

Technical Field

[0001] This invention relates to the field of thermal management of electric vehicles, and more specifically to a thermal management control method for pure electric vehicles based on Q-learning. Background Technology

[0002] In recent years, to alleviate the harm caused by environmental pollution, the country has proposed the "dual carbon" target, requiring the automotive industry to transform towards low-carbon development. Pure electric vehicles have become the future trend of the industry due to their advantages such as zero pollution and zero emissions. However, as the only power source for pure electric vehicles, the power battery is severely affected by temperature, especially in high and low temperature environments, which hinders the further promotion of pure electric vehicles in the market.

[0003] Currently, shifting from a single-mode to an integrated mode of thermal management has become a development trend in the thermal management control of pure electric vehicles. The integrated mode primarily combines battery thermal management, motor thermal management, and passenger compartment thermal management for energy management. However, most existing technologies rely on rule-based or optimization-based control strategies for thermal management. These strategies are often pre-set based on known operating conditions, resulting in poor adaptability to real-time conditions and an inability to meet real-time thermal management control requirements. Therefore, a Q-learning-based thermal management control method for pure electric vehicles is needed to address these issues. Summary of the Invention

[0004] In view of this, the purpose of this invention is to overcome the deficiencies in the prior art and provide a Q-learning-based thermal management control method for pure electric vehicles, which can adjust the thermal management control in real time during practical applications and keep the temperature of the integrated thermal management system within the optimal operating temperature range.

[0005] The Q-learning-based thermal management control method for pure electric vehicles of the present invention includes the following steps:

[0006] S1. Collect current vehicle status information, including vehicle speed, battery temperature, battery SOC, motor temperature, and passenger compartment temperature;

[0007] S2. Determine the required power of the vehicle;

[0008] S3. Based on the vehicle's current state information and the vehicle's required power, the Q-learning algorithm is used to solve for the optimal battery heating power and the optimal passenger compartment heating power;

[0009] S4. Heat the battery and the passenger compartment using the optimal battery heating power and the optimal passenger compartment heating power, respectively.

[0010] Furthermore, the vehicle's required power P is calculated based on the balance equation between driving force and driving resistance. req :

[0011]

[0012] Among them, P f Power consumed by driving resistance; P i Power consumed by ramp resistance; P w Power consumed by air resistance; P j Power consumed to accelerate resistance; u is vehicle speed; G is vehicle weight; f is rolling resistance coefficient; α is road gradient; C d δ is the air resistance coefficient; A is the frontal area; δ is the rotational mass conversion factor; m is the vehicle mass; t is time.

[0013] Furthermore, step S3 specifically includes:

[0014] S31. Initialize the state-action value function Q(s,a) by assigning all elements in Q(s,a) the value 0;

[0015] S32. Given the exploration rate ε, learning rate α, and discount factor γ, and set the number of algorithm iterations N;

[0016] S33. Select the required power P req Battery SOC, Battery Temperature T bat Crew cabin temperature T cbn As a state variable, the battery heating power P is selected. bat And crew cabin heating power P cbn As control variables, design the objective function and constraints;

[0017] S34. Given an initial state s, according to ε - The greedy strategy selects action a for state s:

[0018]

[0019] Where A(s) represents the action in the corresponding state s;

[0020] S35. According to ε - The greedy strategy selects action a in state s and obtains reward r and the next state s'.

[0021] S36. Update the iterative state-action value function Q(s,a):

[0022] Q(s,a)←Q(s,a)+α[r(s,a)+γminQ(s',a')-Q(s,a)];

[0023] Where r is the immediate reward function of the state-action sequence; a' is the action of the next state s';

[0024] S37. If the updated state-action value function satisfies the set error threshold, then the state-action value function converges. Using the converged state-action value function Q(s,a), the action corresponding to the minimum Q(s,a) value is selected as the optimal strategy for each state, and the battery heating power and passenger cabin heating power corresponding to the optimal strategy are taken as the optimal battery heating power and the optimal passenger cabin heating power.

[0025] Furthermore, the objective function is designed, specifically including:

[0026] Considering overall vehicle economy, power, and passenger cabin comfort, an objective function J is established:

[0027]

[0028] Among them, Q * (s,a) is the optimal state-action value function; Represents the cumulative expectation throughout the entire solution process; γ is the discount factor; r t+1 To solve for the state-action immediate reward function at stage t+1;

[0029] The function r is determined according to the following formula. t+1 :

[0030]

[0031] Wherein, J1(x(k),u(k))' is the normalized function of the vehicle economy evaluation function J1(x(k),u(k)) at time k; J2(x(k),u(k))' is the normalized function of the vehicle dynamics evaluation function J2(x(k),u(k)) at time k; J3(x(k),u(k))' is the normalized function of the passenger cabin comfort evaluation function J3(x(k),u(k)) at time k; k takes values ​​of 1, 2, ..., K, where K is the total time length; w1, w2, and w3 are weighting coefficients.

[0032] Furthermore, the constraints are as follows:

[0033]

[0034] Among them, T bat (k) represents the battery temperature at time k, T bat_min T bat_max These represent the upper and lower limits of the battery temperature, respectively; SOC(k) is the remaining battery charge at time k. min SOC maxThese represent the upper and lower limits of the remaining battery capacity, respectively; T cab (k) represents the cabin temperature at time k, T cab_min T cab_max These represent the upper and lower limits of the crew cabin temperature, respectively; P bat (k) represents the battery heating power at time k, P bat_min P bat_max These represent the upper and lower limits of the battery heating power, respectively; P cab (k) represents the crew cabin heating power at time k, P cab_min P cab_max These are the upper and lower limits of the crew cabin heating power, respectively.

[0035] Furthermore, the vehicle economy evaluation function J1(x(k),u(k)) at time k is determined according to the following formula:

[0036] J1(x(k),u(k))=SOC(k+1)-SOC(k);

[0037] Where SOC(k+1) is the remaining battery charge at time k+1; SOC(k) is the remaining battery charge at time k.

[0038] The vehicle dynamics evaluation function J2(x(k),u(k)) at time k is determined according to the following formula:

[0039]

[0040] Among them, P batpwr (k) represents the maximum discharge power of the battery at time k, and t and t+1 represent the current time and the next time, respectively.

[0041] The passenger cabin comfort evaluation function J3(x(k),u(k)) at time k is determined according to the following formula:

[0042]

[0043] Among them, T tar For the target temperature of the crew cabin, T cab (k) represents the temperature of the crew cabin at time k.

[0044] The beneficial effects of this invention are as follows: The Q-learning-based thermal management control method for pure electric vehicles disclosed in this invention obtains the current state information of the vehicle and the power demand of the vehicle, and uses the Q-learning algorithm to solve for the optimal battery heating power and the optimal passenger compartment heating power, thereby controlling the temperature of the integrated thermal management system within the optimal operating temperature range, which to a certain extent effectively improves the adaptability of the thermal management strategy to unknown operating conditions. Attached Figure Description

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0046] Figure 1 This is a schematic diagram of the thermal management control method of the present invention;

[0047] Figure 2 This is a schematic diagram of reinforcement learning according to the present invention;

[0048] Figure 3 This is the power demand probability transition matrix diagram of the present invention; where (a) is the WLTC operating condition and (b) is the NEDC operating condition.

[0049] Figure 4 This is a schematic diagram of the optimal heating power solution process based on the Q-learning algorithm of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings, as shown in the figures:

[0051] The Q-learning-based thermal management control method for pure electric vehicles of the present invention includes the following steps:

[0052] S1. Collect current vehicle status information, including vehicle speed, battery temperature, battery SOC, motor temperature, and passenger compartment temperature;

[0053] S2. Determine the required power of the vehicle;

[0054] S3. Based on the vehicle's current state information and the vehicle's required power, the Q-learning algorithm is used to solve for the optimal battery heating power and the optimal passenger compartment heating power;

[0055] S4. Heat the battery and the passenger compartment using the optimal battery heating power and the optimal passenger compartment heating power, respectively.

[0056] In this embodiment, in step S2, the vehicle power demand P is calculated based on the balance equation of driving force and driving resistance. req :

[0057]

[0058] Among them, P f Power consumed by driving resistance; P i Power consumed by ramp resistance; P w Power consumed by air resistance; P j Power consumed to accelerate resistance; u is vehicle speed; G is vehicle weight; f is rolling resistance coefficient; α is road gradient; C dδ is the air resistance coefficient; A is the frontal area; δ is the rotational mass conversion factor; m is the vehicle mass; t is time.

[0059] Of course, to adapt to future operating conditions, the vehicle's power demand under the next operating condition can be calculated based on the current power demand. Using the maximum likelihood estimation method and the nearest neighbor method, a power demand probability transition matrix is ​​constructed, thus obtaining the vehicle's power demand under different operating conditions, providing data support for subsequent thermal management control. For example, Figure 3 As shown, the demand power probability transition matrix diagrams for the WLTC and NEDC cycles are displayed respectively. Both the WLTC (Worldwideharmonized Light vehicles Test Cycle) and NEDC (New European Driving Cycle) cycles are standardized driving cycles used for vehicle fuel consumption and emissions testing.

[0060] In this embodiment, as Figure 2 and Figure 4 As shown, step S3 specifically includes:

[0061] S31. Initialize the state-action value function Q(s,a) by assigning all elements in Q(s,a) the value 0;

[0062] S32. Given the exploration rate ε, learning rate α, and discount factor γ, and set the number of algorithm iterations N;

[0063] S33. Select the required power P req Battery SOC, Battery Temperature T bat Crew cabin temperature T cbn As a state variable, the battery heating power P is selected. bat And crew cabin heating power P cbn As control variables, design the objective function and constraints;

[0064] S34. Given an initial state s, according to ε - The greedy strategy selects action a for state s:

[0065]

[0066] Where A(s) represents the action in the corresponding state s;

[0067] S35. According to ε - The greedy strategy selects action a in state s and obtains reward r and the next state s'.

[0068] S36. Update the iterative state-action value function Q(s,a):

[0069] Q(s,a)←Q(s,a)+α[r(s,a)+γminQ(s',a')-Q(s,a)];

[0070] Where r is the immediate reward function of the state-action sequence; a' is the action of the next state s';

[0071] S37. If the updated state-action value function satisfies the set error threshold, then the state-action value function converges. Using the converged state-action value function Q(s,a), the action corresponding to the minimum Q(s,a) value is selected as the optimal strategy for each state, and the battery heating power and passenger compartment heating power corresponding to the optimal strategy are taken as the optimal battery heating power and optimal passenger compartment heating power. The error threshold can be set according to the actual operating conditions, for example, a value of 0.01.

[0072] By using the above-mentioned Q-learning algorithm-based solution method, the control strategy is continuously updated iteratively, which has outstanding advantages in handling highly complex calculations and analyses, thereby obtaining the optimal battery heating power and the optimal crew cabin heating power.

[0073] In this embodiment, the objective function is designed, specifically including:

[0074] Under the constraints of the passenger compartment and power battery, an objective function J is established considering the vehicle's economy, power, and passenger compartment comfort:

[0075]

[0076] Among them, Q * (s,a) is the optimal state-action value function; Represents the cumulative expectation throughout the entire solution process; γ is the discount factor; r t+1 To solve for the state-action immediate reward function at stage t+1;

[0077] The function r is determined according to the following formula. t+1 :

[0078]

[0079] Wherein, J1(x(k),u(k))' is the normalized function of the vehicle economy evaluation function J1(x(k),u(k)) at time k; J2(x(k),u(k))' is the normalized function of the vehicle dynamics evaluation function J2(x(k),u(k)) at time k; J3(x(k),u(k))' is the normalized function of the passenger cabin comfort evaluation function J3(x(k),u(k)) at time k; k takes values ​​of 1, 2, ..., K, where K is the total time length; w1, w2, and w3 are weighting coefficients, each with a value of 1 / 3.

[0080] In this embodiment, the constraint condition is:

[0081]

[0082] Among them, T bat (k) represents the battery temperature at time k, T bat_min T bat_max These represent the upper and lower limits of the battery temperature, respectively; SOC(k) is the remaining battery charge at time k. min SOC max These represent the upper and lower limits of the remaining battery capacity, respectively; T cab (k) represents the cabin temperature at time k, T cab_min T cab_max These represent the upper and lower limits of the crew cabin temperature, respectively; P bat (k) represents the battery heating power at time k, P bat_min P bat_max These represent the upper and lower limits of the battery heating power, respectively; P cab (k) represents the crew cabin heating power at time k, P cab_min P cab_max These are the upper and lower limits of the crew cabin heating power, respectively.

[0083] In this embodiment, the vehicle economic evaluation function J1(x(k),u(k)) at time k is determined according to the following formula:

[0084] J1(x(k),u(k))=SOC(k+1)-SOC(k);

[0085] Where SOC(k+1) is the remaining battery charge at time k+1; SOC(k) is the remaining battery charge at time k.

[0086] The vehicle dynamics evaluation function J2(x(k),u(k)) at time k is determined according to the following formula:

[0087]

[0088] Among them, P batpwr(k) represents the maximum discharge power of the battery at time k, and t and t+1 represent the current time and the next time, respectively.

[0089] The passenger cabin comfort evaluation function J3(x(k),u(k)) at time k is determined according to the following formula:

[0090]

[0091] Among them, T tar For the target temperature of the crew cabin, T cab (k) represents the temperature of the crew cabin at time k.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A Q-learning based thermal management control method for a pure electric vehicle, characterized in that: Comprising the following steps: S1. Collecting vehicle current state information, the state information including vehicle speed, battery temperature, battery SOC, motor temperature and passenger cabin temperature; S2. Determining vehicle demand power; S3. Based on the vehicle current state information and the vehicle demand power, using the Q-learning algorithm to solve the optimal battery heating power and the optimal passenger cabin heating power; The step S3 specifically comprises: S31. Initializing the state-action value function Q(s, a), assigning all elements in Q(s, a) to 0; S32. Given the exploration rate ε, the learning rate α and the discount factor γ, and setting the algorithm iteration number N; S33. Select the demand power P req , battery SOC, battery temperature T bat , passenger cabin temperature T cbn As state variables, select the battery heating power P bat and the passenger cabin heating power P cbn as control variables, and design the objective function and the constraint conditions; S34. Given an initial state s, select an action a for state s according to ε - greedy policy selects an action a for state s: Wherein, A(s) is the action corresponding to the state s; S35. According to ε - greedy policy selects an action a in state s and obtains a reward r and next state s'. S36. Updating the iteration state-action value function Q(s, a): Q(s, a)←Q(s, a)+α[r(s, a)+γminQ(s', a')-Q(s, a)]; Wherein, r is the immediate reward function of state-action; a' is the action of the next state s'; S37. If the updated iteration state-action value function meets the set error threshold, the state-action value function converges at this time, and the optimal strategy of each state is selected by using the converged state-action value function Q(s, a), and the battery heating power and the passenger cabin heating power corresponding to the optimal strategy are taken as the optimal battery heating power and the optimal passenger cabin heating power; Designing an objective function, specifically comprising: Considering the vehicle economy, power performance and passenger cabin comfort, an objective function J is established: where Q * (s, a) is the optimal state-action value function; represents the cumulative expected reward over the entire solution process; γ is the discount factor; r t+1 is the state-action immediate reward function at solution stage t + 1. The function r is determined according to the formula t+1 : Wherein, J1(x(k), u(k))' is the normalized function of the kth time vehicle economy evaluation function J1(x(k), u(k)); J2(x(k), u(k))' is the normalized function of the kth time vehicle power performance evaluation function J2(x(k), u(k)); J3(x(k), u(k))' is the normalized function of the kth time passenger cabin comfort evaluation function J3(x(k), u(k)); k takes the value of 1, 2, …, K, K is the total time length; w1, w2, w3 are weight coefficients; S4. Heating the battery and the passenger cabin by the optimal battery heating power and the optimal passenger cabin heating power respectively.

2. The Q-learning based pure electric vehicle thermal management control method of claim 1, wherein: According to the driving force and the running resistance balance equation, the vehicle demand power P is calculated req : Among them, P f Power consumed by driving resistance; P i Power consumed by ramp resistance; P w Power consumed by air resistance; P j Power consumed to accelerate resistance; u is vehicle speed; G is vehicle weight; f is rolling resistance coefficient; α is road gradient; C d δ is the air resistance coefficient; A is the frontal area; δ is the rotational mass conversion factor; m is the vehicle mass; t is time.

3. The Q-learning based pure electric vehicle thermal management control method of claim 1, wherein: The constraint condition is: wherein T bat (k) is the battery temperature at time k, T bat_min , T bat_max are the upper and lower limits of the battery temperature, respectively; SOC(k) is the battery state of charge at time k, SOC min , SOC max are the upper and lower limits of the battery state of charge, respectively; T cab (k) is the passenger cabin temperature at time k, T cab_min , T cab_max are the upper and lower limits of the passenger cabin temperature, respectively; P bat (k) is the battery heating power at time k, P bat_min , P bat_max are the upper and lower limits of the battery heating power, respectively; P cab (k) is the passenger cabin heating power at time k, P cab_min , P cab_max are the upper and lower limits of the passenger cabin heating power, respectively.

4. The Q-learning based pure electric vehicle thermal management control method of claim 1, wherein: The kth time vehicle economy evaluation function J1(x(k), u(k)) is determined according to the following formula: J1(x(k), u(k))=SOC(k+1)-SOC(k); Wherein, SOC(k+1) is the battery remaining capacity at the k+1th time; SOC(k) is the battery remaining capacity at the kth time; The kth time vehicle power performance evaluation function J2(x(k), u(k)) is determined according to the following formula: where P batpwr (k) is the maximum discharging power of the battery at the kth moment, t and t+1 are the current moment and the next moment, respectively. The kth time passenger cabin comfort evaluation function J3(x(k), u(k)) is determined according to the following formula: where T tar is the target temperature of the passenger compartment, T cab (k) is the temperature of the passenger compartment at the kth time.

Images