Driving strategy determination method, device and system and storage medium

Abstract

The invention discloses a driving strategy determination method, device and system and a storage medium. The method comprises the following steps: acquiring a historical speed curve of a vehicle in a target route; importing the speed curve into the intelligent agent; calculating a total reward value corresponding to the historical speed curve through the intelligent agent; when the total reward value corresponding to the historical speed curve of the vehicle is smaller than a preset reward value, circularly updating the driving strategy in the intelligent agent, and determining the updated driving strategy and the calculated current Q value; and when the difference value between the Q value calculated by the updated driving strategy and the maximum Q value is smaller than a preset difference value, it is determined that intelligent agent training is completed so as to determine the driving strategy of the vehicle. By the adoption of the scheme, power can be distributed to the engine and the motor more reasonably, and the utilization rate of vehicle energy is increased.

Description

Technical Field

[0001] The present application relates to the technical field of driving strategy determination, and in particular to a driving strategy determination method, device, system and storage medium. Background Art

[0002] The product of the engine speed and torque is the engine power, and the product of the motor speed and torque is the motor power. When determining the driving strategy of existing hybrid vehicles, the power allocation to the engine and motor is based on the principle of minimum overall loss. When calculating the overall loss, only the engine fuel consumption and the motor power consumption are considered, and other factors are not considered. This loss calculation method is not comprehensive, and the engine and motor power allocation strategy is not reasonable. Therefore, how to provide a driving strategy determination method to more reasonably allocate power to the engine and motor. Summary of the invention

[0003] The present application provides a driving strategy determination method, device, system and storage medium for more reasonably allocating power to an engine and an electric motor.

[0004] The present application provides a driving strategy determination method, comprising:

[0005] Obtain the historical speed curve of the vehicle on the target route;

[0006] Importing the historical speed curve into an intelligent agent, and calculating the total reward value corresponding to the historical speed curve through the intelligent agent, wherein the intelligent agent is used to generate a corresponding driving strategy according to the imported historical speed curve, and the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy and vehicle state in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the total reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the total reward value is negatively correlated with the overall loss of the vehicle during driving;

[0007] When the total reward value corresponding to the historical speed curve of the vehicle is less than the preset reward value, the driving strategy in the agent is cyclically updated, and a current Q value calculated based on the updated driving strategy and the vehicle state is determined, and the current Q value is positively correlated with the total reward value;

[0008] When the difference between the Q value calculated based on the updated driving strategy and the vehicle state and the maximum Q value is less than a preset difference, it is determined that the agent training is completed to determine the driving strategy of the vehicle.

[0009] The beneficial effects of the present application are as follows: when calculating the overall loss of the vehicle, the battery loss of the vehicle is included, so that when determining the overall loss of the vehicle, the calculation result is more accurate and reasonable, and when the difference between the Q value calculated by the updated driving strategy and the maximum Q value is less than the preset difference, it is determined that the training of the intelligent agent is completed to determine the driving strategy of the vehicle. The vehicle uses the driving strategy contained in the intelligent agent to drive the vehicle so that the overall consumption during the driving process is close to the minimum consumption, thereby realizing a more reasonable torque distribution strategy, that is, it can more reasonably distribute power to the engine and the motor, thereby improving the utilization rate of the vehicle's energy.

[0010] In one embodiment, the calculating, by the agent, the total reward value corresponding to the historical speed curve includes:

[0011] Acquiring, by the agent, preset parameter values ​​of each cycle when the vehicle is driven according to the historical speed curve, wherein the preset parameter values ​​include vehicle battery loss;

[0012] Import the preset parameter values ​​of each cycle of the vehicle into the preset calculation model to obtain the reward value of each cycle;

[0013] The reward values ​​of each cycle are accumulated to obtain the total reward value corresponding to the historical speed curve.

[0014] In one embodiment, the vehicle battery loss is obtained by:

[0015] Obtain the cycle life and actual life of the battery under nominal conditions;

[0016] Substitute the cycle life and actual life of the battery under the nominal conditions into the following first preset formula for calculating the severity factor to obtain the severity factor corresponding to the battery life:

[0017]

[0018] Among them, σ(I,T b , SOC) is the severity factor; Γ nom is the total ampere-hour throughput of the nominal battery life; Γ represents the actual ampere-hour throughput of the battery; I nom (t) and I(t) are the nominal condition and actual current value respectively;

[0019] Substituting the preset battery parameters including the severity factor into the following second preset formula for calculating battery life loss, the battery loss corresponding to the historical speed curve is obtained:

[0020]

[0021] Among them, C bis the battery loss; c bat is the loss factor of the replacement battery; σ is the severity factor; I bat (t) is the corresponding power of the battery, Γ represents the actual ampere-hour flow of the battery, It is the battery life consumption ratio of this vehicle driving.

[0022] In one embodiment, the step of importing the preset parameter values ​​of the vehicle into the preset calculation model to obtain a result for characterizing the overall loss of the vehicle includes:

[0023] The vehicle's power loss, fuel loss, battery loss, vehicle start-stop penalty value, and physical constraint penalty value are introduced into the following preset calculation model to calculate a reward value representing the vehicle's overall loss, and the reward value representing the vehicle's overall loss is used as the calculation result of the preset calculation model, wherein the vehicle start-stop penalty value is determined according to the number of vehicle starts and stops, and the greater the number of starts and stops, the greater the vehicle start-stop penalty value, and the physical constraint penalty value is determined according to whether the current battery power is in a preset power range and whether the current torque of the vehicle is in a preset torque range:

[0024] r(t)=-(C f +C e +C b +C Penalty,1 +C Penalty,2 );

[0025] Where r(t) is the reward value representing the overall loss of the vehicle; C f is the fuel consumption; C e is the power loss; C b is the battery consumption; C Penalty,1 is the vehicle start-stop penalty value; C Penalty,2 is the physical constraint penalty value.

[0026] In one embodiment, the fuel consumption is determined as follows:

[0027] Obtaining multiple sets of discrete values ​​corresponding to engine fuel consumption rate, fuel loss per unit time, engine speed and torque, and fuel density;

[0028] Substitute the engine fuel consumption rate, fuel loss per unit time, engine speed, engine torque and fuel density into the following third preset formula to determine the fuel loss:

[0029]

[0030] Among them, C f is the fuel consumption; c f is the fuel consumption per unit time; b e is the engine fuel consumption rate; Tor eng(t) is the engine torque, ω eng (t) is the engine speed; ρ is the fuel density.

[0031] In one embodiment, the physical constraint penalty value is determined as follows:

[0032] Determine whether the current battery power is within a preset power range, and determine whether the current vehicle torque is within a preset torque range;

[0033] When the current battery power is within a preset power range and the current vehicle torque is within a preset torque range, determining that the physical constraint penalty value is 0;

[0034] When the current battery power exceeds a preset power range or the current torque of the vehicle exceeds a preset torque range, the physical constraint penalty value is determined to be a specific value, wherein the specific value makes the reward value of the overall loss of the vehicle less than a preset value.

[0035] In one embodiment, the loop updates the current driving strategy, including:

[0036] Randomly select multiple four-tuple data from the experience pool corresponding to the agent as learning samples, where the four-tuple data includes the state at the historical moment, the action taken by the agent at the historical moment, the reward at the historical moment, and the state at the next historical moment;

[0037] The intelligent agent is cyclically trained by the learning samples, and weight parameters are updated by the stochastic gradient descent method;

[0038] The updated weight parameters are assigned to the Q network to achieve a cyclic update of the driving strategy in the intelligent agent.

[0039] The present application also provides a driving strategy determination device, comprising:

[0040] An acquisition module is used to obtain a historical speed curve of the vehicle on a target route;

[0041] An import module, used to import the historical speed curve into an intelligent agent, and calculate the total reward value corresponding to the historical speed curve through the intelligent agent, wherein the intelligent agent is used to generate a corresponding driving strategy according to the imported historical speed curve, and the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy and vehicle status in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the total reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the overall reward value is negatively correlated with the overall loss of the vehicle during driving;

[0042] A first determination module is used for cyclically updating the driving strategy in the intelligent agent when the total reward value corresponding to the historical speed curve of the vehicle is less than a preset reward value, and determining a current Q value calculated based on the updated driving strategy and the vehicle state, wherein the current Q value is positively correlated with the total reward value;

[0043] The second determination module is used to determine that the agent training is completed when the difference between the Q value calculated based on the updated driving strategy and the vehicle state and the maximum Q value is less than a preset difference, so as to determine the driving strategy of the vehicle.

[0044] In one embodiment, the import module includes:

[0045] An acquisition submodule, used for acquiring, through the intelligent agent, preset parameter values ​​of each cycle when the vehicle drives according to the historical speed curve, wherein the preset parameter values ​​include vehicle battery loss;

[0046] The import submodule is used to import the preset parameter values ​​of each cycle of the vehicle into the preset calculation model to obtain the reward value of each cycle;

[0047] The accumulation submodule is used to accumulate the reward values ​​of each cycle to obtain the total reward value corresponding to the historical speed curve.

[0048] In one embodiment, the vehicle battery loss is obtained by:

[0049] Obtain the cycle life and actual life of the battery under nominal conditions;

[0050] Substitute the cycle life of the battery under the nominal conditions and the actual battery life into the following first preset formula for calculating the severity factor to obtain the severity factor corresponding to the battery life:

[0051]

[0052] Among them, σ(I,T b , SOC) is the severity factor; Γ nom is the total ampere-hour throughput of the nominal battery life; Γ represents the actual ampere-hour throughput of the battery; I nom (t) and I(t) are the nominal condition and actual current value respectively;

[0053] Substituting the preset battery parameters including the severity factor into the following second preset formula for calculating battery life loss, the battery loss corresponding to the historical speed curve is obtained:

[0054]

[0055] Among them, C b is the battery loss; cbat is the loss factor of the replacement battery; σ is the severity factor; I bat (t) is the corresponding power of the battery, Г represents the actual ampere-hour flow of the battery, It is the battery life consumption ratio of this vehicle driving.

[0056] In one embodiment, the import submodule includes:

[0057] The vehicle's power loss, fuel loss, battery loss, vehicle start-stop penalty value, and physical constraint penalty value are introduced into the following preset calculation model to calculate a reward value representing the vehicle's overall loss, and the reward value representing the vehicle's overall loss is used as the calculation result of the preset calculation model, wherein the vehicle start-stop penalty value is determined according to the number of vehicle starts and stops, and the greater the number of starts and stops, the greater the vehicle start-stop penalty value, and the physical constraint penalty value is determined according to whether the current battery power is in a preset power range and whether the current torque of the vehicle is in a preset torque range:

[0058] r(t)=-(C f +C e +C b +C Penalty,1 +C Penalty,2 );

[0059] Where r(t) is the reward value representing the overall loss of the vehicle; C f is the fuel consumption; C e is the power loss; C b is the battery consumption; C Penalty,1 is the vehicle start-stop penalty value; C Penalty,2 is the physical constraint penalty value.

[0060] In one embodiment, the fuel consumption is determined as follows:

[0061] Obtaining multiple sets of discrete values ​​corresponding to engine fuel consumption rate, fuel loss per unit time, engine speed and torque, and fuel density;

[0062] Substitute the engine fuel consumption rate, fuel loss per unit time, engine speed, engine torque and fuel density into the following third preset formula to determine the fuel loss:

[0063]

[0064] Among them, C f is the fuel consumption; c f is the fuel consumption per unit time; b e is the engine fuel consumption rate; Tor eng (t) is the engine torque, ω eng(t) is the engine speed; ρ is the fuel density.

[0065] In one embodiment, the physical constraint penalty value is determined as follows:

[0066] Determine whether the current battery power is within a preset power range, and determine whether the current vehicle torque is within a preset torque range;

[0067] When the current battery power is within a preset power range and the current vehicle torque is within a preset torque range, determining that the physical constraint penalty value is 0;

[0068] When the current battery power exceeds a preset power range or the current torque of the vehicle exceeds a preset torque range, the physical constraint penalty value is determined to be a specific value, wherein the specific value makes the reward value of the overall loss of the vehicle less than a preset value.

[0069] In one embodiment, the loop updates the current driving strategy, including:

[0070] Randomly select multiple four-tuple data from the experience pool corresponding to the agent as learning samples, where the four-tuple data includes the state at the historical moment, the action taken by the agent at the historical moment, the reward at the historical moment, and the state at the next historical moment;

[0071] The intelligent agent is cyclically trained by the learning samples, and weight parameters are updated by the stochastic gradient descent method;

[0072] The updated weight parameters are assigned to the Q network to achieve a cyclic update of the driving strategy in the intelligent agent.

[0073] The present application also provides a driving strategy determination system, comprising:

[0074] at least one processor; and,

[0075] a memory communicatively connected to the at least one processor; wherein,

[0076] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the driving strategy determination method recorded in any of the above embodiments.

[0077] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the driving strategy determination system, the driving strategy determination system can implement the driving strategy determination method recorded in any of the above embodiments.

[0078] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0079] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings:

[0081] Figure 1 This is a flow chart of a driving strategy determination method in one embodiment of the present application;

[0082] Figure 2 This is a schematic diagram of the structure of a driving strategy determination device in one embodiment of the present application;

[0083] Figure 3 This is a schematic diagram of the hardware structure of a driving strategy determination system in one embodiment of the present application. DETAILED DESCRIPTION

[0084] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application.

[0085] Figure 1 Flow chart of a driving strategy determination method in one embodiment of the present application. Figure 1 As shown, the method can be implemented as the following steps S101-S104:

[0086] In step S101, a historical speed curve of the vehicle on the target route is obtained;

[0087] In step S102, the historical speed curve is imported into an intelligent agent, and the overall reward value corresponding to the historical speed curve is calculated by the intelligent agent, wherein the intelligent agent is used to generate a corresponding driving strategy according to the imported historical speed curve, and the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy and vehicle state in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the overall reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the overall reward value is negatively correlated with the overall loss of the vehicle during driving;

[0088] In step S103, when the total reward value corresponding to the historical speed curve of the vehicle is less than the preset reward value, the driving strategy in the agent is cyclically updated, and the current Q value calculated based on the updated driving strategy is determined, and the current Q value is positively correlated with the total reward value;

[0089] In step S104, when the difference between the Q value calculated by the updated driving strategy and the maximum Q value is less than a preset difference, it is determined that the agent training is completed to determine the driving strategy of the vehicle.

[0090] In the present application, the historical speed curve of the vehicle on the target route is obtained; the historical speed curve is imported into an intelligent agent, wherein the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the maximum Q value is usually the maximum Q value of the next state, that is, the expected Q value. The overall reward value corresponding to the historical speed curve is calculated by the intelligent agent, wherein the overall reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the overall reward value is negatively correlated with the overall loss of the vehicle during driving.

[0091] When the total reward value corresponding to the historical speed curve of the vehicle is less than the preset reward value, the current driving strategy is updated cyclically, and the current Q value calculated by the updated driving strategy is determined, that is, the agent can select the action with the highest Q value according to the current state, and update the Q value table by observing the reward value. The current Q value represents the expected return that can be obtained by taking the corresponding torque under a specific state. It can be expressed as Q(s,a), where s represents the vehicle state and a represents the action. The action value function tells the agent the value of taking different actions under a certain state, thereby helping the agent make the best decision. Specifically, a plurality of four-tuple data are randomly selected from the experience pool corresponding to the agent as learning samples, wherein the four-tuple data includes the state at the historical moment, the action taken by the agent at the historical moment, the reward at the historical moment, and the state at the next historical moment; the agent is cyclically trained through the learning sample, and the weight parameters are updated by the stochastic gradient descent method; the updated weight parameters are assigned to the Q network to realize the cyclic update of the driving strategy in the agent. For example, the data stored in the experience pool is (S t ,A t ,r(t),S t+1 ) 4-tuple, where S t represents the state at time t, A t represents the action taken by the agent at time t, r(t) represents the reward at time t, S t+1 Represents the state at time t+1, that is, taking A tThe state at the next moment after the action. The vehicle state includes the battery SOC, vehicle speed, vehicle acceleration, battery temperature and remaining mileage ratio of the vehicle.

[0092] When the difference between the Q value calculated by the updated driving strategy and the maximum Q value is less than the preset difference, it is determined that the agent training is completed to determine the driving strategy of the vehicle.

[0093] It should be noted that in the present application, after the training of the intelligent agent is completed, the trained intelligent agent can be sent to the vehicle that needs to travel on the target route, so that the vehicle traveling on the target route can be driven according to the driving strategy in the intelligent agent, wherein the driving strategy at least includes the torque distribution strategy of the vehicle during driving.

[0094] In reinforcement learning, the goal is to find the optimal value function (optimal strategy) so that the agent can obtain the maximum cumulative reward in the process of interacting with the environment. By continuously updating the action value function, the optimal action value function is finally approached, and the optimal value function is found. It can be understood that when the optimal function is found, the driving strategy in the agent is also the optimal driving strategy that can minimize the loss.

[0095] This application takes battery loss into consideration when calculating losses. Therefore, when formulating driving strategies, it is based on the goal of minimizing the total loss of fuel consumption, electricity consumption, and battery loss.

[0096] The beneficial effect of the present application is that the battery loss of the vehicle is included in the calculation of the overall loss of the vehicle, so that when determining the overall loss of the vehicle, the calculation result is more accurate and reasonable, and when the difference between the Q value calculated by the updated driving strategy and the maximum Q value is less than the preset difference, it is determined that the training of the intelligent agent is completed, and the vehicle is driven using the driving strategy contained in the intelligent agent, which can make the overall consumption of the vehicle close to the minimum consumption during driving, thereby realizing a more reasonable torque distribution strategy, that is, it can more reasonably distribute power to the engine and the motor, thereby improving the utilization rate of the vehicle's energy.

[0097] In one embodiment, the step S103 calculates the total reward value corresponding to the historical speed curve by the agent, including the following steps A1-A3:

[0098] In step A1, the intelligent agent obtains preset parameter values ​​of each cycle when the vehicle drives according to the historical speed curve, and the preset parameter values ​​include vehicle battery loss;

[0099] In step A2, the preset parameter values ​​of each cycle of the vehicle are imported into the preset calculation model to obtain the reward value of each cycle;

[0100] In step A3, the reward values ​​of each cycle are accumulated to obtain the total reward value corresponding to the historical speed curve.

[0101] The vehicle battery loss is obtained by:

[0102] Obtain the cycle life and actual life of the battery under nominal conditions;

[0103] Substitute the cycle life of the battery under the nominal conditions and the actual battery life into the following first preset formula for calculating the severity factor to obtain the severity factor corresponding to the battery life:

[0104]

[0105] Among them, σ(I,T b , SOC) is the severity factor; Γ nom is the total ampere-hour throughput of the nominal battery life; Γ represents the actual ampere-hour throughput of the battery; I nom (t) and I(t) are the nominal condition and actual current value respectively;

[0106] Substituting the preset battery parameters including the severity factor into the following second preset formula for calculating battery life loss, the battery loss corresponding to the historical speed curve is obtained:

[0107]

[0108] Among them, C b is the battery loss; c bat is the loss factor of the replacement battery; σ is the severity factor; I bat (t) is the corresponding power of the battery, Γ represents the actual ampere-hour flow of the battery, It is the battery life consumption ratio of this vehicle driving.

[0109] In one embodiment, the above step A2 can be implemented as the following steps:

[0110] The vehicle's power loss, fuel loss, battery loss, vehicle start-stop penalty value, and physical constraint penalty value are introduced into the following preset calculation model to calculate a reward value representing the vehicle's overall loss, and the reward value representing the vehicle's overall loss is used as the calculation result of the preset calculation model, wherein the vehicle start-stop penalty value is determined according to the number of vehicle starts and stops, and the greater the number of starts and stops, the greater the vehicle start-stop penalty value, and the physical constraint penalty value is determined according to whether the current battery power is in a preset power range and whether the current torque of the vehicle is in a preset torque range:

[0111] r(t)=-(C f +C e +C b+C Penalty,1 +C Penalty,2 );

[0112] Where r(t) is the reward value representing the overall loss of the vehicle; C f is the fuel consumption; C e is the power loss; C b is the battery consumption; C Penalty,1 is the vehicle start-stop penalty value; C Penalty,2 is the physical constraint penalty value.

[0113] The fuel consumption is determined as follows:

[0114] Obtaining multiple sets of discrete values ​​corresponding to engine fuel consumption rate, fuel loss per unit time, engine speed and torque, and fuel density;

[0115] Substitute the engine fuel consumption rate, fuel loss per unit time, engine speed, engine torque and fuel density into the following third preset formula to determine the fuel loss:

[0116]

[0117] Among them, C f is the fuel consumption; c f is the fuel consumption per unit time; b e is the engine fuel consumption rate; Tor eng (t) is the engine torque, ω eng (t) is the engine speed; ρ is the fuel density.

[0118] In this application, the preset parameter values ​​of the vehicle are obtained, wherein the preset parameter values ​​include parameters characterizing battery loss. In order to improve the energy utilization rate of the vehicle, when determining the overall loss of the vehicle, not only the consumption of electricity and oil is considered, but also the hidden cost of battery aging is considered.

[0119] (1) Determine battery loss: In reality, the working conditions of batteries are complex. In order to quantify the impact of complex working conditions on battery life, the concept of severity factor is introduced, which is defined as follows:

[0120]

[0121] Where, σ is the severity factor; Γ nom is the total ampere-hour throughput of the nominal battery life; Г represents the actual ampere-hour throughput of the battery.

[0122] Then, the severity factor, the battery life consumption ratio of this vehicle driving, and the loss coefficient of replacing the battery are substituted into a second preset formula for calculating the battery life loss to obtain the battery loss of this driving. Specifically, the second preset formula may be as follows:

[0123]

[0124] Among them, C b is the battery loss; c bat is the loss factor of the replacement battery; σ is the severity factor; I bat (t) is the corresponding power of the battery, Г represents the actual ampere-hour flow of the battery, It is the battery life consumption ratio of this vehicle driving.

[0125] In one embodiment of the present application, the total ampere-hour throughput of the battery is used as the battery life. In order to determine the ampere-hour throughput of the battery, a battery aging model is constructed. Since battery aging is related to temperature, current size, SOC (state of charge, which refers to the charging state of the car battery, also called the remaining capacity percentage, indicating the battery's ability to continue working), etc., a battery aging model can be constructed:

[0126] Q loss =f(T,I,SOC),

[0127] Among them, Q loss is the battery capacity loss value, T is the battery temperature, I is the current size, SOC is the remaining capacity of the battery, and f is the prediction function of the battery aging model, which can have many forms and is not limited in this application. In this embodiment, the specific form of the battery aging model is as follows:

[0128]

[0129] Among them, Q loss is the battery capacity loss value; SOC is the remaining capacity of the battery; E a I is the battery activation energy; c is the discharge rate; R gas is the gas constant; T b is the battery temperature; Ah is the ampere-hour throughput of the battery; α and β are fitting constants, η is the compensation coefficient, is the power law factor and can be determined in advance through experiments.

[0130] Battery activation energy refers to the energy barrier that needs to be overcome in a chemical reaction to start the reaction. It can be measured by the effect of temperature on the battery reaction rate. The gas constant commonly used in batteries is the ideal gas constant. It is generally believed that when the battery capacity Q loss When the loss reaches 20%, it means that the battery life has reached the end. It can be obtained that the nominal battery cycle life is:

[0131]

[0132] Among them, Γ nom is the total ampere-hour throughput of the nominal battery life, SOC nom E is the nominal battery remaining capacity; a I is the battery activation energy; c,nom is the discharge rate; R gas is the gas constant; is the battery temperature; α and β are fitting constants, η is the compensation coefficient, is the power law factor.

[0133] Corresponding to actual use, the actual battery life can be obtained according to:

[0134]

[0135] Where: Γ represents the actual life ampere-hour throughput; SOC is the remaining capacity of the battery; E a I is the battery activation energy; c is the discharge rate; R gas is the gas constant; T b is the battery temperature; Ah is the ampere-hour throughput of the battery; α and β are fitting constants, η is the compensation coefficient, is the power law factor and can be determined in advance through experiments.

[0136] (2) Determining fuel loss: obtaining a plurality of discrete values ​​corresponding to an engine fuel consumption rate, a fuel loss per unit time, an engine speed and a torque, and a fuel density; substituting the engine fuel consumption rate, the fuel loss per unit time, the engine speed, the engine torque and the fuel density into a third preset formula to determine the fuel loss.

[0137]

[0138] Among them, C f is the fuel consumption; c f is the fuel consumption per unit time; b e is the engine fuel consumption rate; Tor eng (t) is the engine torque, ω eng (t) is the engine speed; ρ is the fuel density.

[0139] (3) Determine the power loss: Obtain the power loss per unit time and battery power, and substitute them into the following formula to determine the corresponding power loss.

[0140]

[0141] Among them, CC e is the power loss; c e is the power consumption per unit time; P b (t) is the power of the battery.

[0142] Then, after determining the various losses of the vehicle, the preset parameters of the vehicle are imported into the preset calculation model, wherein the calculation result of the preset calculation model represents the overall loss of the vehicle, and the larger the calculation result, the smaller the overall loss of the vehicle. Specifically, the vehicle's power loss, fuel loss, battery loss, vehicle start-stop penalty value, and physical constraint penalty value are imported into the following preset calculation model to calculate the reward value representing the overall loss of the vehicle as the calculation result of the preset calculation model:

[0143] r(t)=-(C f +C e +C b +C Penalty,1 +C Penalty,2 );

[0144] Where r(t) is the reward value representing the overall loss of the vehicle; C f is the fuel consumption; C e is the power loss; C b is the battery consumption; C Penalty,1 It is the vehicle start-stop penalty value to avoid frequent engine start-stop. This value can be calibrated. Penalty,2 It is a physical constraint penalty value, which is determined based on whether the current battery power is in a preset power range and whether the current torque of the vehicle is in a preset torque range. When the current battery power is in the preset power range and the current torque of the vehicle is in the preset torque range, the physical constraint penalty value is determined to be 0; when the current battery power exceeds the preset power range or the current torque of the vehicle exceeds the preset torque range, the physical constraint penalty value is determined to be a specific value, wherein the specific value makes the reward value of the overall loss of the vehicle less than the preset value.

[0145] The operating parameter values ​​of the engine and the motor when the calculation result of the preset calculation model is maximum are obtained, wherein the operating parameter values ​​include at least torque and speed; in this embodiment, in order to reasonably distribute the operating power of the engine and the motor, the target parameter values ​​of the vehicle drive device corresponding to the throttle opening are obtained, wherein the target parameter values ​​are the target speed and the target torque. The vehicle drive device includes an engine and a motor, and the corresponding relationship between the throttle opening and the operating parameters of the engine and the motor can be obtained in advance through experiments. Therefore, during the operation of the vehicle, the target speed and target torque corresponding to the engine and the motor can be obtained respectively according to the current throttle opening.

[0146] Finally, the engine and the motor of the vehicle are controlled according to the operating parameter values ​​of the engine and the operating parameter values ​​of the motor when the overall loss of the vehicle is minimized.

[0147] Through this application, for daily repetitive routes with high speed, such as daily commuting, the routes are fixed every day, and the overlap of daily operating conditions (speed curves) is very high. The user clicks to confirm the collection of operating condition data in commuting mode through the interactive interface, and the equipment collects the car's driving speed, driving distance, etc.; when the trip is over, the user clicks to confirm that this section of the route is used as a commuting route, and sets the oil price and electricity price, and then uploads this section of data to the cloud. The cloud trains the model based on the data uploaded by the user and the set oil price and electricity price; after the training is completed, the trained intelligent agent is updated to the vehicle controller on the user's car. Through this method, the energy in the commuting route is distributed, and an approximate global optimum under the commuting route can be achieved, so that the driving cost is optimized and the energy utilization rate is improved. Give full play to the energy-saving advantages of hybrid vehicles on fixed driving routes; in addition, since the battery aging cost is taken into account, the battery aging rate is slowed down through reasonable battery and engine power distribution.

[0148] Figure 2 FIG. 1 is a schematic diagram of a driving strategy determination device in an embodiment of the present application. Figure 2 As shown, the device comprises:

[0149] An acquisition module 201 is used to acquire a historical speed curve of a vehicle on a target route;

[0150] An import module 202 is used to import the historical speed curve into an intelligent agent, and calculate the total reward value corresponding to the historical speed curve through the intelligent agent, wherein the intelligent agent is used to generate a corresponding driving strategy according to the imported historical speed curve, and the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy and vehicle state in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the total reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the overall reward value is negatively correlated with the overall loss of the vehicle during driving;

[0151] The first determination module 203 is used to cyclically update the driving strategy in the agent when the total reward value corresponding to the historical speed curve of the vehicle is less than the preset reward value, and determine the current Q value calculated based on the updated driving strategy and the vehicle state, wherein the current Q value is positively correlated with the total reward value;

[0152] The second determination module 204 is used to determine that the agent training is completed when the difference between the Q value calculated based on the updated driving strategy and the vehicle state and the maximum Q value is less than a preset difference, so as to determine the driving strategy of the vehicle.

[0153] In one embodiment, the import module includes:

[0154] An acquisition submodule, used for acquiring, through the intelligent agent, preset parameter values ​​of each cycle when the vehicle drives according to the historical speed curve, wherein the preset parameter values ​​include vehicle battery loss;

[0155] The import submodule is used to import the preset parameter values ​​of each cycle of the vehicle into the preset calculation model to obtain the reward value of each cycle;

[0156] The accumulation submodule is used to accumulate the reward values ​​of each cycle to obtain the total reward value corresponding to the historical speed curve.

[0157] In one embodiment, the vehicle battery loss is obtained by:

[0158] Obtain the cycle life and actual life of the battery under nominal conditions;

[0159] Substitute the cycle life of the battery under the nominal conditions and the actual battery life into the following first preset formula for calculating the severity factor to obtain the severity factor corresponding to the battery life:

[0160]

[0161] Among them, σ(I,T b , SOC) is the severity factor; Γ nom is the total ampere-hour throughput of the nominal battery life; Γ represents the actual ampere-hour throughput of the battery; I nom (t) and I(t) are the nominal condition and actual current value respectively;

[0162] Substituting the preset battery parameters including the severity factor into the following second preset formula for calculating battery life loss, the battery loss corresponding to the historical speed curve is obtained:

[0163]

[0164] Among them, C b is the battery loss; c bat is the loss factor of the replacement battery; σ is the severity factor; I bat (t) is the corresponding power of the battery, Γ represents the actual ampere-hour flow of the battery, It is the battery life consumption ratio of this vehicle driving.

[0165] In one embodiment, the import submodule includes:

[0166] The vehicle's power loss, fuel loss, battery loss, vehicle start-stop penalty value, and physical constraint penalty value are introduced into the following preset calculation model to calculate a reward value representing the vehicle's overall loss, and the reward value representing the vehicle's overall loss is used as the calculation result of the preset calculation model, wherein the vehicle start-stop penalty value is determined according to the number of vehicle starts and stops, and the greater the number of starts and stops, the greater the vehicle start-stop penalty value, and the physical constraint penalty value is determined according to whether the current battery power is in a preset power range and whether the current torque of the vehicle is in a preset torque range:

[0167] r(t)=-(C f +C e +C b +C Penalty,1 +C Penalty,2 );

[0168] Where r(t) is the reward value representing the overall loss of the vehicle; C f is the fuel consumption; C e is the power loss; C b is the battery consumption; C Penalty,1 is the vehicle start-stop penalty value; C Penalty,2 is the physical constraint penalty value.

[0169] In one embodiment, the fuel consumption is determined as follows:

[0170] Obtaining multiple sets of discrete values ​​corresponding to engine fuel consumption rate, fuel loss per unit time, engine speed and torque, and fuel density;

[0171] Substitute the engine fuel consumption rate, fuel loss per unit time, engine speed, engine torque and fuel density into the following third preset formula to determine the fuel loss:

[0172]

[0173] Among them, C f is the fuel consumption; c f is the fuel consumption per unit time; b e is the engine fuel consumption rate; Tor eng (t) is the engine torque, ω eng (t) is the engine speed; ρ is the fuel density.

[0174] In one embodiment, the physical constraint penalty value is determined as follows:

[0175] Determine whether the current battery power is within a preset power range, and determine whether the current vehicle torque is within a preset torque range;

[0176] When the current battery power is within a preset power range and the current vehicle torque is within a preset torque range, determining that the physical constraint penalty value is 0;

[0177] When the current battery power exceeds a preset power range or the current torque of the vehicle exceeds a preset torque range, the physical constraint penalty value is determined to be a specific value, wherein the specific value makes the reward value of the overall loss of the vehicle less than a preset value.

[0178] In one embodiment, the loop updates the current driving strategy, including:

[0179] Randomly select multiple four-tuple data from the experience pool corresponding to the agent as learning samples, where the four-tuple data includes the state at the historical moment, the action taken by the agent at the historical moment, the reward at the historical moment, and the state at the next historical moment;

[0180] The intelligent agent is cyclically trained by the learning samples, and weight parameters are updated by the stochastic gradient descent method;

[0181] The updated weight parameters are assigned to the Q network to achieve a cyclic update of the driving strategy in the intelligent agent.

[0182] Figure 3 FIG. 1 is a schematic diagram of the hardware structure of a driving strategy determination system in one embodiment of the present application. Figure 3 As shown, the driving strategy determination system includes:

[0183] at least one processor 320; and,

[0184] A memory 304 in communication with the at least one processor 320; wherein,

[0185] The memory 304 stores instructions that can be executed by the at least one processor 320, and the instructions are executed by the at least one processor 320 to implement the driving strategy determination method described in any of the above embodiments.

[0186] Reference Figure 3 The driving strategy determination system 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .

[0187] The processing component 302 generally controls the overall operation of the driving strategy determination system 300. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

[0188] The memory 304 is configured to store various types of data to support the operation of the driving strategy determination system 300. Examples of such data include instructions for any application or method operating on the driving strategy determination system 300, such as text, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0189] The power supply component 306 provides power to various components of the driving strategy determination system 300. The power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the driving strategy determination system 300.

[0190] The multimedia component 308 includes a screen that provides an output interface between the driving strategy determination system 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 may also include a front camera and / or a rear camera. When the driving strategy determination system 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0191] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the driving strategy determination system 300 is in an operation mode, such as an alarm mode, a recording mode, a voice recognition mode, and a voice output mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

[0192] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0193] The sensor assembly 314 includes one or more sensors for providing various aspects of status assessment for the driving strategy determination system 300. For example, the sensor assembly 314 may include a sound sensor. In addition, the sensor assembly 314 may detect the on / off state of the driving strategy determination system 300, the relative positioning of the components, such as the display and keypad of the driving strategy determination system 300, and the sensor assembly 314 may also detect the operating state of the driving strategy determination system 300 or a component of the driving strategy determination system 300, such as the operating state of the wind plate, the structural state, the operating state of the discharge scraper, etc., the orientation or acceleration / deceleration of the driving strategy determination system 300 and the temperature change of the driving strategy determination system 300. The sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, a material stack thickness sensor or a temperature sensor.

[0194] The communication component 316 is configured to enable the driving strategy determination system 300 to provide the ability to communicate with other devices and cloud platforms in a wired or wireless manner. The driving strategy determination system 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0195] In an exemplary embodiment, the driving strategy determination system 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the driving strategy determination method described in any of the above embodiments.

[0196] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the driving strategy determination system, the driving strategy determination system can implement the driving strategy determination method recorded in any of the above embodiments.

[0197] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0198] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0199] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0200] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0201] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A driving strategy determination method, characterized in that: include: Obtain the historical speed curve of the vehicle on the target route; Importing the historical speed curve into an intelligent agent, and calculating the total reward value corresponding to the historical speed curve through the intelligent agent, wherein the intelligent agent is used to generate a corresponding driving strategy according to the imported historical speed curve, and the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy and vehicle state in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the total reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the total reward value is negatively correlated with the overall loss of the vehicle during driving; When the total reward value corresponding to the historical speed curve of the vehicle is less than the preset reward value, the driving strategy in the agent is cyclically updated, and a current Q value calculated based on the updated driving strategy and the vehicle state is determined, and the current Q value is positively correlated with the total reward value; When the difference between the Q value calculated based on the updated driving strategy and the vehicle state and the maximum Q value is less than a preset difference, it is determined that the agent training is completed to determine the driving strategy of the vehicle.

2. The method according to claim 1, characterized in that Calculating the total reward value corresponding to the historical speed curve by the agent includes: Acquiring, by the agent, preset parameter values ​​of each cycle when the vehicle is driven according to the historical speed curve, wherein the preset parameter values ​​include vehicle battery loss; Import the preset parameter values ​​of each cycle of the vehicle into the preset calculation model to obtain the reward value of each cycle; The reward values ​​of each cycle are accumulated to obtain the total reward value corresponding to the historical speed curve.

3. The method according to claim 2, characterized in that The vehicle battery loss is obtained by: Obtain the cycle life and actual life of the battery under nominal conditions; Substitute the cycle life and actual life of the battery under the nominal conditions into the following first preset formula for calculating the severity factor to obtain the severity factor corresponding to the battery life: Among them, σ(I,T b , SOC) is the severity factor; Γ nom is the total ampere-hour throughput of the nominal battery life; Γ represents the actual ampere-hour throughput of the battery; I nom (t) and I(t) are the nominal condition and actual current value respectively; Substituting the preset battery parameters including the severity factor into the following second preset formula for calculating battery life loss, the battery loss corresponding to the historical speed curve is obtained: Among them, C b is the battery loss; c bat is the loss factor of the replacement battery; σ is the severity factor; I bat (t) is the corresponding power of the battery, Γ represents the actual ampere-hour flow of the battery, It is the battery life consumption ratio of this vehicle driving.

4. The method according to claim 2, characterized in that The method of importing the preset parameter values ​​of the vehicle into the preset calculation model to obtain a result for characterizing the overall loss of the vehicle includes: The vehicle's power loss, fuel loss, battery loss, vehicle start-stop penalty value, and physical constraint penalty value are introduced into the following preset calculation model to calculate a reward value representing the vehicle's overall loss, and the reward value representing the vehicle's overall loss is used as the calculation result of the preset calculation model, wherein the vehicle start-stop penalty value is determined according to the number of vehicle starts and stops, and the greater the number of starts and stops, the greater the vehicle start-stop penalty value, and the physical constraint penalty value is determined according to whether the current battery power is in a preset power range and whether the current torque of the vehicle is in a preset torque range: r(t)=-(C f +C e +C b +C Penalty,1 +C Penalty,2 ); Among them, r(t) is the reward value representing the overall loss of the vehicle; C f is the fuel consumption; C e is the power loss; C b is the battery consumption; C Penalty,1 is the vehicle start-stop penalty value; C Penalty,2 is the physical constraint penalty value.

5. The method according to claim 4, characterized in that The fuel loss is determined as follows: Obtaining multiple sets of discrete values ​​corresponding to engine fuel consumption rate, fuel loss per unit time, engine speed and torque, and fuel density; Substitute the engine fuel consumption rate, fuel loss per unit time, engine speed, engine torque and fuel density into the following third preset formula to determine the fuel loss: Among them, C f is the fuel consumption; c f is the fuel consumption per unit time; b e is the engine fuel consumption rate; Tor eng (t) is the engine torque, ω eng (t) is the engine speed; ρ is the fuel density.

6. The method according to claim 4, characterized in that The physical constraint penalty value is determined as follows: Determine whether the current battery power is within a preset power range, and determine whether the current vehicle torque is within a preset torque range; When the current battery power is within a preset power range and the current vehicle torque is within a preset torque range, determining that the physical constraint penalty value is 0; When the current battery power exceeds a preset power range or the current torque of the vehicle exceeds a preset torque range, the physical constraint penalty value is determined to be a specific value, wherein the specific value makes the reward value of the overall loss of the vehicle less than a preset value.

7. The method according to claim 1, characterized in that The loop updates the current driving strategy, including: Randomly select multiple four-tuple data from the experience pool corresponding to the agent as learning samples, where the four-tuple data includes the state at the historical moment, the action taken by the agent at the historical moment, the reward at the historical moment, and the state at the next historical moment; The intelligent agent is cyclically trained by the learning samples, and weight parameters are updated by the stochastic gradient descent method; The updated weight parameters are assigned to the Q network to achieve a cyclic update of the driving strategy in the intelligent agent.

8. A driving strategy determination device, characterized in that: include: An acquisition module is used to obtain a historical speed curve of the vehicle on a target route; An import module, used to import the historical speed curve into an intelligent agent, and calculate the total reward value corresponding to the historical speed curve through the intelligent agent, wherein the intelligent agent is used to generate a corresponding driving strategy according to the imported historical speed curve, and the intelligent agent is composed of a Q neural network and a target neural network, the Q neural network is used to calculate the current Q value according to the driving strategy and vehicle status in the intelligent agent, and the target neural network is used to calculate the maximum Q value; the total reward value is used to characterize the overall loss of the vehicle during driving, the overall loss includes the vehicle battery loss, and the overall reward value is negatively correlated with the overall loss of the vehicle during driving; A first determination module is used for cyclically updating the driving strategy in the intelligent agent when the total reward value corresponding to the historical speed curve of the vehicle is less than a preset reward value, and determining a current Q value calculated based on the updated driving strategy and the vehicle state, wherein the current Q value is positively correlated with the total reward value; The second determination module is used to determine that the agent training is completed when the difference between the Q value calculated based on the updated driving strategy and the vehicle state and the maximum Q value is less than a preset difference, so as to determine the driving strategy of the vehicle.

9. A driving strategy determination system, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the driving strategy determination method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor corresponding to the driving strategy determination system, the driving strategy determination system can implement the driving strategy determination method as described in any one of claims 1 to 7.

Images