A fuel saving control method

Abstract

The present invention discloses a fuel-saving control method, which includes: Step 1, constructing a fuel-saving control strategy; Step 2, fusing the visual information Camera outside the vehicle obtained by the vehicle itself with the vehicle speed v, acceleration acc, and CVT transmission ratio Ratio CVT into the state of deep reinforcement learning; Step 3, setting a reward function r(s,a) according to the state s; Step 4, inputting the state s into a neural network, and using the deep reinforcement learning action to control the throttle opening A throttle and the change amount △Ratio of the CVT transmission ratio CVT to output a fuel-saving control strategy. The present invention can reduce the fuel consumption of the vehicle.

Description

Technical Field

[0001] The invention relates to a method for controlling automobile energy saving and emission reduction, and in particular to a method for controlling fuel saving. Background Art

[0002] In the past few decades, with the booming economy and the improvement of people's living standards, people's requirements for daily travel have greatly increased, which has promoted the rapid development of the automobile industry. At the same time, adverse effects such as ecological environment deterioration and fossil fuel shortages have also followed. Therefore, the rapid development of the automobile industry is the primary cause of environmental pollution and energy shortages. Nowadays, human beings attach great importance to environmental and energy issues. Regulators in developed countries, led by the United States and the European Union, are formulating stricter emission standards to encourage automobile manufacturers to upgrade existing technologies and produce more efficient and energy-saving vehicles.

[0003] Countries around the world have put forward higher requirements for the energy-saving and emission-reduction performance of automobiles. Therefore, the introduction of more advanced technologies has become an urgent problem for automobile manufacturers. At present, electric vehicle technology is not perfect, so cars based on internal combustion engines will still occupy a large market share in the future. Due to its unique transmission structure, the mechanical continuously variable transmission (CVT) can achieve continuous changes in the transmission ratio. Therefore, CVT can ensure that the engine works in the optimal working area, thereby greatly reducing fuel consumption and pollution emissions. In this context, cars equipped with CVT transmissions have obvious advantages in energy saving and emission reduction due to their more flexible speed regulation performance. At present, the relationship between the engine, CVT transmission state and fuel consumption is analyzed for different working conditions such as automobile acceleration, constant speed and deceleration, and a fuel-saving optimization strategy for CVT-equipped cars is proposed. There are also optimal control theories to optimize the acceleration and coasting cycles of cars to reduce fuel consumption. There are also integrated control of engine torque and CVT transmission ratio, and the engine target torque and target transmission ratio are determined by calculating the engine operating point with the minimum instantaneous fuel consumption. However, these studies focus on optimizing engine efficiency and transmission efficiency, while ignoring the impact of external information on automobile fuel consumption, resulting in poor optimization results. Therefore, in order to achieve the best optimization effect, it is necessary to comprehensively consider the optimal efficiency of the engine and transmission as well as the predicted information of other vehicle behaviors or traffic light status. Summary of the invention

[0004] An object of the present invention is to provide a fuel saving control method to overcome or at least alleviate at least one of the above-mentioned drawbacks of the prior art.

[0005] To achieve the above object, the present invention provides a fuel saving control method, which comprises:

[0006] Step 1, construct a fuel saving control strategy as described below;

[0007] J i =ω1L di +ω2L fi +ω3L Ri (1)

[0008] Among them, J i is the instantaneous reward at step i, L di is the driving speed reward of the i-th step, L fi is the fuel consumption reward at step i, L Ri is the reward for the transmission ratio change in the i-th step, and ω1, ω2, and ω3 are weights set corresponding to their respective rewards, as shown in formula (2):

[0009]

[0010] Among them, Ratio cvt is the transmission ratio of the car's CVT transmission, Ratio ref is the reference transmission ratio of the CVT transmission of the car, fuel is the fuel consumption of the car, v is the driving speed of the car, abs is the absolute value, ω4, ω5 and ω6 are the weights set corresponding to the respective car parameters;

[0011] Step 2: Combine the external visual information Camera obtained by the vehicle with the vehicle's speed v, acceleration acc, and CVT transmission ratio Ratio CVT According to the following formula (3), it is fused into the state s of deep reinforcement learning;

[0012] s=[Camera,v,acc,Ratio CVT ] (3)

[0013] Step 3, according to the state s, set the reward function r(s,a);

[0014] r(s,a)=bJ (4)

[0015] Among them, b is the deviation of the adjustment range of r(s,a), J is the reward, and a is the action, including the control action of the throttle opening and the change of the CVT transmission ratio;

[0016] Step 4: Input the state s into the neural network and use deep reinforcement learning to calculate the throttle opening A according to the action provided by the following formula (5): throttle and CVT transmission ratio change ΔRatio CVT Control and output fuel saving control strategy:

[0017]

[0018] Furthermore, it is set that the round ends when a rear-end collision or a red light run occurs, and the round reward J e Add to instant reward J i In the above equation, we get J=J i +J e .

[0019] Furthermore, the visual information includes image information of the behavior of the preceding vehicle and the status of traffic lights acquired by the vehicle-mounted camera.

[0020] Due to the above technical solution, the present invention has the following advantages: the present invention uses a vehicle-mounted camera to collect real-time visual information, and then integrates the real-time visual information processed by CNN (the full name in English is "convolutional neural network", and the full name in Chinese is "convolutional neural network") with the inherent state information of the vehicle, and uses it as the state input of deep reinforcement learning, outputs the optimal fuel-saving control action, and reduces the fuel consumption of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the principle of a SAC-based DRL algorithm provided in an embodiment of the present invention.

[0022] Figure 2 A schematic diagram of observing cars and traffic lights in a city cycle provided by an embodiment of the present invention, wherein a vehicle-mounted camera collects image information.

[0023] Figure 3 The convolutional neural network provided in the embodiment of the present invention collects image information through the vehicle camera, processes the image through the convolutional neural network, and inputs the processed image into the neural network of deep reinforcement learning together with other information such as speed and acceleration.

[0024] Figure 4 A structural diagram of the data transmission and power system provided for an embodiment of the present invention.

[0025] Figure 5 The real-time communication time provided by the embodiment of the present invention.

[0026] Figure 6 Schematic diagram of a model building platform provided in an embodiment of the present invention.

[0027] Figure 7 A schematic diagram of a WLTC standard operating speed curve provided in an embodiment of the present invention.

[0028] Figure 8 A schematic diagram of a WLTC operating condition experimental environment test provided by an embodiment of the present invention.

[0029] Fig. 9 A schematic diagram of losses and rewards provided in an embodiment of the present invention.

[0030] Fig.10 A schematic diagram for comparing vehicle driving speeds provided by an embodiment of the present invention.

[0031] Fig.11 A schematic diagram for comparing CVT speed ratio change results provided in an embodiment of the present invention.

[0032] Fig.12 A schematic diagram for comparing fuel consumption per unit time provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0034] The fuel-saving control method aims to improve fuel economy while ensuring the power of the vehicle.

[0035] The fuel saving control method provided in this embodiment includes:

[0036] Step 1: First, determine the starting point and end point of the car to achieve global path planning. Secondly, optimize the real-time driving strategy of the car to reduce fuel consumption as much as possible within a reasonable time range. In the driving cycle, the driving speed directly determines the time required for the entire driving cycle, so the driving speed reward is used to replace the time reward required for the driving cycle. These two target rewards are combined with the safe driving target reward, and the optimal control is achieved by optimizing the reward at each time point, as shown in the following formula (1);

[0037] J i =ω1L di +ω2L fi +ω3L Ri (1)

[0038] Among them, J i is the instantaneous reward at step i, L di is the driving speed reward of the i-th step, L fi is the fuel consumption reward at step i, L Ri is the reward for the transmission ratio change in the i-th step, and ω1, ω2, and ω3 are weights set corresponding to their respective rewards, as shown in formula (2):

[0039]

[0040] Among them, Ratio cvt is the transmission ratio of the car's CVT transmission, Ratio refis the reference transmission ratio of the CVT transmission of the car, fuel is the fuel consumption of the car, v is the driving speed of the car, abs is the absolute value, ω4, ω5 and ω6 are weights set according to the corresponding car parameters. The numerical requirements of the above weights are to achieve the driving speed v, fuel consumption fuel and CVT transmission ratio Ratio cvt For example, to ensure the car's driving speed v, if only the fuel consumption reward is set, the car will choose to stop because the fuel consumption reward is the largest when it stops.

[0041] Step 2: Combine the external visual information Camera obtained by the vehicle with the vehicle's speed v, acceleration acc, and CVT transmission ratio Ratio CVT According to the following formula (3), the state s of deep reinforcement learning is fused. The visual information Camera may include image information of the behavior of the preceding vehicle and the state of the traffic light acquired by the vehicle-mounted camera.

[0042] s=[Camera,v,acc,Ratio CVT ] (3)

[0043] Step 3: According to the state s, solve the optimization problem proposed in (1) in the SAC-based DRL framework and set the reward function r(s,a);

[0044] r(s,a)=bJ (4)

[0045] Among them, J is the reward; a is the action, including the control action of throttle opening and CVT transmission ratio change; b is the deviation of adjusting the range of r(s,a). For example: if the range of J is (0,10), the value of deviation b is -5, then the final reward range is (-5,5). By setting the deviation, the final reward range can be made consistent with the deep reinforcement learning algorithm.

[0046] In one embodiment, collision and red light running may occur in the experiment. Therefore, the round is set to end when rear-end collision and red light running occur, and the round reward J is e Add to instant reward J i In the above equation, we get J=J i +J e .

[0047] According to the state s, action a and reward function r(s,a) defined in the above embodiment, the proposed SAC-DRL strategy is Figure 1 Given, two Q networks are trained independently to eliminate over-estimation and improve training speed. Figure 2 The external information collected by the vehicle camera is Figure 3The Q network is composed of a convolutional neural network and a fully connected layer. The policy network has the same hidden layer structure as the Q network, and it outputs a two-dimensional vector representing the distribution to determine the action. The network is updated using the Adam optimizer, and the relevant hyperparameters are given in Table 1.

[0048]

[0049] Step 4: According to the engine steady-state output torque model, the engine fuel consumption rate model and the engine optimal fuel economy curve, it is found that the engine speed and output power can be freely adjusted by controlling the throttle opening. Therefore, the speed, torque, power and other information can be obtained by determining the engine throttle opening and the CVT transmission ratio change. Therefore, the state s is input into the neural network, and the action provided by deep reinforcement learning is given by the following formula (5). The action a is continuously distributed within the allowable range, and the throttle opening A is set. throttle and CVT transmission ratio change ΔRatio CVT Carry out continuous control and output fuel saving control strategy:

[0050]

[0051] This embodiment uses an on-board camera to collect real-time visual information, and then integrates the real-time visual information processed by CNN with the inherent state information of the car, and uses it as the state input of deep reinforcement learning to output the optimal fuel-saving control action to reduce the fuel consumption of the car. The deep reinforcement learning algorithm can be implemented, but is not limited to, using SAC-DRL. SAC is an Actor-Critic deep reinforcement learning algorithm based on the maximum entropy reinforcement learning framework. In the SAC algorithm, the goal of the Actor needs to maximize the reward and maximize the information entropy. It can ensure the randomness of the action as much as possible while completing the task of maximizing the reward.

[0052] In one embodiment, by simulating the fuel consumption of the car in the following scene and the traffic light scene in the urban driving cycle, the internal power system of the car is simulated using Simulink simulation software, and the speed v, acceleration acc, fuel consumption fuel and the transmission ratio Ratio of the CVT transmission are calculated. cvt The simulation software Simulink is used to obtain the vehicle's external vision. The Carla simulation software is used to simulate the vehicle's external vision. The deep learning algorithm is used to optimize the vehicle's driving speed in the above two scenarios, thereby achieving the goal of saving fuel.

[0053] Carla simulation software is an open source autonomous driving simulator that runs simulations based on the Unreal Engine UE4. The car's visual information is obtained based on this simulator.

[0054] In the joint simulation of Carla simulation software and Simulink simulation software, Python is used between Simulink simulation software and Carla simulation software to exchange information between Carla simulation software and Simulink simulation software. Figure 8 As shown, the main program is written in Python, which controls the start and stop of Carla and Simulink, sets parameters and reads simulation results by calling the Python API inside Carla and Simulink.

[0055] Figure 5 The time taken by the Python main program to interact with the Carla simulator, Simulink, and the fuel-saving control strategy based on deep reinforcement learning.

[0056]

[0057] Table 2 shows the average time delay of the joint simulation software communication. When the car is traveling at 120 km / h, the delay time will cause the car to move forward 0.59 m, which can be ignored for the study of fuel-saving control strategy. Therefore, the joint simulation meets the experimental environment proposed in this embodiment. Figure 6 shown.

[0058] In order to evaluate the proposed DRL model, the Carla simulator is used to simulate the information of the ego vehicle, surrounding vehicles, and traffic lights. In the Carla simulation software, a vehicle camera is installed on the front windshield of the ego vehicle to collect visual information including the status of the leading vehicle and traffic lights, such as Figure 2 shown.

[0059] A city cycle simulation based on Carla simulation software was established, the starting and ending points of the car were defined, and the A* algorithm was used for global path planning. The external computer vision system and the internal power system of the car were simulated using Carla simulation software and Simulink simulation software. The car's driving speed was obtained from the car's power system and passed to the Carla simulation software as the driving speed of the virtual car equipped with an on-board camera. At this time, the car was equipped with a monocular camera to realize real-time collection of visual information.

[0060] The embodiment of the present invention builds a Carla-Simulink joint simulation model to evaluate the performance of the proposed method. In order to make the proposed optimization algorithm easier to understand, Figure 4The data transmission and power system structure diagram is provided in the figure. The two types of observations include road traffic information collected from the on-board camera, external visual observations such as traffic light status, and the inherent state of the car itself such as speed, acceleration, and CVT transmission ratio. At each time step, the DRL algorithm receives observations of the environment to determine the output of the engine and CVT transmission. Afterwards, the fuel consumption of the car (the result of the decision) is sent back to the DRL.

[0061] like Figure 4 As shown, the possible contributions of the method proposed in this embodiment mainly include the following aspects.

[0062] (1) By comprehensively processing real-time traffic information obtained by on-board visual sensors, such as road traffic information and traffic light information, the fuel economy of vehicles equipped with CVT transmissions is improved, and a Carla-Simulink joint simulation model based on deep reinforcement learning is built.

[0063] (2) Based on SAC continuous deep reinforcement learning, the throttle opening and CVT transmission ratio change are continuously controlled to generate continuous and reasonable engine torque and speed control signals and CVT transmission ratio signals respectively. Compared with the deep value network (DQN), this strategy shows accelerated convergence and enhanced optimization performance.

[0064] (3) Using the Carla simulation software, we established an urban driving cycle with real-time visual information, including car-following scenarios and traffic light scenarios. Unlike existing driving cycles that only provide vehicle speed information, our driving cycle provides image data at each time step, which can be used in future CVT vehicles with image modules and has good application prospects.

[0065] To evaluate the effectiveness of the above SAC-DRL-based fuel-saving control method, the DRL model was implemented based on Pytorch, an open source deep learning platform. All simulations in this experiment were performed on a personal computer equipped with an Intel Core i7-11800H CPU (clocked at 2.3 GHz) and a GeForceRTX 3060 graphics processor (GPU) to assist in accelerating the training phase.

[0066] During the training process, the car first drives along the standard WLTC speed trajectory without vision. The WLTC speed curve is as follows Figure 7 As shown, it takes 1800 seconds to travel 23.14 km.

[0067] from Figure 7 As can be seen from the figure, the car is simulated driving in the experimental environment under the WLTC condition. At this time, the MPC-based fuel saving control strategy is adopted. The results are as follows Figure 8As shown, its speed change curve.

[0068] The speed curve is similar to the standard WLTC operating condition, and only when the vehicle speed is high does a significant speed difference appear. Therefore, it is proved that the proposed experimental environment can simulate the WLTC operating condition well, and the experimental environment can be used to simulate and compare the fuel consumption of automobiles.

[0069] The loss function and reward are used to evaluate the convergence process of the SAC-DRL strategy, such as Fig. 9 As shown in the figure, due to being in an unfamiliar environment, about 100,000 steps ago, the agent randomly selected actions on a large scale to familiarize itself with the environment, resulting in a large loss value. As the training progresses, the loss value gradually decreases after 300,000 steps and approaches zero after about 600,000 steps. Therefore, it can be regarded as the convergence of the learning strategy.

[0070] By using the reward formula 22 as the reward function, the agent is expected to choose actions that reach the destination while saving fuel as much as possible and punish behaviors that violate traffic regulations. The average reward value changes with training as shown in Fig. 9 As shown in the figure, the study found that when the agent learns environmental feedback and strives to adjust the goal to make it close to the optimal goal, the reward in the initial stage is fluctuating. After 190 cycles, the reward tends to stabilize at a higher level, which shows that the agent has successfully raised all goals to the expected value. The above results show that SAC has been well trained and converged, and can perform subsequent optimization tasks.

[0071]

[0072]

[0073] The training process is crucial to evaluating the overall performance of DRL. Table 3 compares the training time of the proposed SAC strategy with that of the classic DQN. Thanks to the Actor-Critic architecture and the application of maximum entropy, the strategy is more random, outputs more near-optimal actions, and improves performance. From Table 3, it can be observed that the agent based on the SAC strategy reaches the training goal faster. The training cycle required by SAC is much smaller than that of DQN, requiring only 40% of the training time of DQN. This shows that the training efficiency of DRL based on SAC is significantly improved, greatly accelerating the convergence speed.

[0074] In order to evaluate the fuel saving performance of the DRL strategy in car following and traffic light scenarios, the proposed strategy is compared with the MPC method without considering the prediction of car following and traffic light scenarios for a simple evaluation.

[0075] The present invention takes photos through a vehicle-mounted camera and inputs them into a convolutional neural network, which is then input into a deep neural network, with the goal of making decisions that optimize the fuel economy of the vehicle. Therefore, the present invention uses the obtained computer vision information as a variable. The MPC-based method that does not process the visual system is used as a benchmark to verify the effectiveness of the proposed vision-based method.

[0076] The following three methods are involved:

[0077] (1) An automobile fuel-saving control method based on MPC without computer vision.

[0078] (2) A DQN-based automobile fuel-saving control method involving computer vision.

[0079] (3) Automobile fuel-saving control method based on SAC including computer vision.

[0080] In the experiment, the three different algorithm models were compared in terms of driving speed, CVT transmission ratio, and fuel consumption. The results are as follows: Figure 10-12 The three algorithms use the same parameter settings to ensure a fair comparison.

[0081] Fig.10 The speed trajectory data collected in the urban driving cycle proposed by the present invention is shown. It can be seen that when the car based on the DRL method performs the following task, by observing the behavior of the preceding vehicle, the car can maintain economic driving better than the MPC-based method, and by observing the state of the traffic lights, the car can better optimize the driving speed before the traffic lights. The present invention uses a convolutional neural network to recognize and predict images containing traffic lights. In addition, existing studies have shown that the driving state of the car depends largely on the driving state of the preceding vehicle and the states of nearby cars and traffic lights. Therefore, digital processing and analysis of real-time traffic information is beneficial to optimizing the fuel economy of the car.

[0082] The fuel-saving control method based on deep reinforcement learning realizes the coordinated control of the engine and CVT transmission. In the control of the longitudinal power system of the vehicle, the transmission ratio of the CVT affects the working range of the engine. Real-time optimization of the transmission ratio change of the CVT can ensure that the engine works in the optimal working range. The DRL-based strategy uses the MPC-based shifting strategy as a reference to improve training efficiency. Among them, the transmission ratio change based on DQN is discrete to [-0.1, -0.05, 0, 0.05, 0.1], and the transmission ratio change based on SAC is continuously distributed in [-0.1, 0.1], aiming to optimize the MPC-based shifting strategy. Fig.11The three different control methods of CVT speed ratio control strategies are shown in Figure 2. In general, the distribution characteristics of the control strategy based on DRL and the control strategy based on MPC are similar. At the same time, since the reward function of DRL contains multiple optimization objectives such as fuel consumption, the CVT transmission ratio based on the DRL strategy is kept within a reasonable range and the shifting strategy based on MPC is optimized.

[0083] The control variables of the DRL-based control strategy are the throttle opening and the CVT transmission ratio change. Fig.12 It can be seen that the fuel consumption per unit time based on the DRL strategy is significantly lower than that of the MPC strategy. Therefore, it can be said that the DRL algorithm has the ability to learn the throttle opening changes and CVT shifting strategies.

[0084]

[0085] The fuel consumption of the three control strategies is shown in Table 4. Taking the results obtained by the MPC strategy as a benchmark, among the DRL-based strategies, the DQN strategy reduced fuel consumption by 4.46%, and the SAC strategy reduced fuel consumption by 6.41%. Therefore, the introduction of the fuel-saving control method of the computer vision system is beneficial to saving fuel. The two DRL strategies produced a fuel consumption difference of 1.95%, which is mainly due to the fact that the action control of the SAC strategy is continuous. Under the action of maximum entropy, the action output is more random and the control effect is better.

[0086] This example takes a fuel-powered vehicle equipped with a CVT transmission as the research object, and proposes a fuel-saving control method that combines deep reinforcement learning with computer vision in the following and traffic light scenarios in the urban driving cycle to optimize the fuel economy of the vehicle. To verify this method, a Carla-Simulink joint simulation model was built, an urban road driving cycle with visual information was constructed in Carla, and a vehicle power system model was constructed in Simulink. The average time for each set of actions output in the simulation was 17.55ms.

[0087] In addition, we conducted a comparative study on whether to combine with a computer vision system and different DRL algorithms, and found that the SAC algorithm based on a computer vision system is most suitable for fuel-saving control of automobiles. The simulation results show that the SAC and DQN fuel-saving control strategies based on computer vision have achieved a fuel consumption difference of 4.46% and 6.41% respectively compared with the MPC-based control strategy under the condition of synchronous control of the engine and CVT transmission. At the same time, the training speed of the SAC-based control strategy is faster, which is only 40% of the training time of the DQN-based control strategy.

[0088] In this embodiment, we only consider the real-time image data collected from the vehicle-mounted camera. With the advancement of sensor and cloud technology, exploring the optimal fuel-saving control method by collecting information from other types of sensors, such as GPS, radar, lidar, etc., and obtaining information through vehicle-to-vehicle and vehicle-to-infrastructure is the future development direction.

[0089] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fuel saving control method, characterized in that: include: Step 1, construct a fuel saving control strategy as described below; J i =ω1L di +ω2L fi +ω3L Ri (1) Among them, J i is the instantaneous reward at step i, L di is the driving speed reward of the i-th step, L fi is the fuel consumption reward at step i, L Ri is the reward for the transmission ratio change in the i-th step, and ω1, ω2, and ω3 are weights set corresponding to their respective rewards, as shown in formula (2): Among them, Ratio cvt is the transmission ratio of the car's CVT transmission, Ratio ref is the reference transmission ratio of the CVT transmission of the car, fuel is the fuel consumption of the car, v is the driving speed of the car, abs is the absolute value, ω4, ω5 and ω6 are the weights set corresponding to the respective car parameters; Step 2: Combine the external visual information Camera obtained by the vehicle with the vehicle's speed v, acceleration acc, and CVT transmission ratio Ratio CVT According to the following formula (3), it is fused into the state s of deep reinforcement learning; s=[Camera,v,acc,Ratio CVT ] (3) Step 3, according to the state s, set the reward function r(s,a); r(s,a)=bJ (4) Among them, b is the deviation of the adjustment range of r(s,a), J is the reward, and a is the action, including the control action of the throttle opening and the change of the CVT transmission ratio; Step 4: Input the state s into the neural network and use deep reinforcement learning to calculate the throttle opening A according to the action provided by the following formula (5): throttle and CVT transmission ratio change ΔRatio CVT Control and output fuel saving control strategy:

2. The fuel saving control method according to claim 1, characterized in that: Set the round to end when a rear-end collision or red light running occurs, and the round reward J e Add to instant reward J i In the above equation, we get J=J i +J e .

3. The fuel saving control method according to claim 1 or 2, characterized in that: Visual information includes image information about the behavior of the preceding vehicle and the status of traffic lights obtained by the on-board camera.

Images