Energy management method for hybrid electric vehicle based on traffic density perception and reinforcement learning

Abstract

The present application relates to the field of intelligent management of vehicles, and proposes a hybrid vehicle energy management method based on traffic density perception and reinforcement learning, which improves the traffic density accuracy by comprehensively perceiving the traffic density, accurately judges the actual traffic scene state, and realizes efficient energy management adjustment in different traffic density scenes, and then predicts the vehicle speed through a hybrid prediction model, which not only effectively captures the long-term dependence and periodic characteristics in the speed sequence, but also accurately captures the random state transition characteristics and instantaneous changes of the speed, further improving the speed prediction accuracy, and through the energy management optimization based on the reinforcement learning framework, combined with the multi-source state of the vehicle, the fuel consumption balance and battery SOC maintenance balance under different working conditions are realized, and the adaptability and effectiveness of the hybrid vehicle energy management method are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle management, and in particular to a hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning. Background Technology

[0002] With the rapid development of the new energy vehicle industry, the number of new energy vehicles of various types is increasing day by day. Among them, hybrid vehicles, with their collaborative architecture of "internal combustion engine-electric motor", have shown unique advantages in terms of driving range and energy efficiency, and therefore occupy a crucial position in new energy vehicles. As the functional requirements and technical requirements for new energy vehicles continue to increase, the requirements for the energy management capabilities of hybrid vehicles are also gradually increasing.

[0003] In existing technologies, when traditional energy management methods for hybrid vehicles are applied to real-world traffic scenarios, the contradiction between control accuracy and adaptability to operating conditions becomes increasingly prominent. Existing energy management technologies often suffer from three problems: First, existing energy management methods based on experience-driven rule control are difficult to adapt to the nonlinear driving demands in urban conditions due to static threshold setting mechanisms. Second, existing optimization algorithms based on dynamic programming and model predictive control are limited by vehicle speed prediction models within a finite field of view (especially since they are not coupled with real-time traffic density parameters), resulting in a significant decrease in optimization effectiveness on actual roads. Third, machine learning methods, represented by long short-term memory networks and reinforcement learning, generally suffer from insufficient fusion of multi-source traffic information, leading to a lack of environmental perception dimension in the decision-making model.

[0004] Therefore, how to design a hybrid vehicle energy management method to improve its adaptability and stability in actual traffic scenarios, thereby achieving a more reasonable optimization that meets actual needs, has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, the present invention proposes a hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning. By comprehensively perceiving traffic flow density, the accuracy of traffic flow density is improved to accurately determine the actual traffic scenario state, enabling efficient energy management and adjustment under different traffic flow density scenarios. Furthermore, vehicle speed prediction is performed through a hybrid prediction model, which not only effectively captures the long-term dependence and periodic characteristics in the vehicle speed sequence, but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving the accuracy of vehicle speed prediction. Moreover, through energy management optimization based on a reinforcement learning framework, combined with the multi-source state of the vehicle, fuel consumption balance and battery SOC maintenance balance under different operating conditions are achieved. The present invention improves the adaptability and effectiveness of hybrid vehicle energy management methods.

[0006] This invention proposes a hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning, comprising:

[0007] Real-time traffic scene images are acquired and preprocessed, and traffic flow density perception processing is performed according to a deep learning algorithm to obtain traffic flow density data. The traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception.

[0008] Vehicle speed is predicted according to a hybrid prediction model, which is based on a long short-term memory network and a Markov chain;

[0009] Energy management optimization is performed based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. The energy management optimization is based on a reinforcement learning framework.

[0010] In summary, the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning improves traffic flow density accuracy by comprehensively perceiving traffic flow density, thereby accurately judging the actual traffic scenario state and enabling efficient energy management and adjustment under different traffic flow density scenarios. Furthermore, the hybrid prediction model for vehicle speed prediction not only effectively captures the long-term dependencies and periodic features in the vehicle speed sequence but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving prediction accuracy. Additionally, the energy management optimization based on a reinforcement learning framework, combined with the vehicle's multi-source states, achieves fuel consumption balance and battery SOC maintenance balance under different operating conditions. Therefore, this invention improves the adaptability and effectiveness of the hybrid vehicle energy management method. Specifically, the process involves acquiring and preprocessing real-time traffic scene images, performing traffic flow density perception processing using a deep learning algorithm to obtain traffic flow density data. This traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception, improving the accuracy of traffic flow density and enabling accurate judgment of the actual traffic scene state. This allows for efficient energy management and adjustment under different traffic flow density scenarios. Vehicle speed prediction is then performed using a hybrid prediction model based on Long Short-Term Memory (LSTM) networks and Markov chains. This model effectively captures not only the long-term dependencies and periodic features in the vehicle speed sequence but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving the accuracy of speed prediction. Finally, energy management optimization is performed based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. This energy management optimization is based on a reinforcement learning framework, combined with the vehicle's multi-source states, to achieve fuel consumption balance and battery SOC maintenance balance under different operating conditions. This invention improves the adaptability and effectiveness of hybrid vehicle energy management methods.

[0011] Furthermore, the step of acquiring and preprocessing real-time traffic scene images specifically includes:

[0012] Acquire real-time traffic scene images, which include lane vehicle distribution images;

[0013] The real-time traffic scene image is converted to grayscale. The specific algorithm for grayscale conversion is as follows:

[0014] ,

[0015] in, I gray This represents the grayscale image pixel values ​​of a real-time traffic scene image. k r , k g , k b R represents the scaling factor, and G and B represent the red, green, and blue channel pixel values ​​of the real-time traffic scene image.

[0016] Gaussian filtering is performed using a two-dimensional Gaussian filtering function. The specific algorithm for Gaussian filtering is as follows:

[0017] ,

[0018] in, G(x,y) This represents a two-dimensional Gaussian filter function. σ This represents the standard deviation of a two-dimensional Gaussian filter function. f(x,y) , I(x,y) These represent the original grayscale image of the real-time traffic scene and the real-time traffic scene image after Gaussian filtering, respectively. x,y These represent the x and y coordinates of a pixel, respectively.

[0019] Furthermore, the step of performing traffic flow density perception processing based on deep learning algorithms to obtain traffic flow density data specifically includes:

[0020] Vehicle targets are detected in preprocessed real-time traffic scene images using a target detection model based on a deep learning algorithm to obtain vehicle target information, including vehicle target location information and vehicle target sequence information. Visual traffic density is then perceived based on this vehicle target information to obtain visual traffic density data. The loss function of the target detection model is as follows:

[0021] ,

[0022] ,

[0023] ,

[0024] in, This represents the total loss of the object detection model. , Let represent the confidence loss and localization loss of the object detection model, respectively. p This indicates the probability that the target exists. q This indicates the score of the intersection and union of objectives. This indicates the weights for balancing positive and negative samples. Indicates the modulation factor. Indicates the first i A specified box value, For the first j Anchor box values, n , m These represent the total number of box values ​​and anchor box values, respectively;

[0025] Speed ​​and traffic density are perceived based on vehicle speed information in different lanes.

[0026] Furthermore, the step of sensing speed and traffic density based on vehicle speed information of different lanes specifically includes:

[0027] The speed-vehicle flow density data is calculated based on the real-time vehicle speed. The specific algorithm for calculating the speed-vehicle flow density data is as follows:

[0028] ,

[0029] in, Represents speed and traffic density data. k j Indicates blockage density, k m Indicates the optimal density. k f Represents free flow density. v f Indicates the free flow velocity. v m The characteristic velocity representing high-density congestion flow, v k Indicates the current vehicle speed. a 1. a 2 represents the traffic density interval coefficient;

[0030] The comprehensive traffic flow density data is calculated based on visual traffic flow density data and speed traffic flow density data. The specific algorithm for calculating the comprehensive traffic flow density data is as follows:

[0031] ,

[0032] in, k Indicates the overall traffic density. Represents speed and traffic density data. k vision Indicates visual traffic density;

[0033] Based on comprehensive traffic density data, traffic flow status can be divided into free flow, transitional flow, and congested flow.

[0034] Furthermore, the step of predicting vehicle speed based on the hybrid prediction model to obtain the final vehicle speed prediction specifically includes:

[0035] Vehicle speed is predicted based on a hybrid prediction model, which is based on a long short-term memory network and a Markov chain. The long short-term memory network is time-driven, and the Markov chain is state-driven.

[0036] The specific algorithm for the Long Short-Term Memory network is as follows:

[0037] ,

[0038] ,

[0039] ,

[0040] in, This represents the input feature vector of the Long Short-Term Memory network. This indicates the vehicle speed at the current moment. This represents the acceleration at the current moment. This indicates the slope angle at the current moment. Represents a d-dimensional real vector. This represents the hidden state of the Long Short-Term Memory network. LSTM represents the cell state of a Long Short-Term Memory (LSS) network. This indicates the speed predicted by the Long Short-Term Memory network model. This represents the weight matrix of the output layer of a Long Short-Term Memory (LSTM) network. This represents the bias term of the output layer of the Long Short-Term Memory network;

[0041] The specific algorithm for the Markov chain is as follows:

[0042] ,

[0043] ,

[0044] in, c k This represents the center velocity value of the discrete state within the vehicle speed range. v min and v max These represent the minimum and maximum values ​​within the vehicle speed range, respectively. k Indicates the overall traffic density. KThis represents the number of discrete states within a vehicle speed range. This indicates the vehicle speed predicted based on the Markov chain. This represents the elements of the state transition matrix constructed based on historical data;

[0045] Obtain the vehicle speed prediction values ​​for the Long Short-Term Memory network and the Markov chain, respectively.

[0046] Furthermore, the steps of obtaining the vehicle speed prediction values ​​for the Long Short-Term Memory network and the Markov chain, respectively, further include:

[0047] The final vehicle speed prediction is obtained by fusing the vehicle speed prediction values ​​from multi-head attention mechanism, long short-term memory network, and Markov chain. The specific algorithm for obtaining the final vehicle speed prediction is as follows:

[0048] ,

[0049] ,

[0050] ,

[0051] ,

[0052] ,

[0053] in, This represents the hidden state of the Long Short-Term Memory network. This indicates the vehicle speed predicted based on the Markov chain. z t This represents a mixture of hidden states from a Long Short-Term Memory (LSTM) network and vehicle speed predictions based on Markov chains. Indicates the hidden dimension. q t , k t These represent the query vector and the key vector, respectively. W q , W k Represents the learnable projection matrix. d k Represents the attention dimension. Indicates attention weights. This represents the prediction weights of the Long Short-Term Memory network. This indicates the speed predicted by the Long Short-Term Memory network model. This indicates the predicted final vehicle speed.

[0054] Furthermore, the step of optimizing energy management based on the traffic flow density data and the final vehicle speed prediction to obtain the final energy management strategy specifically includes:

[0055] An intelligent agent is constructed based on traffic flow density data, final vehicle speed prediction, and multi-source vehicle state data, including battery SOC status, engine speed, engine torque, and required power.

[0056] The specific algorithm for constructing the intelligent agent is as follows:

[0057] ,

[0058] in, Represents an intelligent agent. The agent state variables representing the battery's SOC state. The agent state variable representing engine speed. The agent state quantity representing the engine torque. The agent's state variables representing the required power. The agent state variables representing traffic flow density data This represents the agent's state variables for predicting the final vehicle speed.

[0059] The algorithm for adjusting the engagement ratio of the engine and motor in a hybrid vehicle based on power distribution is as follows:

[0060] ,

[0061] ,

[0062] ,

[0063] in, This indicates the engagement ratio of the engine and electric motor in a hybrid vehicle. Indicates engine power. Indicates motor power. This indicates the power requirement of hybrid vehicles;

[0064] A reward function is constructed, and the weighting coefficients of fuel consumption reward and electricity consumption reward are dynamically adjusted. Under congested conditions, the weighting coefficient of electricity consumption reward is increased to balance the reward function towards battery SOC. Under free-flowing conditions, the weighting coefficient of fuel consumption reward is increased to balance the reward function towards fuel economy. The specific algorithm for the reward function is as follows:

[0065] ,

[0066] in, Represents the reward function, w 1 and w 2 represents the weighting coefficients for fuel consumption rewards and electricity consumption rewards, respectively. This refers to the engine's instantaneous fuel consumption. Indicates the reference SOC value. Parameters representing the suppression of power jitter. This indicates the power jitter value.

[0067] This invention proposes a hybrid vehicle energy management system based on traffic flow density perception and reinforcement learning, comprising:

[0068] The traffic density perception module is used to acquire real-time traffic scene images and perform preprocessing, and to perform traffic density perception processing according to deep learning algorithms to obtain traffic density data. The traffic density perception processing includes visual traffic density perception and speed traffic density perception.

[0069] The vehicle speed prediction module is used to predict vehicle speed according to a hybrid prediction model to obtain the final vehicle speed prediction. The hybrid prediction model is based on a long short-term memory network and a Markov chain.

[0070] An energy management optimization module is used to perform energy management optimization based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. The energy management optimization is based on a reinforcement learning framework.

[0071] The present invention also provides a storage medium that stores one or more programs, which, when executed by a processor, implement the hybrid vehicle energy management method based on vehicle flow density perception and reinforcement learning as described above.

[0072] The present invention also provides a computer device, the computer device including a memory and a processor, wherein:

[0073] The memory is used to store computer programs;

[0074] When the processor executes the computer program stored in the memory, it implements the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning as described above. Attached Figure Description

[0075] Figure 1 This is a flowchart of the hybrid vehicle energy management method based on traffic density perception and reinforcement learning proposed in the first embodiment of the present invention;

[0076] Figure 2 This is a schematic diagram of the hybrid vehicle energy management system based on traffic flow density perception and reinforcement learning proposed in the second embodiment of the present invention.

[0077] Figure 3This is a morning rush hour road map according to the first embodiment of the present invention;

[0078] Figure 4 This is a comparison chart of fuel consumption under different operating conditions according to the first embodiment of the present invention;

[0079] Figure 5 This is a comparison chart of battery SOC under the morning peak operating conditions according to the first embodiment of the present invention.

[0080] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0081] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0082] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0084] Please see Figure 1 The diagram shows a flowchart of the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning proposed in the first embodiment of the present invention. This hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning includes steps S01 to S03, wherein:

[0085] Step S01: Acquire real-time traffic scene images and perform preprocessing; use deep learning algorithms to perform traffic flow density perception processing to obtain traffic flow density data.

[0086] It should be noted that in this embodiment, the traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception, and acquires a real-time traffic scene image, which includes a lane vehicle distribution image.

[0087] The real-time traffic scene image is converted to grayscale. The specific algorithm for grayscale conversion is as follows:

[0088] ,

[0089] in, I gray This represents the grayscale image pixel values ​​of a real-time traffic scene image. k r , k g , k b R represents the scaling factor, and G and B represent the red, green, and blue channel pixel values ​​of the real-time traffic scene image.

[0090] Gaussian filtering is performed using a two-dimensional Gaussian filtering function. The specific algorithm for Gaussian filtering is as follows:

[0091] ,

[0092] in, G(x,y) This represents a two-dimensional Gaussian filter function. σ This represents the standard deviation of a two-dimensional Gaussian filter function. f(x,y) , I(x,y) These represent the original grayscale image of the real-time traffic scene and the real-time traffic scene image after Gaussian filtering, respectively. x,y These represent the x and y coordinates of a pixel, respectively.

[0093] Vehicle targets are detected in preprocessed real-time traffic scene images using a target detection model based on a deep learning algorithm to obtain vehicle target information, including vehicle target location information and vehicle target sequence information. Visual traffic density is then perceived based on this vehicle target information to obtain visual traffic density data. The loss function of the target detection model is as follows:

[0094] ,

[0095] ,

[0096] ,

[0097] in, This represents the total loss of the object detection model. , Let represent the confidence loss and localization loss of the object detection model, respectively. p This indicates the probability that the target exists. q This indicates the score of the intersection and union of objectives. This indicates the weights for balancing positive and negative samples. Indicates the modulation factor. Indicates the first i A specified box value, For the first j Anchor box values, n , m These represent the total number of box values ​​and anchor box values, respectively;

[0098] Speed ​​and traffic density are perceived based on vehicle speed information in different lanes.

[0099] The speed-vehicle flow density data is calculated based on the real-time vehicle speed. The specific algorithm for calculating the speed-vehicle flow density data is as follows:

[0100] ,

[0101] in, Represents speed and traffic density data. k j Indicates blockage density, k m Indicates the optimal density. k f Represents free flow density. v f Indicates the free flow velocity. v m The characteristic velocity representing high-density congestion flow, v k Indicates the current vehicle speed. a 1. a 2 represents the traffic density interval coefficient;

[0102] The comprehensive traffic flow density data is calculated based on visual traffic flow density data and speed traffic flow density data. The specific algorithm for calculating the comprehensive traffic flow density data is as follows:

[0103] ,

[0104] in, k Indicates the overall traffic density. Represents speed and traffic density data. k vision Indicates visual traffic density;

[0105] Based on comprehensive traffic density data, traffic flow status can be divided into free flow, transitional flow, and congested flow.

[0106] Step S02: Predict vehicle speed based on the hybrid prediction model to obtain the final vehicle speed prediction;

[0107] It should be noted that in this embodiment, the hybrid prediction model is based on a long short-term memory network and a Markov chain. The vehicle speed is predicted according to the hybrid prediction model. The hybrid prediction model is based on a long short-term memory network and a Markov chain. The long short-term memory network is based on time-driven and the Markov chain is based on state-driven.

[0108] The specific algorithm for the Long Short-Term Memory network is as follows:

[0109] ,

[0110] ,

[0111] ,

[0112] in, This represents the input feature vector of the Long Short-Term Memory network. This indicates the vehicle speed at the current moment. This represents the acceleration at the current moment. This indicates the slope angle at the current moment. Represents a d-dimensional real vector. This represents the hidden state of the Long Short-Term Memory network. LSTM represents the cell state of a Long Short-Term Memory (LSS) network. This indicates the speed predicted by the Long Short-Term Memory network model. This represents the weight matrix of the output layer of a Long Short-Term Memory (LSTM) network. This represents the bias term of the output layer of the Long Short-Term Memory network;

[0113] The specific algorithm for the Markov chain is as follows:

[0114] ,

[0115] ,

[0116] in, c k This represents the center velocity value of the discrete state within the vehicle speed range. v min and v max These represent the minimum and maximum values ​​within the vehicle speed range, respectively. k Indicates the overall traffic density. K This represents the number of discrete states within a vehicle speed range. This indicates the vehicle speed predicted based on the Markov chain. This represents the elements of the state transition matrix constructed based on historical data;

[0117] Obtain the vehicle speed prediction values ​​for the Long Short-Term Memory network and the Markov chain, respectively.

[0118] The final vehicle speed prediction is obtained by fusing the vehicle speed prediction values ​​from multi-head attention mechanism, long short-term memory network, and Markov chain. The specific algorithm for obtaining the final vehicle speed prediction is as follows:

[0119] ,

[0120] ,

[0121] ,

[0122] ,

[0123] ,

[0124] in, This represents the hidden state of the Long Short-Term Memory network. This indicates the vehicle speed predicted based on the Markov chain. z t This represents a mixture of hidden states from a Long Short-Term Memory (LSTM) network and vehicle speed predictions based on Markov chains. Indicates the hidden dimension. q t , k t These represent the query vector and the key vector, respectively. W q , W k Represents the learnable projection matrix. d k Represents the attention dimension. Indicates attention weights. This represents the prediction weights of the Long Short-Term Memory network. This indicates the speed predicted by the Long Short-Term Memory network model. This indicates the predicted final vehicle speed.

[0125] Step S03: Optimize energy management based on traffic density data and final vehicle speed prediction to obtain the final energy management strategy;

[0126] It should be noted that in this embodiment, the energy management optimization is based on a reinforcement learning framework, and an intelligent agent is constructed based on traffic flow density data, final vehicle speed prediction and vehicle multi-source state data. The vehicle multi-source state data includes battery SOC state, engine speed, engine torque and required power.

[0127] The specific algorithm for constructing the intelligent agent is as follows:

[0128] ,

[0129] in, Represents an intelligent agent. The agent state variables representing the battery's SOC state. The agent state variable representing engine speed. The agent state quantity representing the engine torque. The agent's state variables representing the required power. The agent state variables representing traffic flow density data This represents the agent's state variables for predicting the final vehicle speed.

[0130] The algorithm for adjusting the engagement ratio of the engine and motor in a hybrid vehicle based on power distribution is as follows:

[0131] ,

[0132] ,

[0133] ,

[0134] in, This indicates the engagement ratio of the engine and electric motor in a hybrid vehicle. Indicates engine power. Indicates motor power. This indicates the power requirement of hybrid vehicles;

[0135] Taking the morning rush hour as an example, please refer to the road map for the morning rush hour. Figure 3 For a comparison of battery SOC under different energy management strategies during morning peak operating conditions, please refer to [link / reference]. Figure 5 For a comparison of fuel consumption under different operating conditions, please refer to [link / reference]. Figure 4 ;

[0136] A reward function is constructed, and the weighting coefficients of fuel consumption reward and electricity consumption reward are dynamically adjusted. Under congested conditions, the weighting coefficient of electricity consumption reward is increased to balance the reward function towards battery SOC. Under free-flowing conditions, the weighting coefficient of fuel consumption reward is increased to balance the reward function towards fuel economy. The specific algorithm for the reward function is as follows:

[0137] ,

[0138] in, Represents the reward function, w 1 and w 2 represents the weighting coefficients for fuel consumption rewards and electricity consumption rewards, respectively. This refers to the engine's instantaneous fuel consumption. Indicates the reference SOC value. Parameters representing the suppression of power jitter. This indicates the power jitter value.

[0139] In summary, the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning improves traffic flow density accuracy by comprehensively perceiving traffic flow density, thereby accurately judging the actual traffic scenario state and enabling efficient energy management and adjustment under different traffic flow density scenarios. Furthermore, the hybrid prediction model for vehicle speed prediction not only effectively captures the long-term dependencies and periodic features in the vehicle speed sequence but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving prediction accuracy. Additionally, the energy management optimization based on a reinforcement learning framework, combined with the vehicle's multi-source states, achieves fuel consumption balance and battery SOC maintenance balance under different operating conditions. Therefore, this invention improves the adaptability and effectiveness of the hybrid vehicle energy management method. Specifically, the process involves acquiring and preprocessing real-time traffic scene images, performing traffic flow density perception processing using a deep learning algorithm to obtain traffic flow density data. This traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception, improving the accuracy of traffic flow density and enabling accurate judgment of the actual traffic scene state. This allows for efficient energy management and adjustment under different traffic flow density scenarios. Vehicle speed prediction is then performed using a hybrid prediction model based on Long Short-Term Memory (LSTM) networks and Markov chains. This model effectively captures not only the long-term dependencies and periodic features in the vehicle speed sequence but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving the accuracy of speed prediction. Finally, energy management optimization is performed based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. This energy management optimization is based on a reinforcement learning framework, combined with the vehicle's multi-source states, to achieve fuel consumption balance and battery SOC maintenance balance under different operating conditions. This invention improves the adaptability and effectiveness of hybrid vehicle energy management methods.

[0140] Please see Figure 2 The diagram shows a schematic representation of the hybrid vehicle energy management system based on traffic flow density perception and reinforcement learning proposed in the second embodiment of the present invention. The system includes:

[0141] The traffic density perception module 10 is used to acquire real-time traffic scene images and perform preprocessing, and to perform traffic density perception processing according to deep learning algorithms to obtain traffic density data. The traffic density perception processing includes visual traffic density perception and speed traffic density perception.

[0142] Vehicle speed prediction module 20 is used to predict vehicle speed according to a hybrid prediction model to obtain a final vehicle speed prediction. The hybrid prediction model is based on a long short-term memory network and a Markov chain.

[0143] The energy management optimization module 30 is used to perform energy management optimization based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. The energy management optimization is based on a reinforcement learning framework.

[0144] The present invention also proposes a computer storage medium storing one or more programs that, when executed by a processor, implement the above-described hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning.

[0145] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to realize the above-mentioned hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning.

[0146] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0147] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0148] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0149] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0150] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning, characterized in that, The application relates to a vehicle energy management method based on real-time traffic scene image processing. The method comprises the following steps: Real-time traffic scene images are acquired and preprocessed, and vehicle flow density sensing processing is performed according to a deep learning algorithm to acquire vehicle flow density data, wherein the vehicle flow density sensing processing comprises visual vehicle flow density sensing and speed vehicle flow density sensing. The vehicle flow density sensing processing according to the deep learning algorithm to acquire the vehicle flow density data specifically comprises the following steps: ， ， ， wherein, represents a total loss of the target detection model, , respectively represent a confidence loss and a positioning loss of the target detection model, p represents a predicted target existence probability, q represents a target intersection over union score, represents a weight balancing positive and negative samples, represents a modulation factor, represents a i th calibrated box value, is a j th anchor box value, n , m respectively represent a total number of the box value and the anchor box value; Vehicle target information is acquired by performing vehicle target detection on the preprocessed real-time traffic scene images according to a target detection model based on the deep learning algorithm, the vehicle target information comprises vehicle target position information and vehicle target serial number information, visual vehicle flow density data is acquired by performing visual vehicle flow density sensing according to the vehicle target information, and a loss function of the target detection model is specifically as follows: Speed vehicle flow density sensing is performed according to vehicle speed information of different lanes. The speed vehicle flow density sensing according to the vehicle speed information of different lanes specifically comprises the following steps: ， wherein, represents the speed flow density data, k j represents the jam density, k m represents the optimal density, k f represents the free flow density, v f represents the free flow speed, v m represents the characteristic speed of high-density congestion flow, v k represents the current vehicle speed, a 1、 a 2 represents the traffic flow density interval coefficient; Speed vehicle flow density data is calculated according to real-time vehicle speed, and a specific algorithm of the speed vehicle flow density data is as follows: ， wherein, k represents the combined traffic density, represents the speed traffic density data, k vision represents the visual traffic density; Comprehensive vehicle flow density data is calculated according to the visual vehicle flow density data and the speed vehicle flow density data, and a specific algorithm of the comprehensive vehicle flow density data is as follows: A vehicle flow state is divided into a free flow state, a transition flow state and a congestion flow state according to the comprehensive vehicle flow density data. Speed prediction is performed according to a hybrid prediction model to acquire final speed prediction, the hybrid prediction model is based on a long short-term memory network and a Markov chain.

2. The traffic flow density perception and reinforcement learning based energy management method for a hybrid vehicle according to claim 1, wherein, Energy management optimization is performed according to the vehicle flow density data and the final speed prediction to acquire a final energy management strategy, the energy management optimization is based on a reinforcement learning framework. The step of acquiring real-time traffic scene images and performing preprocessing specifically comprises the following steps: Real-time traffic scene images are acquired, and lane vehicle distribution images are included in the real-time traffic scene images. ， wherein I gray represents a pixel value of a gray image of a real-time traffic scene image, k r , k g , k b represents a proportion adjustment coefficient, R, G, B represent red, green, and blue channel pixel values of a real-time traffic scene image; Gray processing is performed on the real-time traffic scene images, and a specific algorithm of the gray processing is as follows: ， wherein, Gaussian filtering processing is performed according to a two-dimensional Gaussian filtering function, and a specific algorithm of the Gaussian filtering processing is as follows: denotes a two-dimensional Gaussian filter function, G(x,y) denotes a standard deviation of the two-dimensional Gaussian filter function, sigma , f(x,y) y) denote a gray-scale image of the original real-time traffic scene image and a Gaussian filtered real-time traffic scene image, respectively, I(x, denote a horizontal coordinate and a vertical coordinate of a pixel, respectively. 3.The traffic flow density perception and reinforcement learning based energy management method for hybrid electric vehicle according to claim 1, wherein, x,y The step of performing speed prediction according to a hybrid prediction model to acquire final speed prediction specifically comprises the following steps: Speed prediction is performed according to a hybrid prediction model, the hybrid prediction model is based on a long short-term memory network and a Markov chain, the long short-term memory network is based on time sequence driving, and the Markov chain is based on state driving. ， ， ， wherein, denotes an input feature vector of the long short-term memory network, denotes a vehicle speed at a current time, denotes an acceleration at a current time, denotes a slope angle at a current time, denotes a d-dimensional real vector, denotes a hidden state of the long short-term memory network, denotes a cell state of the long short-term memory network, LSTM denotes a long short-term memory network model, denotes a speed predicted by the long short-term memory network model, denotes a weight matrix of an output layer of the long short-term memory network, denotes a bias term of the output layer of the long short-term memory network; A specific algorithm of the long short-term memory network is as follows: ， ， wherein, c k a center speed value representing a discrete state of a speed interval, v min and v max represent a minimum value and a maximum value of a speed interval, respectively, k represent a comprehensive traffic density, K represent a number of discrete states of a speed interval, represent a speed based on a Markov chain prediction, represent an element of a state transition matrix constructed based on historical data; A specific algorithm of the Markov chain is as follows:

4. The traffic flow density perception and reinforcement learning based energy management method for a hybrid vehicle according to claim 3, wherein, Long short-term memory network speed prediction values and Markov chain speed prediction values are respectively acquired. After the long short-term memory network speed prediction values and the Markov chain speed prediction values are respectively acquired, the following step is further included: ， ， ， ， ， wherein, denotes the hidden state of the long short-term memory network, denotes the vehicle speed predicted based on Markov chain, z t denotes the hybrid feature of the hidden state of the long short-term memory network and the vehicle speed predicted based on Markov chain, denotes the hidden dimension, q t , k t denote the query vector and the key vector, respectively, W q , W k denotes the learnable projection matrix, d k denotes the attention dimension, denotes the attention weight, denotes the long short-term memory network prediction weight, denotes the speed predicted by the long short-term memory network model, denotes the final vehicle speed prediction.

5. The traffic flow density perception and reinforcement learning based energy management method for a hybrid vehicle according to claim 1, wherein, Fusion decision is performed according to a multi-head attention mechanism, the long short-term memory network speed prediction values and the Markov chain speed prediction values to acquire final speed prediction, and a specific algorithm of acquiring the final speed prediction is as follows: The step of performing energy management optimization according to the vehicle flow density data and the final speed prediction to acquire a final energy management strategy specifically comprises the following steps: An agent is constructed according to traffic density data, final vehicle speed prediction and vehicle multi-source state data, the vehicle multi-source state data including battery SOC state, engine speed, engine torque and demand power; The specific algorithm for constructing the agent is as follows: ， wherein represents an agent, represents an agent state quantity for the battery SOC state, represents an agent state quantity for the engine speed, represents an agent state quantity for the engine torque, represents an agent state quantity for the required power, represents an agent state quantity for the traffic density data, represents an agent state quantity for the final speed prediction; The intervention ratio of the engine and the motor of the hybrid vehicle is adjusted based on power distribution, and the specific algorithm for adjusting the intervention ratio of the engine and the motor of the hybrid vehicle is as follows: ， ， ， wherein, represents the intervention ratio of the engine and the motor of the hybrid vehicle, represents the engine power, represents the motor power, represents the required power of the hybrid vehicle; A reward function is constructed, and an oil consumption reward weight coefficient and an electricity consumption reward weight coefficient are dynamically adjusted, the electricity consumption reward weight coefficient is increased in a congestion condition to adjust the reward function to balance the battery SOC, and the oil consumption reward weight coefficient is increased in a smooth condition to adjust the reward function to the fuel economy, and the specific algorithm for the reward function is as follows: ， wherein, represents a reward function, w 1 and w 2 represent an oil consumption reward weight coefficient and an electric consumption reward weight coefficient, respectively, is an engine instantaneous oil consumption, represents a reference SOC value, represents a parameter for suppressing power fluctuation, represents a power fluctuation value.

6. A hybrid vehicle energy management system based on traffic density perception and reinforcement learning, characterized in that, It comprises: A traffic flow density perception module is configured to acquire real-time traffic scene images and perform preprocessing, and to perform traffic flow density perception processing according to a deep learning algorithm to acquire traffic flow density data, the traffic flow density perception processing including visual traffic flow density perception and speed traffic flow density perception; The step of performing traffic flow density perception processing according to a deep learning algorithm to acquire traffic flow density data specifically comprises: A target detection model based on a deep learning algorithm is used to perform vehicle target detection on the preprocessed real-time traffic scene images to acquire vehicle target information, the vehicle target information including vehicle target position information and vehicle target serial number information, and visual traffic flow density perception is performed according to the vehicle target information to acquire visual traffic flow density data, and the loss function of the target detection model is specifically as follows: ， ， ， wherein, represents a total loss of the target detection model, , respectively represent a confidence loss and a positioning loss of the target detection model, p represents a predicted target existence probability, q represents a target intersection over union score, represents a weight balancing positive and negative samples, represents a modulation factor, represents a first i th bounding box value, is a first j th anchor box value, n , m respectively represent a total number of the bounding box value and the anchor box value; Speed traffic flow density perception is performed according to vehicle speed information of different lanes; The step of performing speed traffic flow density perception according to vehicle speed information of different lanes specifically comprises: Speed traffic flow density data is calculated according to real-time vehicle speed, and the specific algorithm for the speed traffic flow density data is as follows: ， wherein, denotes the speed-flow density data, k j denotes the jam density, k m denotes the optimal density, k f denotes the free-flow density, v f denotes the free-flow speed, v m denotes the characteristic speed of the high-density congested flow, v k denotes the current vehicle speed, a 1、 a 2 denotes the flow density interval coefficient; Comprehensive traffic flow density data is calculated according to visual traffic flow density data and speed traffic flow density data, and the specific algorithm for the comprehensive traffic flow density data is as follows: ， wherein, k represents the combined traffic density, represents the speed traffic density data, k vision represents the visual traffic density; The traffic flow state is divided into a free flow state, a transition flow state and a congestion flow state according to the comprehensive traffic flow density data; A vehicle speed prediction module is configured to perform vehicle speed prediction according to a hybrid prediction model to acquire final vehicle speed prediction, the hybrid prediction model being based on a long short-term memory network and a Markov chain; An energy management optimization module is configured to perform energy management optimization according to the traffic flow density data and the final vehicle speed prediction to acquire a final energy management strategy, the energy management optimization being based on a reinforcement learning framework.

7. A storage medium, characterized by The storage medium stores one or more programs, which are executed by the processor to implement the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to any one of claims 1-5.

8. A computer device, comprising: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer programs stored on the memory to implement the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to any one of claims 1-5.

Images