Hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning

Abstract

The invention relates to the field of vehicle intelligent management, and provides a hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning, which improves the traffic flow density precision through comprehensive traffic flow density perception so as to accurately judge the actual traffic scene state and realize efficient energy management and adjustment in traffic scenes with different traffic flow densities. And vehicle speed prediction is carried out through a hybrid prediction model, so that long-term dependence and periodicity characteristics in a vehicle speed sequence are effectively captured, random state transition characteristics and vehicle speed instantaneous changes of the vehicle speed are accurately captured, the vehicle speed prediction precision is further improved, and through energy management optimization based on a reinforcement learning framework, the vehicle speed prediction accuracy is improved. In combination with the multi-source state of the vehicle, fuel consumption balance and battery SOC maintenance balance under different working conditions are achieved, and the adaptability and effectiveness of the hybrid electric vehicle energy management method are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent management of vehicles, and in particular to a hybrid vehicle energy management method based on traffic density perception and reinforcement learning. BACKGROUND

[0002] With the rapid development of new energy vehicle industry, the number of various new energy vehicles is increasing day by day. Among them, hybrid vehicles have unique advantages in terms of endurance mileage and energy efficiency improvement due to the collaborative architecture of "internal combustion engine-motor", and thus play a crucial role in new energy vehicles. With the increasing functional requirements and technical requirements of new energy vehicles, the energy management capability of hybrid vehicles is also gradually improved.

[0003] In the prior art, the traditional energy management method of hybrid vehicles applied to actual traffic scenes has increasingly highlighted the contradiction between control accuracy and working condition adaptability. The existing energy management technology system often has three dimensional problems. First, the existing energy management method based on experience-driven rule control is difficult to adapt to nonlinear driving demand under urban working conditions due to the static threshold setting mechanism. Second, the optimization algorithm based on dynamic programming and model predictive control is limited by the vehicle speed prediction model under limited vision (especially without coupling real-time traffic density parameters), and the optimization effect on actual roads is significantly attenuated. Third, the machine learning method represented by long short-term memory network and reinforcement learning generally has the problem of insufficient fusion of multi-source traffic information, resulting in missing environmental perception dimension of the decision model.

[0004] Therefore, how to design a hybrid vehicle energy management method to improve adaptability and stability in actual traffic scenes, so as to obtain more reasonable optimization that meets actual demand, has become a problem to be solved. SUMMARY

[0005] Therefore, how to design a hybrid vehicle energy management method to improve adaptability and stability in actual traffic scenes, so as to obtain more reasonable optimization that meets actual demand, has become a problem to be solved.

[0006] The hybrid vehicle energy management method based on traffic density perception and reinforcement learning provided by the present application comprises: acquire real-time traffic scene images and pre-process them, perform traffic flow density perception processing according to a deep learning algorithm to obtain traffic flow density data, the traffic flow density perception processing including visual traffic flow density perception and speed traffic flow density perception; perform speed prediction according to a hybrid prediction model to obtain final speed prediction, the hybrid prediction model being based on a long short-term memory network and a Markov chain; perform energy management optimization according to the traffic flow density data and the final speed prediction to obtain a final energy management strategy, the energy management optimization being based on a reinforcement learning framework.

[0007] In summary, according to the above-mentioned hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning, the traffic flow density accuracy is improved through comprehensive traffic flow density perception to accurately determine the actual traffic scene state, so as to efficiently regulate energy management in different traffic flow density traffic scenes. The hybrid prediction model is used for speed prediction, which not only effectively captures the long-term dependence and periodic characteristics in the speed sequence, but also accurately captures the random state transition characteristics and speed instantaneous changes of the speed, further improving the speed prediction accuracy. The energy management optimization based on the reinforcement learning framework combines the multi-source state of the vehicle to balance the fuel consumption and maintain the balance of the battery SOC under different working conditions, thereby improving the adaptability and effectiveness of the hybrid vehicle energy management method. Specifically, real-time traffic scene images are acquired and pre-processed, traffic flow density perception processing is performed according to a deep learning algorithm to obtain traffic flow density data, the traffic flow density perception processing including visual traffic flow density perception and speed traffic flow density perception, which improves the traffic flow density accuracy to accurately determine the actual traffic scene state, so as to efficiently regulate energy management in different traffic flow density traffic scenes. A hybrid prediction model is used for speed prediction to obtain final speed prediction, the hybrid prediction model being based on a long short-term memory network and a Markov chain, which not only effectively captures the long-term dependence and periodic characteristics in the speed sequence, but also accurately captures the random state transition characteristics and speed instantaneous changes of the speed, further improving the speed prediction accuracy. Energy management optimization is performed according to the traffic flow density data and the final speed prediction to obtain a final energy management strategy, the energy management optimization being based on a reinforcement learning framework, which combines the multi-source state of the vehicle to balance the fuel consumption and maintain the balance of the battery SOC under different working conditions, thereby improving the adaptability and effectiveness of the hybrid vehicle energy management method.

[0008] Further, the step of acquiring real-time traffic scene images and pre-processing them specifically includes: acquiring real-time traffic scene images, the real-time traffic scene images including lane vehicle distribution images; The real-time traffic scene image is converted to grayscale. The specific algorithm for grayscale conversion is as follows: , in, I gray This represents the grayscale image pixel values ​​of a real-time traffic scene image. k r , k g , k b R represents the scaling factor, and G and B represent the red, green, and blue channel pixel values ​​of the real-time traffic scene image. Gaussian filtering is performed using a two-dimensional Gaussian filtering function. The specific algorithm for Gaussian filtering is as follows: , in, G(x,y) This represents a two-dimensional Gaussian filter function. σ This represents the standard deviation of a two-dimensional Gaussian filter function. f(x,y) , I(x,y) These represent the original grayscale image of the real-time traffic scene and the real-time traffic scene image after Gaussian filtering, respectively. x,y These represent the x and y coordinates of a pixel, respectively.

[0009] Furthermore, the step of performing traffic flow density perception processing based on deep learning algorithms to obtain traffic flow density data specifically includes: Vehicle targets are detected in preprocessed real-time traffic scene images using a target detection model based on a deep learning algorithm to obtain vehicle target information, including vehicle target location information and vehicle target sequence information. Visual traffic density is then perceived based on this vehicle target information to obtain visual traffic density data. The loss function of the target detection model is as follows: , , , in, This represents the total loss of the object detection model. , Let represent the confidence loss and localization loss of the object detection model, respectively. p This indicates the probability that the target exists. q This indicates the score of the intersection and union of objectives. This indicates the weights for balancing positive and negative samples. Indicates the modulation factor. Indicates the first i A specified box value, For the firstj an anchor frame value, n , m respectively represent the total number of frame values and anchor frame values; According to the speed information of different lanes, speed and traffic density perception is performed.

[0010] Further, the step of performing speed and traffic density perception according to the speed information of different lanes specifically includes: According to the real-time vehicle speed, speed and traffic density data is calculated, and the specific algorithm of the speed and traffic density data is as follows: , wherein, denotes the speed and traffic density data, k j denotes the jam density, k m denotes the optimal density, k f denotes the free flow density, v f denotes the free flow speed, v m denotes the characteristic speed of high-density congestion flow, v k denotes the current vehicle speed, a 1, a 2 denotes the traffic density interval coefficient; According to the visual traffic density data and the speed and traffic density data, comprehensive traffic density data is calculated, and the specific algorithm of the comprehensive traffic density data is as follows: , wherein, k denotes the comprehensive traffic density, denotes the speed and traffic density data, k vision denotes the visual traffic density; According to the comprehensive traffic density data, the traffic state is divided into free flow state, transition flow state, and congestion flow state.

[0011] Further, the step of performing speed prediction according to the hybrid prediction model to obtain the final speed prediction specifically includes: According to the hybrid prediction model, speed prediction is performed, and the hybrid prediction model is based on a long short-term memory network and a Markov chain, the long short-term memory network is based on time sequence driving, and the Markov chain is based on state driving; The specific algorithm of the long short-term memory network is as follows: , , , wherein, represents an input feature vector of the long short-term memory network, represents a vehicle speed at a current time, represents an acceleration at a current time, represents a slope angle at a current time, represents a d-dimensional real vector, represents a hidden state of the long short-term memory network, represents a cell state of the long short-term memory network, LSTM represents a long short-term memory network model, represents a speed predicted by the long short-term memory network model, represents a weight matrix of an output layer of the long short-term memory network, represents a bias term of the output layer of the long short-term memory network; The specific algorithm of the Markov chain is as follows: , , wherein, c k represents a center speed value of a discrete state of a vehicle speed interval, v min and v max respectively represent a minimum value and a maximum value of a vehicle speed interval, k represents a comprehensive traffic density, K represents a number of discrete states of a vehicle speed interval, represents a vehicle speed predicted based on the Markov chain, represents an element of a state transition matrix constructed based on historical data; The long short-term memory network vehicle speed prediction value and the Markov chain vehicle speed prediction value are obtained respectively.

[0012] Further, the step of obtaining the long short-term memory network vehicle speed prediction value and the Markov chain vehicle speed prediction value respectively further comprises: According to the multi-head attention mechanism, the long short-term memory network vehicle speed prediction value and the Markov chain vehicle speed prediction value, a fusion decision is made to obtain a final vehicle speed prediction, and the specific algorithm for obtaining the final vehicle speed prediction is as follows: , , , , , wherein, denotes the hidden state of the long short-term memory network, denotes the vehicle speed predicted based on Markov chain, z t denotes the hybrid feature of the hidden state of the long short-term memory network and the vehicle speed predicted based on Markov chain, denotes the hidden dimension, q t , k t denotes the query vector and the key vector, respectively, W q , W k denotes the learnable projection matrix, d k denotes the attention dimension, denotes the attention weight, denotes the long short-term memory network prediction weight, denotes the speed predicted by the long short-term memory network model, denotes the final vehicle speed prediction.

[0013] Further, the step of performing energy management optimization according to the traffic density data and the final vehicle speed prediction to obtain a final energy management strategy specifically comprises: constructing an agent according to the traffic density data, the final vehicle speed prediction and vehicle multi-source state data, wherein the vehicle multi-source state data includes battery SOC state, engine speed, engine torque, demand power; The specific algorithm for constructing the agent is as follows: , wherein, denotes the agent, denotes the agent state quantity of the battery SOC state, denotes the agent state quantity of the engine speed, denotes the agent state quantity of the engine torque, denotes the agent state quantity of the demand power, denotes the agent state quantity of the traffic density data, denotes the agent state quantity of the final vehicle speed prediction; adjusting the intervention ratio of the engine and the motor of the hybrid vehicle based on the power distribution, and the specific algorithm for adjusting the intervention ratio of the engine and the motor of the hybrid vehicle is as follows: , , , wherein, a ratio of intervention of an engine and a motor of the hybrid vehicle, an engine power, a motor power, a required power of the hybrid vehicle; constructing a reward function, and dynamically adjusting a fuel consumption reward weight coefficient and an electricity consumption reward weight coefficient, increasing the electricity consumption reward weight coefficient in a congestion condition to adjust the reward function to balance the battery SOC, and increasing the fuel consumption reward weight coefficient in a smooth condition to adjust the reward function to be biased towards fuel economy, a specific algorithm of the reward function being as follows: , wherein, a reward function, w 1 and w 2 respectively represent a fuel consumption reward weight coefficient and an electricity consumption reward weight coefficient, an instantaneous fuel consumption of the engine, a reference SOC value, a parameter for suppressing power jitter, a power jitter value.

[0014] The hybrid vehicle energy management system based on traffic density perception and reinforcement learning provided by the application comprises: a traffic density perception module, configured to acquire real-time traffic scene images and perform preprocessing, and perform traffic density perception processing according to a deep learning algorithm to acquire traffic density data, wherein the traffic density perception processing comprises visual traffic density perception and speed traffic density perception; a vehicle speed prediction module, configured to perform vehicle speed prediction according to a hybrid prediction model to acquire a final vehicle speed prediction, wherein the hybrid prediction model is based on a long short-term memory network and a Markov chain; an energy management optimization module, configured to perform energy management optimization according to the traffic density data and the final vehicle speed prediction to acquire a final energy management strategy, wherein the energy management optimization is based on a reinforcement learning framework.

[0015] The application further provides a storage medium storing one or more programs, and the programs are executed by a processor to implement the hybrid vehicle energy management method based on traffic density perception and reinforcement learning.

[0016] The application further provides a computer device comprising a memory and a processor, wherein: the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory to implement the hybrid vehicle energy management method based on traffic density perception and reinforcement learning. Attached Figure Description

[0017] Figure 1 This is a flowchart of the hybrid vehicle energy management method based on traffic density perception and reinforcement learning proposed in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the hybrid vehicle energy management system based on traffic flow density perception and reinforcement learning proposed in the second embodiment of the present invention. Figure 3 This is a morning rush hour road map according to the first embodiment of the present invention; Figure 4 This is a comparison chart of fuel consumption under different operating conditions according to the first embodiment of the present invention; Figure 5 This is a comparison chart of battery SOC under the morning peak operating conditions according to the first embodiment of the present invention.

[0018] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0020] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] Please see Figure 1 The diagram shows a flowchart of the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning proposed in the first embodiment of the present invention. This hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning includes steps S01 to S03, wherein: Step S01: Acquire real-time traffic scene images and perform preprocessing; use deep learning algorithms to perform traffic flow density perception processing to obtain traffic flow density data. It should be noted that in this embodiment, the traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception, and acquires a real-time traffic scene image, which includes a lane vehicle distribution image. The real-time traffic scene image is converted to grayscale. The specific algorithm for grayscale conversion is as follows: , in, I gray This represents the grayscale image pixel values ​​of a real-time traffic scene image. k r , k g , k b R represents the scaling factor, and G and B represent the red, green, and blue channel pixel values ​​of the real-time traffic scene image. Gaussian filtering is performed using a two-dimensional Gaussian filtering function. The specific algorithm for Gaussian filtering is as follows: , in, G(x,y) This represents a two-dimensional Gaussian filter function. σ This represents the standard deviation of a two-dimensional Gaussian filter function. f(x,y) , I(x,y) These represent the original grayscale image of the real-time traffic scene and the real-time traffic scene image after Gaussian filtering, respectively. x,y These represent the x and y coordinates of a pixel, respectively. Vehicle targets are detected in preprocessed real-time traffic scene images using a target detection model based on a deep learning algorithm to obtain vehicle target information, including vehicle target location information and vehicle target sequence information. Visual traffic density is then perceived based on this vehicle target information to obtain visual traffic density data. The loss function of the target detection model is as follows: , , , in, This represents the total loss of the object detection model. , Let represent the confidence loss and localization loss of the object detection model, respectively. p This indicates the probability that the target exists. q This indicates the score of the intersection and union of objectives. This indicates the weights for balancing positive and negative samples. Indicates the modulation factor. Indicates the first i A specified box value, For the first j Anchor box values, n , m These represent the total number of box values ​​and anchor box values, respectively; Speed ​​and traffic density are perceived based on vehicle speed information in different lanes. The speed-vehicle flow density data is calculated based on the real-time vehicle speed. The specific algorithm for calculating the speed-vehicle flow density data is as follows: , in, Represents speed and traffic density data. k j Indicates blockage density, k m Indicates the optimal density. k f Represents free flow density, v f Indicates the free flow velocity. v m The characteristic velocity representing high-density congestion flow, v k Indicates the current vehicle speed. a 1. a 2 represents the traffic density interval coefficient; The comprehensive traffic flow density data is calculated based on visual traffic flow density data and speed traffic flow density data. The specific algorithm for calculating the comprehensive traffic flow density data is as follows: , in, k Indicates the overall traffic density. Represents speed and traffic density data. k vision Indicates visual traffic density; Based on comprehensive traffic density data, traffic flow status can be divided into free flow, transitional flow, and congested flow.

[0023] Step S02: Predict vehicle speed based on the hybrid prediction model to obtain the final vehicle speed prediction; It should be noted that in this embodiment, the hybrid prediction model is based on a long short-term memory network and a Markov chain. The vehicle speed is predicted according to the hybrid prediction model. The hybrid prediction model is based on a long short-term memory network and a Markov chain. The long short-term memory network is based on time-driven and the Markov chain is based on state-driven. The specific algorithm for the Long Short-Term Memory network is as follows: , , , in, This represents the input feature vector of the Long Short-Term Memory network. This indicates the vehicle speed at the current moment. This represents the acceleration at the current moment. This indicates the slope angle at the current moment. Represents a d-dimensional real vector. This represents the hidden state of the Long Short-Term Memory network. LSTM represents the cell state of a Long Short-Term Memory (LSS) network. This indicates the speed predicted by the Long Short-Term Memory network model. This represents the weight matrix of the output layer of a Long Short-Term Memory (LSTM) network. This represents the bias term of the output layer of the Long Short-Term Memory network; The specific algorithm for the Markov chain is as follows: , , in, c k This represents the center velocity value of the discrete state within the vehicle speed range. v min and v max These represent the minimum and maximum values ​​within the vehicle speed range, respectively. k Indicates the overall traffic density. K This represents the number of discrete states within a vehicle speed range. This indicates the vehicle speed predicted based on the Markov chain. This represents the elements of the state transition matrix constructed based on historical data; Obtain the vehicle speed prediction values ​​for the Long Short-Term Memory network and the Markov chain, respectively. The final vehicle speed prediction is obtained by fusing the vehicle speed prediction values ​​from multi-head attention mechanism, long short-term memory network, and Markov chain. The specific algorithm for obtaining the final vehicle speed prediction is as follows: , , , , , in, This represents the hidden state of the Long Short-Term Memory network. This indicates the vehicle speed predicted based on the Markov chain. z t This represents a mixture of hidden states from a Long Short-Term Memory (LSTM) network and vehicle speed predictions based on Markov chains. Indicates the hidden dimension. q t , k t These represent the query vector and the key vector, respectively. W q , W k Represents the learnable projection matrix. d k Represents the attention dimension. Indicates attention weights. This represents the prediction weights of the Long Short-Term Memory network. This indicates the speed predicted by the Long Short-Term Memory network model. This indicates the predicted final vehicle speed.

[0024] Step S03: Optimize energy management based on traffic density data and final vehicle speed prediction to obtain the final energy management strategy; It should be noted that in this embodiment, the energy management optimization is based on a reinforcement learning framework, and an intelligent agent is constructed based on traffic flow density data, final vehicle speed prediction and vehicle multi-source state data. The vehicle multi-source state data includes battery SOC state, engine speed, engine torque and required power. The specific algorithm for constructing the intelligent agent is as follows: , in, Represents an intelligent agent, The agent state variables representing the battery's SOC state. The agent state variable representing engine speed. The agent state quantity representing the engine torque. The agent's state variables representing the required power. The agent state variables representing traffic flow density data This represents the agent's state variables for predicting the final vehicle speed. The algorithm for adjusting the engagement ratio of the engine and motor in a hybrid vehicle based on power distribution is as follows: , , , in, This indicates the engagement ratio of the engine and electric motor in a hybrid vehicle. Indicates engine power. Indicates motor power. This indicates the power requirement of hybrid vehicles; Taking the morning rush hour as an example, please refer to the road map for the morning rush hour. Figure 3 For a comparison of battery SOC under different energy management strategies during morning peak operating conditions, please refer to [link / reference]. Figure 5 For a comparison of fuel consumption under different operating conditions, please refer to [link / reference]. Figure 4 ; A reward function is constructed, and the weighting coefficients of fuel consumption reward and electricity consumption reward are dynamically adjusted. Under congested conditions, the weighting coefficient of electricity consumption reward is increased to balance the reward function towards battery SOC. Under free-flowing conditions, the weighting coefficient of fuel consumption reward is increased to balance the reward function towards fuel economy. The specific algorithm for the reward function is as follows: , in, Represents the reward function, w 1 and w 2 represents the weighting coefficients for fuel consumption rewards and electricity consumption rewards, respectively. This refers to the engine's instantaneous fuel consumption. Indicates the reference SOC value. Parameters representing the suppression of power jitter. This indicates the power jitter value.

[0025] In summary, the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning improves traffic flow density accuracy by comprehensively perceiving traffic flow density, thereby accurately judging the actual traffic scenario state and enabling efficient energy management and adjustment under different traffic flow density scenarios. Furthermore, the hybrid prediction model for vehicle speed prediction not only effectively captures the long-term dependencies and periodic features in the vehicle speed sequence but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving prediction accuracy. Additionally, the energy management optimization based on a reinforcement learning framework, combined with the vehicle's multi-source states, achieves fuel consumption balance and battery SOC maintenance balance under different operating conditions. Therefore, this invention improves the adaptability and effectiveness of the hybrid vehicle energy management method. Specifically, the process involves acquiring and preprocessing real-time traffic scene images, performing traffic flow density perception processing using a deep learning algorithm to obtain traffic flow density data. This traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception, improving the accuracy of traffic flow density and enabling accurate judgment of the actual traffic scene state. This allows for efficient energy management and adjustment under different traffic flow density scenarios. Vehicle speed prediction is then performed using a hybrid prediction model based on Long Short-Term Memory (LSTM) networks and Markov chains. This model effectively captures not only the long-term dependencies and periodic features in the vehicle speed sequence but also accurately captures the random state transition characteristics and instantaneous changes in vehicle speed, further improving the accuracy of speed prediction. Finally, energy management optimization is performed based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. This energy management optimization is based on a reinforcement learning framework, combined with the vehicle's multi-source states, to achieve fuel consumption balance and battery SOC maintenance balance under different operating conditions. This invention improves the adaptability and effectiveness of hybrid vehicle energy management methods.

[0026] Please see Figure 2 The diagram shows a schematic representation of the hybrid vehicle energy management system based on traffic flow density perception and reinforcement learning proposed in the second embodiment of the present invention. The system includes: The traffic density perception module 10 is used to acquire real-time traffic scene images and perform preprocessing, and to perform traffic density perception processing according to deep learning algorithms to obtain traffic density data. The traffic density perception processing includes visual traffic density perception and speed traffic density perception. Vehicle speed prediction module 20 is used to predict vehicle speed according to a hybrid prediction model to obtain a final vehicle speed prediction. The hybrid prediction model is based on a long short-term memory network and a Markov chain. The energy management optimization module 30 is used to perform energy management optimization based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. The energy management optimization is based on a reinforcement learning framework.

[0027] The present invention also proposes a computer storage medium storing one or more programs that, when executed by a processor, implement the above-described hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning.

[0028] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to realize the above-mentioned hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning.

[0029] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0030] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0031] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0032] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0033] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning, characterized in that, include: Real-time traffic scene images are acquired and preprocessed, and traffic flow density perception processing is performed according to a deep learning algorithm to obtain traffic flow density data. The traffic flow density perception processing includes visual traffic flow density perception and speed traffic flow density perception. Vehicle speed is predicted according to a hybrid prediction model, which is based on a long short-term memory network and a Markov chain; Energy management optimization is performed based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. The energy management optimization is based on a reinforcement learning framework.

2. The hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to claim 1, characterized in that, The steps of acquiring and preprocessing real-time traffic scene images specifically include: Acquire real-time traffic scene images, which include lane vehicle distribution images; The real-time traffic scene image is converted to grayscale. The specific algorithm for grayscale conversion is as follows: ， in, I gray This represents the grayscale image pixel values ​​of a real-time traffic scene image. k r , k g , k b R represents the scaling factor, and G and B represent the red, green, and blue channel pixel values ​​of the real-time traffic scene image. Gaussian filtering is performed using a two-dimensional Gaussian filtering function. The specific algorithm for Gaussian filtering is as follows: ， in, G(x,y) This represents a two-dimensional Gaussian filter function. σ This represents the standard deviation of a two-dimensional Gaussian filter function. f(x,y) , I(x, y) These represent the original grayscale image of the real-time traffic scene and the real-time traffic scene image after Gaussian filtering, respectively. x,y These represent the x and y coordinates of a pixel, respectively.

3. The hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to claim 1, characterized in that, The step of performing traffic flow density perception processing based on deep learning algorithms to obtain traffic flow density data specifically includes: Vehicle targets are detected in preprocessed real-time traffic scene images using a target detection model based on a deep learning algorithm to obtain vehicle target information, including vehicle target location information and vehicle target sequence information. Visual traffic density is then perceived based on this vehicle target information to obtain visual traffic density data. The loss function of the target detection model is as follows: ， ， ， in, This represents the total loss of the object detection model. , Let represent the confidence loss and localization loss of the object detection model, respectively. p This indicates the probability that the target exists. q This indicates the score of the intersection and union of objectives. This indicates the weights for balancing positive and negative samples. Indicates the modulation factor. Indicates the first i A specified box value, For the first j Anchor box values, n , m These represent the total number of box values ​​and anchor box values, respectively; Speed ​​and traffic density are perceived based on vehicle speed information in different lanes.

4. The hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to claim 3, characterized in that, The step of sensing speed and traffic density based on vehicle speed information of different lanes specifically includes: The speed-vehicle flow density data is calculated based on the real-time vehicle speed. The specific algorithm for calculating the speed-vehicle flow density data is as follows: ， in, Represents speed and traffic density data. k j Indicates blockage density, k m Indicates the optimal density. k f Represents free flow density. v f Indicates the free flow velocity. v m The characteristic velocity representing high-density congestion flow, v k Indicates the current vehicle speed. a 1. a 2 represents the traffic density interval coefficient; The comprehensive traffic flow density data is calculated based on visual traffic flow density data and speed traffic flow density data. The specific algorithm for calculating the comprehensive traffic flow density data is as follows: ， in, k Indicates the overall traffic density. Represents speed and traffic density data. k vision Indicates visual traffic density; Based on comprehensive traffic density data, traffic flow status can be divided into free flow, transitional flow, and congested flow.

5. The hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to claim 1, characterized in that, The step of predicting vehicle speed based on a hybrid prediction model to obtain the final vehicle speed prediction specifically includes: Vehicle speed is predicted based on a hybrid prediction model, which is based on a long short-term memory network and a Markov chain. The long short-term memory network is time-driven, and the Markov chain is state-driven. The specific algorithm for the Long Short-Term Memory network is as follows: ， ， ， in, This represents the input feature vector of the Long Short-Term Memory network. This indicates the vehicle speed at the current moment. This represents the acceleration at the current moment. This indicates the slope angle at the current moment. Represents a d-dimensional real vector. This represents the hidden state of the Long Short-Term Memory network. LSTM represents the cell state of a Long Short-Term Memory (LSS) network. This indicates the speed predicted by the Long Short-Term Memory network model. This represents the weight matrix of the output layer of a Long Short-Term Memory (LSTM) network. This represents the bias term of the output layer of the Long Short-Term Memory network; The specific algorithm for the Markov chain is as follows: ， ， in, c k This represents the center velocity value of the discrete state within the vehicle speed range. v min and v max These represent the minimum and maximum values ​​within the vehicle speed range, respectively. k Indicates the overall traffic density. K This represents the number of discrete states within a vehicle speed range. This indicates the vehicle speed predicted based on the Markov chain. This represents the elements of the state transition matrix constructed based on historical data; Obtain the vehicle speed prediction values ​​for the Long Short-Term Memory network and the Markov chain, respectively.

6. The hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to claim 5, characterized in that, The steps of obtaining the vehicle speed prediction values ​​of the Long Short-Term Memory network and the Markov chain respectively are followed by: The final vehicle speed prediction is obtained by fusing the vehicle speed prediction values ​​from multi-head attention mechanism, long short-term memory network, and Markov chain. The specific algorithm for obtaining the final vehicle speed prediction is as follows: ， ， ， ， ， in, This represents the hidden state of the Long Short-Term Memory network. This indicates the vehicle speed predicted based on the Markov chain. z t This represents a mixture of hidden states from a Long Short-Term Memory (LSTM) network and vehicle speed predictions based on Markov chains. Indicates the hidden dimension. q t , k t These represent the query vector and the key vector, respectively. W q , W k Represents the learnable projection matrix. d k Represents the attention dimension. Indicates attention weights. This represents the prediction weights of the Long Short-Term Memory network. This indicates the speed predicted by the Long Short-Term Memory network model. This indicates the predicted final vehicle speed.

7. The hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning according to claim 1, characterized in that, The step of optimizing energy management based on the traffic flow density data and the final vehicle speed prediction to obtain the final energy management strategy specifically includes: An intelligent agent is constructed based on traffic flow density data, final vehicle speed prediction, and multi-source vehicle state data, including battery SOC status, engine speed, engine torque, and required power. The specific algorithm for constructing the intelligent agent is as follows: ， in, Represents an intelligent agent. The agent state variables representing the battery's SOC state. The agent state variable representing engine speed. The agent state quantity representing the engine torque. The agent's state variables representing the required power. The agent state variables representing traffic flow density data This represents the agent's state variables for predicting the final vehicle speed. The algorithm for adjusting the engagement ratio of the engine and motor in a hybrid vehicle based on power distribution is as follows: ， ， ， in, This indicates the engagement ratio of the engine and electric motor in a hybrid vehicle. Indicates engine power. Indicates motor power. This indicates the power requirement of hybrid vehicles; A reward function is constructed, and the weighting coefficients of fuel consumption reward and electricity consumption reward are dynamically adjusted. Under congested conditions, the weighting coefficient of electricity consumption reward is increased to balance the reward function towards battery SOC. Under free-flowing conditions, the weighting coefficient of fuel consumption reward is increased to balance the reward function towards fuel economy. The specific algorithm for the reward function is as follows: ， in, Represents the reward function, w 1 and w 2 represents the weighting coefficients for fuel consumption rewards and electricity consumption rewards, respectively. This refers to the engine's instantaneous fuel consumption. Indicates the reference SOC value. Parameters representing the suppression of power jitter. This indicates the power jitter value.

8. A hybrid vehicle energy management system based on traffic flow density perception and reinforcement learning, characterized in that, include: The traffic density perception module is used to acquire real-time traffic scene images and perform preprocessing, and to perform traffic density perception processing according to deep learning algorithms to obtain traffic density data. The traffic density perception processing includes visual traffic density perception and speed traffic density perception. The vehicle speed prediction module is used to predict vehicle speed according to a hybrid prediction model to obtain the final vehicle speed prediction. The hybrid prediction model is based on a long short-term memory network and a Markov chain. An energy management optimization module is used to perform energy management optimization based on the traffic flow density data and the final vehicle speed prediction to obtain a final energy management strategy. The energy management optimization is based on a reinforcement learning framework.

9. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning as described in any one of claims 1-7.

10. A computer device, characterized in that, The computer device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the hybrid vehicle energy management method based on traffic flow density perception and reinforcement learning as described in any one of claims 1-7.

Images