Connected hybrid electric vehicle energy management method fusing multi-vehicle motion interaction perception

Abstract

The application discloses a kind of fusion multi-vehicle motion interaction perception's network connection hybrid electric vehicle energy management method, which collects the time window in the multi-vehicle system time sequence driving information in the self-perception range, constructs time sequence driving information encoder based on LSTM network, and carries out feature coding to each vehicle time sequence driving information, obtains time sequence driving information feature vector;Combined with complete undirected graph, the multi-vehicle interaction relationship topological graph in the self-perception range is established;The multi-vehicle interaction relationship topological graph is cut, and a multi-vehicle interaction perception model, hybrid electric vehicle "driving behavior-comprehensive energy consumption" simulation model is constructed;Based on the above model, the energy management strategy based on maximum entropy deep reinforcement learning is obtained;And initialize "driving behavior-comprehensive energy consumption" simulation model, energy management strategy model and its evaluation model, train the energy management strategy model of fusion multi-vehicle interaction perception by reinforcement learning SoftActor-Critc algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile energy management, in particular to a networked hybrid electric vehicle energy management method fusing multi-vehicle motion interaction perception. BACKGROUND

[0002] Hybrid electric vehicles, with the advantages of energy saving, environmental protection, and long endurance, are one of the mainstream technical solutions for new energy vehicles at present. How to ensure the real-time collaboration and efficient work of multi-power sources of the hybrid power system through energy management is the key to realizing energy saving and performance improvement. At present, with the improvement of vehicle computing power and the rapid development of artificial intelligence technology, intelligent optimization algorithms such as deep reinforcement learning have been gradually applied in hybrid electric vehicle energy management to solve the problems of weak working condition adaptability and insufficient optimality of traditional energy management algorithms. However, urban road traffic often exhibits characteristics such as dense traffic flow and variable driving conditions, which brings great challenges to real-time and dynamic hybrid power system energy management, for example: (1) Multi-vehicle interaction in dynamic and dense urban roads makes the hybrid electric vehicle driving scene more complex, with high uncertainty of driving conditions, which leads to the need to improve the stability of energy management optimization effect; (2) Traditional optimization type energy management strategies are difficult to establish a mapping relationship between complex multi-vehicle motion interaction and efficient energy management strategies, and the utilization degree of networked traffic information by the energy management strategy needs to be improved; (3) Ordinary feedforward neural network type intelligent energy management strategies usually only support fixed dimension state information input, which is difficult to adapt to the uncertain number of multi-vehicle systems and uncertain vehicle-vehicle relative motion relationship. Therefore, for hybrid electric vehicles, how to design an energy management strategy with multi-vehicle motion interaction perception capability, strong adaptability to high dynamic driving conditions on urban roads, and stable optimization performance is a technical problem that needs to be solved to realize deep energy saving of networked hybrid electric vehicles. SUMMARY

[0003] The present application proposes a networked hybrid electric vehicle energy management method fusing multi-vehicle motion interaction perception to solve the technical problems in the background art.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] A networked hybrid electric vehicle energy management method fusing multi-vehicle motion interaction perception, comprising the following steps:

[0006] S100, within the self-vehicle perception range, collect multi-vehicle system time series driving information within a certain time window, construct a time series driving information encoder based on an LSTM network, and encode the time series driving information of each vehicle to output its time series driving information feature vector;

[0007] S200, let the time sequence driving information feature vector be a node, and combine the complete undirected graph to establish a multi-vehicle interaction relationship topology graph in the self-vehicle perception range;

[0008] S300, the multi-vehicle interaction relationship topology graph is cropped, and a multi-vehicle interaction perception model is constructed;

[0009] S400, a hybrid electric vehicle driving behavior-comprehensive energy consumption simulation model is constructed;

[0010] S500, based on the multi-vehicle interaction perception model, the driving behavior-comprehensive energy consumption simulation model, an energy management strategy model based on maximum entropy deep reinforcement learning is constructed;

[0011] S600, based on the reinforcement learning SoftActor-Critc algorithm, the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model are initialized, and a final energy management strategy model is obtained by training fusion.

[0012] Further, the method for constructing the time sequence driving information encoder of the LSTM network in step S100 of the present application comprises:

[0013] The time sequence driving information observation of each vehicle is encoded:

[0014]

[0015] Wherein, And The LSTM network unit hidden state and unit state, The information observation.

[0016] Further, the method for constructing the multi-vehicle interaction relationship topology graph in the self-vehicle perception range in step S200 of the present application is:

[0017] Each vehicle in the multi-vehicle system is modeled as a node of the interaction relationship topology graph, and the feature encoding vector of the time sequence driving information observation of each vehicle node is taken as its attribute vector;

[0018] The interaction relationship between the vehicles in the multi-vehicle system is modeled as an edge of the interaction relationship topology graph, and an edge vector is defined to describe the importance of the interaction relationship between the vehicles; the complete undirected graph is used to model the interaction relationship of the multi-vehicle system in the self-vehicle perception range, and the key elements of the model are defined as follows:

[0019] (1) The node of the interaction relationship topology graph and the attribute vector of the node:

[0020] The vehicle i (i=1, 2, …, N) is defined as the node of the graph network G, and the feature encoding vector of the time sequence driving information observation is defined as the attribute vector of the node

[0021] The set of all vehicle node attributes is denoted as The graph with the i-th vehicle as the central node is denoted as G. i ;

[0022] (2) Edges and edge vectors of the interaction topology graph:

[0023] The interactions between vehicles are represented as edges in graph G, E = {w1, w2, ..., w N Let} be the set of edges, and let w be the edge vector. i Used to describe the importance of the interaction between vehicle i and other vehicles.

[0024] Furthermore, the method for constructing a multi-vehicle interaction perception model in step S300 of the present invention includes:

[0025] Constructing a self-attention network to compute the importance of multi-vehicle interaction relationships;

[0026] Hard attention and reparameterization methods are used to prune the multi-vehicle interaction relationship topology graph to obtain a reduced multi-vehicle interaction relationship subgraph.

[0027] A soft self-attention method is used to extract the motion interaction relationship representations of other vehicles related to each node's vehicle.

[0028] The multi-vehicle interaction perception model also includes:

[0029] (1) LSTM encoding layer: The temporal driving information encoder established in step S100, whose input is the attribute vectors of the multi-vehicle interaction relationship topology graph. The output is Encoder parameters are shared among multiple vehicles;

[0030] (2) A stochastic hard self-attention layer, which is a vector composed of the set of node attributes in graph G. The input sequence is a sequence whose length N is determined by the number of nodes in the real-time multi-vehicle system rather than being a fixed value; the output is the attention score matrix A. attn Value matrix V, binarized attention score matrix A hard ;

[0031] Using learnable weight matrices Generate query matrix Q qry =W q X, bond matrix K = W k X, Value matrix V = W v X, calculates the attention score matrix A through dot product. attn and A′ attn :

[0032]

[0033] where softmax(·) normalizes by column, A attn and A' attn is used to describe the importance of the interaction between each vehicle within the perception range of the ego vehicle, A' attn is the edge set of the multi-vehicle interaction topology graph established in step one.

[0034] In this layer, the binary attention score matrix A hard between vehicles is learned through hard self-attention, the multi-vehicle interaction topology graph is pruned to extract effective vehicle-vehicle interaction relationships, and the Gumble-Softmax reparameterization method is introduced to obtain a random sampling output A hard under the differentiable premise.

[0035]

[0036] where gum(·) is the Gumble-Softmax function mapped by column, and the input is a column vector, and its output formula is as follows

[0037]

[0038] where a i is the attention score, τ is the temperature coefficient, and Gumbel noise term g i = -log(-log(u i )), u i ~ U(0, 1).

[0039] (3) Deterministic soft self-attention layer, the input is A attn , A hard , and the value matrix V, and the output is a multi-vehicle motion interaction representation.

[0040] First, according to the attention weight A hard , remove the irrelevant vehicle nodes in the graph G and disconnect the irrelevant edges: if the value of the element in the i-th row and j-th column of A hard is 0, it is determined that there is no interaction between vehicle i and j, otherwise it is considered that there is an interaction between the two vehicles, so that the vehicle nodes with effective interaction are determined and the multi-vehicle interaction subgraph G sub after primary reduction is obtained.

[0041] Then, based on the primary reduced multi-vehicle interaction subgraph G sub , the importance of the effective interaction is calculated using soft self-attention: the normalized attention score matrix A soft is calculated to represent the importance of the effective interaction.

[0042]

[0043] Finally, the driving state information of the surrounding vehicles is selectively extracted, and a multi-vehicle motion interaction correlation representation B is constructed ′ :

[0044] B'=Attention(Q qry ,K,V)=VA soft (6)

[0045] wherein K is a key matrix, and V is a value matrix.

[0046] Further, the hybrid electric vehicle "driving behavior-comprehensive energy consumption" simulation model in step S400 of the present application comprises: a city road network model, a driving behavior model, and a hybrid electric vehicle quasi-static energy consumption model.

[0047] The city road network model is constructed based on the city traffic simulation platform SUMO.

[0048] The driving behavior model is constructed based on the pre-installed car following model, lane changing model and intersection model of the SUMO system, and is a driving behavior decision unit, taking the surrounding traffic state of the vehicle as input and outputting the vehicle acceleration and speed.

[0049] The quasi-static energy consumption model of the hybrid electric vehicle is constructed based on the vehicle longitudinal dynamics, taking the vehicle speed and acceleration as input to update and calculate the vehicle power system state and trip energy consumption, and is specifically composed as follows:

[0050] The vehicle dynamics model: according to the expected acceleration and current speed output by the driving behavior model, the required driving / braking power P req :

[0051] P req =F req v,F req =F j +F f +F i +F ω ,F j =ma acc ,F f =mgf cosθ road ,

[0052] F i =mg sinθ road ,F ω =C D A front v 2 / 21.15 (7)

[0053] wherein F reqDemand-driven / braking force, v is vehicle speed (m / s), m is vehicle curb weight (kg), a acc acceleration (m / s²) 2 ), f is the rolling resistance coefficient, θ road For road slope, C D A is the air drag coefficient. front The frontal area of ​​the vehicle (m²) 2 ), where g is the acceleration due to gravity;

[0054] Engine fuel consumption model: Based on the control actions of the energy management strategy (engine speed, engine torque), calculate the instantaneous fuel consumption of the engine at the current moment.

[0055] m f =T eng n eng f eng (T eng ,n eng ) / (3.6×10 6 (8)

[0056] Where, m f Fuel consumption per unit time, T eng n is the engine output torque. eng f is the engine speed. eng (T eng ,n eng The specific fuel consumption (g / kW·h) is obtained through engine bench testing. The given control action should not exceed the engine's external characteristics and maximum speed limit; otherwise, it is an infeasible control action.

[0057] Drive motor / generator power consumption model: Given the required drive / braking power of the entire vehicle, engine speed and torque, calculate the required drive motor / generator power P at the current moment based on the powertrain configuration. mot / gen :

[0058] P mot / gen =P req η -sgn(Preq) (9)

[0059] Where η is the motor efficiency characteristic table obtained through bench testing. If the speed and torque of the drive motor / generator exceed its external characteristics when a given control action is given, it is an infeasible control action.

[0060] Battery internal resistance model: The power battery provides / recovers the driving / generating power of all drive motors / generators in the hybrid system, thus based on the power battery power P. batt Calculate the current I of the power battery. batt State of Charge (SoC) Change Rate:

[0061]

[0062] where, U oc is the open circuit voltage of the battery pack, R is the internal resistance of the battery pack, Q batt is the capacity of the battery pack, under a given control action, if the current breaks the maximum value, I batt is calculated as an imaginary number, the battery charging and discharging power breaks the power limit, and all are unfeasible control actions.

[0063] Further, the key elements of the energy management strategy method summarized in step S500 of the present application include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model.

[0064] The key elements define the implementation method as follows:

[0065] (1) Define the state space S: define the state vector s as the observation of the vehicle power system state o power , the motion state of the multi-vehicle system o N} The vector formed after concatenation, s∈S, and each element in the state vector is Min-Max normalized. The vehicle power system state describes the observation of the vehicle power system state, constraints, including but not limited to the state of charge SoC of the power battery, the degree of deviation of the state of charge from the expected stable value ΔSoC, the demand driving / braking power P req and torque T req of the vehicle, the demand acceleration acc, the vehicle speed v, the external characteristic constraints of the power system components, the power battery charging and discharging power limit, and the travel time.

[0066] The state space here also adds the constraint state representation of the power system and components to help strategy learning and avoid unsafe control actions. The observation of the motion state of the multi-vehicle system is the time series driving information observation of the multi-vehicle system described in step S100.

[0067] When the SoC in the state vector is lower than the specified lower limit value or higher than the specified upper limit value, or the travel time reaches the maximum value T, the state is considered to be a terminal state.

[0068] (2) Define the action space A: the action quantity is determined according to the power system configuration and its control degrees of freedom, including but not limited to taking the engine speed T eng and the engine torque W eng as the action vector a=[T eng , W eng ], a∈A.

[0069] (3) The reward function r.

[0070] First, the engine instantaneous fuel consumption m eng The weighted sum of the deviations of the power battery SoC is used as the reward function: r = -κm eng -λΔSoC 2 , r≤0; where κ and λ are weighting factors, and κ is a conditional value: κ=0.6 if and only if the engine efficiency is higher than the set threshold, otherwise κ=1;

[0071] Next, we will perform Min-Max normalization and set limits for r: Where clip(x,x) min ,x max ) with preset boundary value x min x max The input x is truncated, and due to the influence of the action amount, r min Unable to determine precisely, a smaller reward threshold is chosen as r. min ;

[0072] Define the triggering event to adjust the reward function r: Define a constant penalty term ρ = 0.4, r = r - nρ if and only if the engine start-stop or infeasible control action event occurs (occurring n times); Define a constant bonus = 8, r = r + bonus when the entire trip is completed and the SoC is still within the stable range, and r = r - bonus if the trip terminates early or the entire trip is completed but the SoC exceeds the stable range.

[0073] (4) Energy management strategy model a~π θ (·|s): Its structure consists of a multi-vehicle interactive perception model and a fully connected feedforward neural network, with parameter set θ. The model incorporates the observations of the multi-vehicle system {o1,o2,…,o…}. N The input multi-vehicle interaction perception model outputs a multi-vehicle interaction association representation b′1 related to the vehicle itself, and then b′1 is correlated with the powertrain system state o. power A cascaded, parallel-input three-layer fully connected feedforward neural network is used, with the number of neurons in the output layer equal to twice the dimension of the action vector, outputting the mean μ of each action vector. θ With variance σ θ During training, the action vector is obtained through reparameterized sampling: a θ (s,ξ)=tanh(μ θ ,σ θ ⊙ξ), Where ξ represents Gaussian noise, and ⊙ represents element-wise multiplication;

[0074] (5) Energy Management Strategy Evaluation Model Q: Four fully connected neural networks with identical structures are established as the energy management strategy evaluation model, with parameter sets Φ1, Φ2, Φ3, Φ4, Φ5, Φ6, Φ7, Φ8, Φ9, Φ1 ... targ1 Φ targ2The number of neurons in each input layer of the network is equal to the sum of the dimensions of the state vector and the action vector, and the number of neurons in the output layer is 1; there are 4 hidden layers, and the number of neurons decreases layer by layer; the activation function of the output layer is a linear activation function, and the activation functions of the other layers are ReLU linear rectified functions.

[0075] Furthermore, the method for training and fusing to obtain the final energy management strategy model in step S600 of the present invention includes: initialization of the "driving behavior-integrated energy consumption" simulation model, initialization of the energy management strategy model and its evaluation model, generation of energy management strategy training data, interactive training of the energy management strategy, saving of the energy management strategy, and evaluation and online application of the energy management strategy.

[0076] Furthermore, the method for obtaining the final energy management strategy model through training fusion of the present invention also includes:

[0077] (1) The Xavier initialization method is used to initialize the strategy model parameters θ and the energy management strategy evaluation model parameters Φ1 and Φ2, and assign values ​​to Φ. targ1 =Φ1,Φ targ2 =Φ2, update rate τ = 0.005, discount rate γ = 0.99, entropy coefficient α = 0.01, maximum number of updates N max The number of updates, k = 0;

[0078] (2) Initialization of the simulation environment for "driving behavior-comprehensive energy consumption": Configure traffic flow within the road network, with an average vehicle speed of 72 km / h, and the vehicle speed adjustment coefficient follows a normal distribution with a mean of 1 and a variance of 0.2; initial SoC of the controlled vehicle's power system battery = 0.5; termination state flag d = 0; initialize storage space. The training data is stored with a sample size of 2e5; the initial simulation environment is denoted as s′, r=HEV(a,s), that is, the model calculates the model state and reward value at the next moment based on the current model state and control action;

[0079] (3) For the controlled vehicle, obtain its state vector s at the current moment;

[0080] (4) Obtain the motion vector a~π of the controlled vehicle's power system control. θ (·|s), and execute the action in the simulation model: s′,r=HEV(a,s);

[0081] (5) Determine if s' is a terminated state: If not, store the data tuple (s,a,r,s',d) as a sample in the storage space. Update the current environment state s = s'; otherwise, d = 1, and store the data tuple (s, a, r, s', d) as a sample into the storage space. And perform step (2) once;

[0082] (6) When the storage space contains more than or equal to 100 samples, step (7) is performed, otherwise step (3) is performed;

[0083] (7) Randomly sample 256 samples from the storage space : B = {(s, a, r, s')}, and calculate the evaluation model update target:

[0084]

[0085]

[0086] (8) Update the Q network parameters Φ1, Φ2 using gradient descent:

[0087]

[0088] (9) Update the strategy model parameters θ using gradient descent:

[0089]

[0090] (10) Update Φ targ1 , Φ targ2 : Φ targi = τΦ targi + (1-τ)τΦ i , i = 1, 2;

[0091] (11) Update the parameter update times: k = k + 1; if k ≤ N max , return to step (3), otherwise perform step (12);

[0092] (12) Save the energy management strategy model a ~ π θ (·|s) and its parameters θ, complete the training of the energy management strategy of the fusion multi-vehicle interactive perception.

[0093] Further, the energy management strategy evaluation and online application implementation method of the present application comprises: in each control period, obtaining the state vector s of the controlled vehicle and its surrounding multi-vehicle system, calculating π θ (·|s) to obtain the mean value μ θ of the action vector as the control action a, sending it to the power system of the controlled vehicle and performing the control action, completing the online application of the strategy.

[0094] From the above technical solutions, the application realizes the energy management of the hybrid electric vehicle with multi-vehicle motion interaction perception, supports the controlled vehicle to extract real-time motion information representation of other vehicles associated with the energy optimization of the hybrid power system, and can improve the optimization performance and working condition adaptability of the hybrid electric vehicle energy management in a multi-vehicle system in a high dynamic and dense urban road. BRIEF DESCRIPTION OF DRAWINGS

[0095] Figure 1 The application is a multi-vehicle motion interaction perception fusion networked hybrid electric vehicle energy management method flowchart.

[0096] Figure 2 It is a multi-vehicle interaction relationship topology diagram.

[0097] Figure 3 It is a multi-vehicle interaction perception model diagram.

[0098] Figure 4 It is a random type energy management strategy network structure diagram fusion multi-vehicle interaction perception. DETAILED DESCRIPTION

[0099] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments.

[0100] As shown in the fusion multi-vehicle motion interaction perception networked hybrid electric vehicle energy management method described in the embodiments, the method specifically includes the following steps: Figure 1 S100, collecting multi-vehicle system time sequence driving information in a certain time window within the self-vehicle perception range, constructing a time sequence driving information encoder based on an LSTM network, and performing feature coding on the time sequence driving information of each vehicle to output a time sequence driving information feature vector thereof;

[0101] S200, taking the time sequence driving information feature vector as a node, and combining a complete undirected graph to establish a multi-vehicle interaction relationship topology diagram within the self-vehicle perception range;

[0102] S300, cropping the multi-vehicle interaction relationship topology diagram to construct a multi-vehicle interaction perception model;

[0103] S400, constructing a hybrid electric vehicle driving behavior-comprehensive energy consumption simulation model;

[0104] S500, based on the multi-vehicle interaction perception model and the driving behavior-comprehensive energy consumption simulation model, constructing an energy management strategy model based on maximum entropy deep reinforcement learning;

[0105]

[0106] ​S600, based on reinforcement learning SoftActor-Critc algorithm to driving behavior-comprehensive energy consumption simulation model, energy management strategy model initialization, training fusion to obtain the final energy management strategy model.

[0107] The following will be described in detail:

[0108] S100, in the self-vehicle perception range, collect the multi-vehicle system time series driving information in a certain time window, construct a time series driving information encoder based on the LSTM network, and encode the time series driving information of each vehicle, output the time series driving information feature vector thereof;

[0109] The self-vehicle perception range is determined by the real-time transmission range of the vehicle networking communication data of the self-vehicle and the surrounding vehicles. At time t, when the number of vehicles in the self-vehicle perception range is N, the time series driving information observation of vehicle i (i = 1, 2, …, N) is To describe the vehicle position, lateral and longitudinal speed, heading angle and other driving state observations. At time t, collect the time series driving information observations of the multi-vehicle system in the self-vehicle perception range in the historical time window T:

[0110] The method for constructing a time series driving information encoder by an LSTM network includes:

[0111] The time series driving information observations of each vehicle are encoded:

[0112]

[0113] Wherein, And The LSTM network unit hidden state and unit state are respectively.

[0114] S200, let the time series driving information feature vector be a node, and combine a complete undirected graph to establish a multi-vehicle interaction relationship topology graph in the self-vehicle perception range;

[0115] The method for establishing a multi-vehicle interaction relationship topology graph in the self-vehicle perception range by a complete undirected graph includes:

[0116] Model each vehicle in the multi-vehicle system as a node of the interaction relationship topology graph, and encode the time series driving information observations of each vehicle node as its attribute vector;

[0117] Model the interaction relationship between vehicles in the multi-vehicle system as an edge of the interaction relationship topology graph, and define an edge vector to describe the importance of the interaction relationship between vehicles;

[0118] As shown in Figure 2 , a complete undirected graph is used to model the interaction relationship of the multi-vehicle system in the self-vehicle perception range, and the key elements of the model are defined as follows:

[0119] (1) Nodes and attribute vectors of the interaction relationship topology graph:

[0120] Vehicle i (i = 1, 2, ..., N) is defined as a node in the graph network G.

[0121] The feature encoding vector of the time-series driving information observation is defined as the attribute vector of the node.

[0122] The set of all vehicle node attributes is denoted as The graph with the i-th vehicle as the central node is denoted as G. i .

[0123] (2) Edges and edge vectors of the interaction topology graph:

[0124] The interactions between vehicles are represented as edges in graph G, E = {w1, w2, ..., w N Let} be the set of edges, and let w be the edge vector. i Used to describe the importance of the interaction between vehicle i and other vehicles.

[0125] S300, trim the multi-vehicle interaction relationship topology diagram, and construct a multi-vehicle interaction perception model;

[0126] The method for constructing a multi-vehicle interaction perception model includes: constructing a self-attention network to calculate the importance of multi-vehicle interaction relationships; using hard attention and reparameterization methods to prune the multi-vehicle interaction relationship topology graph to obtain a reduced multi-vehicle interaction relationship subgraph; and then using a soft self-attention method to extract the motion interaction association representations of other vehicles related to each node.

[0127] like Figure 3 As shown, the multi-vehicle interaction perception model also includes:

[0128] (1) LSTM encoding layer: The temporal driving information encoder established in step S100. Its input is the attribute vectors of the multi-vehicle interaction relationship topology graph. The output is The encoder shares parameters among multiple vehicles.

[0129] (2) Stochastic hard self-attention layer. It is a vector composed of the set of node attributes in graph G. The input sequence is a sequence whose length N can be determined based on the number of nodes in the real-time multi-vehicle system rather than being a fixed value; the output is the attention score matrix A. attn Value matrix V, binarized attention score matrix A hard .

[0130] First, using a learnable weight matrix Generate query matrix Q qry= W q X, key matrix K = W k X, value matrix V = W v X, compute attention score matrix A by dot product attn and A' attn :

[0131]

[0132] where softmax(·) normalizes by column, A attn and A' attn is used to describe the importance of the interaction between each vehicle within the perception range of the ego vehicle. A' attn is the edge set of the multi-vehicle interaction topology graph established in step one.

[0133] Next, the binary attention score matrix A hard (0 or 1, indicating whether there is an effective interaction between the two vehicles) between the vehicles is learned by hard self-attention at this layer, and the multi-vehicle interaction topology graph is pruned to extract the effective vehicle-vehicle interaction. Specifically, the Gumble-Softmax reparameterization method is introduced to obtain a random sampling output A hard :

[0134]

[0135] where gum(·) is the Gumble-Softmax function mapped by column, taking a column vector as input, and its output is as follows.

[0136]

[0137] where α i is the attention score, τ is the temperature coefficient, and the Gumbel noise term g i = -log(-log(u i )), u i ~ U(0, 1).

[0138] (3) Deterministic soft self-attention layer. Its input is A attn , A hard , value matrix V, and the output is the multi-vehicle motion interaction correlation representation

[0139] First, according to the attention weight A hard , remove the irrelevant vehicle nodes in the graph G and disconnect the irrelevant edges: if A hardIf the value of the element in the i-th row and the j-th column is 0, it is determined that the vehicles i and j have no interaction relationship, otherwise it is considered that the two vehicles have an interaction relationship, thereby explicitly determining the vehicle nodes with effective interaction relationship and obtaining the multi-vehicle interaction subgraph G sub .

[0140] Then, based on the primary reduced multi-vehicle interaction subgraph G sub , the importance of the effective interaction relationship is calculated by using soft self-attention: the normalized attention score matrix A soft is calculated to represent the importance of the effective interaction relationship.

[0141]

[0142] Finally, the driving state information of the surrounding vehicles is selectively extracted to construct a multi-vehicle motion interaction correlation representation B ′ :

[0143] B′=Attention(Q qry ,K,V)=VA soft (6)

[0144] S400, construct a hybrid electric vehicle "driving behavior-comprehensive energy consumption" simulation model;

[0145] The hybrid electric vehicle "driving behavior-comprehensive energy consumption" simulation model includes: a city road network model, a driving behavior model, and a hybrid electric vehicle quasi-static energy consumption model.

[0146] (1) City road network model: the city road network is constructed based on the city traffic simulation platform SUMO.

[0147] (2) Driving behavior model: the driving behavior model of the vehicle is constructed based on the pre-installed car following model, lane changing model and intersection model of the SUMO system, which is a driving behavior decision unit, taking the traffic state around the vehicle as input and outputting the vehicle acceleration and speed.

[0148] (3) Based on the vehicle longitudinal dynamics, a hybrid electric vehicle quasi-static energy consumption model is constructed, taking the vehicle speed and acceleration as input to update and calculate the vehicle power system state and trip energy consumption, which is specifically composed as follows.

[0149] (3.1) Vehicle dynamics model. According to the expected acceleration and current speed output by the driving behavior model, the whole vehicle demand driving / braking power P req :

[0150] P req =F req v,F req =F j +F f +F i+F ω ,F j =ma acc ,F f =mgf cosθ road ,

[0151] F i =mg sinθ road ,F ω =C D A front v 2 / 21.15 (7)

[0152] where F req is the demand driving / braking force, v is the vehicle speed (m / s), m is the vehicle mass (kg), a acc is the acceleration (m / s 2 ), f is the rolling resistance coefficient, θ road is the road slope, C D is the air resistance coefficient, A front is the vehicle frontal area (m 2 ), and g is the gravity acceleration.

[0153] (3.2) Engine fuel consumption model. It calculates the engine instantaneous fuel consumption at current time according to the control actions (engine speed, engine torque) of the energy management strategy:

[0154] m f =T eng n eng f eng (T e g ,n eng ) / (3.6×10 6 )(8)

[0155] where m f is the fuel consumption per unit time, T eng is the engine output torque, n eng is the engine speed, f eng (T eng ,n eng ) is the engine specific fuel consumption table (g / kW·h), which is obtained through engine bench test. The given control actions should not exceed the engine external characteristics and maximum speed limit, otherwise it is infeasible control action.

[0156] (3.3) Driving motor / generator power consumption model. It calculates the required driving motor / generator power P mo t / gen at current time according to the powertrain configuration when the vehicle demand driving / braking power, engine speed and torque are given:

[0157]

[0158] where η is the efficiency characteristic of the motor obtained by bench test. If the speed and torque of the driving motor / generator exceed its external characteristic at a given control action, the control action is infeasible.

[0159] (3.4) Battery resistance model. The power provided / recovered by the power battery provides / drives all driving motors / generators in the hybrid power system, so as to calculate the current I batt , state of charge SoC change rate of the power battery according to the power P batt of the power battery:

[0160]

[0161] where U oc is the open circuit voltage of the battery pack, R is the internal resistance of the battery pack, and Q batt is the capacity of the battery pack. If the current exceeds the maximum value, I batt is calculated as an imaginary number, and the battery charging and discharging power exceeds the power limit, which are all infeasible control actions.

[0162] S500, based on the multi-vehicle interaction perception model and the "driving behavior-comprehensive energy consumption" simulation model, an energy management strategy model based on maximum entropy deep reinforcement learning is constructed

[0163] The key elements of the energy management strategy model include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model.

[0164] Wherein, the key element definition implementation method is as follows:

[0165] (1) Define the state space S. Define the state vector s as the observation of the vehicle power system state o power , the motion state of the multi-vehicle system {o1, o2, …, o N} After concatenation, the vector s∈S, and each element in the state vector is Min-Max normalized. The vehicle power system state describes the observation of the vehicle power system state, constraints, etc., including but not limited to the state of charge SoC of the power battery, the degree of deviation of the state of charge from the expected stable value ΔSoC, the demand driving / braking power P req and torque T req, demand acceleration acc, vehicle speed v, external characteristic constraints of powertrain components, power battery charge-discharge power limits, travel time, etc. In particular, the state space here should add the constraint state representation of the powertrain and components to help policy learning and avoid unsafe control actions. The observation of the motion state of the multi-vehicle system is the time sequence travel information observation of the multi-vehicle system described in step S100.

[0166] When the SoC in the state vector is lower than the lower limit (0.2) or higher than the upper limit (0.9), or the travel time reaches the maximum value T, the state is considered to be a terminal state.

[0167] (2) Define the action space A.

[0168] The action quantity is determined according to the powertrain configuration and its control degrees of freedom, including but not limited to the engine speed T eng , the engine torque W eng , the power battery charge-discharge power, etc. eng , the power battery charge-discharge power, etc. eng , as the action vector a = [T eng , W eng ], a ∈ A.

[0169] (3) Reward function r.

[0170] First, the weighted sum of the engine instantaneous fuel consumption m eng and the SoC deviation of the power battery is taken as the reward function: r = -km eng - λΔSoC 2 , r ≤ 0. Where κ and λ are weight factors, and κ is a conditional value: κ = 0.6 only if the engine operating efficiency is higher than a set threshold, otherwise κ = 1.

[0171] Next, r is Min-Max normalized and limited: Where clip(x, x min , x max ) truncates the input x with preset boundary values x min , x max . Due to the influence of the action quantity, r min cannot be accurately determined, and a smaller reward threshold (less than 90% of r) can be selected as r min .

[0172] Finally, define the trigger event to adjust the reward function r: define a constant penalty term ρ = 0.4, and r = r - nρ only when events such as engine start-stop, unfeasible control actions, etc. (n times) occur; define a constant bonus = 8, and r = r + bonus when the entire trip ends and the SoC is still in the stable range, and r = r - bonus if the trip is terminated early or the entire trip ends but the SoC exceeds the stable range.

[0173] (4) Energy management strategy model a ~ π θ (·|s). Its structure is composed of the multi-vehicle interaction perception model in step three and a fully connected feedforward neural network, as shown in Figure 4 Fig. 3. The parameter set is θ, and the model inputs the multi-vehicle system observation {o1, o2, …, o N} into the multi-vehicle interaction perception model and outputs the multi-vehicle interaction related representation b'1 related to the ego vehicle, which is then concatenated with the power system state o power and input into a three-layer fully connected feedforward neural network. The number of output layer neurons is equal to twice the dimension of the action vector, respectively outputting the mean μ θ and variance σ θ of each action quantity. In the training process, the action vector is obtained by reparameterization sampling: a θ (s, ξ) = tanh(μ θ , σ θ ⊙ ξ), where ξ is Gaussian noise and ⊙ is element-wise multiplication.

[0174] (5) Energy management strategy evaluation model Q. Four fully connected neural networks with the same structure are established as the energy management strategy evaluation model, and the parameter sets are Φ1, Φ2, Φ targ1 , and Φ targ2 , respectively. The number of input layer neurons of each network is equal to the sum of the dimensions of the state vector and the action vector, and the number of output layer neurons is 1; the hidden layer has 4 layers, and the number of neurons decreases layer by layer; the output layer activation function is a linear activation function, and the other layers are linear rectification functions ReLU.

[0175] S600, based on the reinforcement learning SoftActor-Critc algorithm, the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model are initialized, and the final energy management strategy model is obtained by training fusion.

[0176] The method of training and fusing the energy management strategy model based on the reinforcement learning SoftActor-Critc (SAC) algorithm and the multi-vehicle interaction perception further includes: initializing the "driving behavior-comprehensive energy consumption" simulation model, the energy management strategy model and its evaluation model, initializing the "driving behavior-comprehensive energy consumption" simulation model, initializing the energy management strategy model and its evaluation model, generating energy management strategy training data, interactive training of energy management strategy, saving energy management strategy, energy management strategy evaluation and online and application, and the specific implementation method is as follows:

[0177] (1) The Xavier initialization method is used to initialize the policy model parameters θ and the energy management strategy evaluation model parameters Φ1, Φ2, and assign Φ targ1 = Φ1 and Φ targ2= Φ2, update rate τ = 0.005, discount rate γ = 0.99, entropy coefficient α = 0.01, maximum number of updates N max , update number k = 0;

[0178] (2) "Driving behavior-comprehensive energy consumption" simulation environment initialization. Configure the vehicle flow in the road network, the average speed of each vehicle is 72 km / h, and the speed adjustment coefficient obeys the normal distribution (mean value is 1 and variance is 0.2); the initial SoC of the controlled vehicle power system battery is 0.5; the termination state marker d = 0; initialize the storage space to store training data, and the training data sample capacity is 2e5; record the initialized simulation environment as s', r = HEV(a, s), that is, the model calculates the model state and reward value at the next time according to the current model state and control action.

[0179] (3) For the controlled vehicle, obtain its state vector s at the current time;

[0180] (4) Obtain the action vector a ~ π θ (|s) of the controlled vehicle power system control, and execute the action in the simulation model: s', r = HEV(a, s);

[0181] (5) Determine whether s' is the termination state: if not, store the data tuple (s, a, r, s', d) as a sample in the storage space , update the environment state s = s' at the current time; otherwise, d = 1, store the data tuple (s, a, r, s', d) as a sample in the storage space , and execute step (2) once;

[0182] (6) When the sample capacity in the storage space is greater than or equal to 100, execute step (7), otherwise execute step (3);

[0183] (7) Randomly sample 256 samples from the storage space : B = {(s, a, r, s')}, calculate the evaluation model update target:

[0184]

[0185] (8) Update the Q network parameters Φ1, Φ2 using gradient descent:

[0186]

[0187] (9) Update the strategy model parameters θ using gradient descent:

[0188]

[0189] (10) update Φ targ1 , Φ targ2 targi = τΦ targi + (1-τ)Φ i , i = 1, 2;

[0190] (11) update parameter update times: k = k + 1; if k≤N max , return to step (3), otherwise execute step (12);

[0191] (12) save the energy management strategy model a~π θ (·|s) and its parameters θ, complete the training of the energy management strategy of the fused multi-vehicle interactive perception.

[0192] The implementation method of strategy evaluation and online application: in each control period, the state vector s of the controlled vehicle and its surrounding multi-vehicle system is obtained, the action vector mean value μ θ (·|s) is calculated by using π θ as the control action a, which is sent to the power system of the controlled vehicle and the control action is executed, completing the online application of the strategy.

[0193] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of them can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0194] ​It is to be noted that, in the present text, the relative terms such as first and second, and the like are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements in the list, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element. Each of the embodiments in the present specification is described in a relevant manner, and the same or similar parts between the embodiments can be mutually referred to, and each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0195] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for energy management of a connected hybrid electric vehicle by fusing multi-vehicle motion interaction perception, characterized in that, The method comprises the following steps: S100, collecting multi-vehicle system timing driving information in a certain time window within the self-vehicle perception range, constructing a timing driving information encoder based on an LSTM network, and performing feature coding on the timing driving information of each vehicle to output a timing driving information feature vector of each vehicle; S200, taking the timing driving information feature vector as a node, and combining a complete undirected graph to establish a multi-vehicle interaction relationship topology graph within the self-vehicle perception range; S300, cropping the multi-vehicle interaction relationship topology graph to construct a multi-vehicle interaction perception model; S400, constructing a hybrid electric vehicle driving behavior-comprehensive energy consumption simulation model; S500, based on the multi-vehicle interaction perception model and the driving behavior-comprehensive energy consumption simulation model, constructing an energy management strategy model based on maximum entropy deep reinforcement learning; S600, initializing the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model based on the reinforcement learning Soft Actor-Critc algorithm, and training and fusing to obtain a final energy management strategy model; In step S300, the method for constructing the multi-vehicle interaction perception model comprises: constructing a self-attention network to calculate the importance of the multi-vehicle interaction relationship; using a hard attention and a reparameterization method to crop the multi-vehicle interaction relationship topology graph to obtain a reduced multi-vehicle interaction relationship subgraph; using a soft self-attention method to extract the motion interaction correlation representation related to each node vehicle; constructing a hybrid electric vehicle driving behavior-comprehensive energy consumption simulation model comprises: a city road network model, a driving behavior model, and a hybrid electric vehicle quasi-static energy consumption model; The city road network model is constructed based on the city traffic simulation platform SUMO; The driving behavior model is constructed based on the SUMO system preset car following model, lane changing model, and intersection model, and is a driving behavior decision unit, which takes the surrounding traffic state of the vehicle as input and outputs the vehicle acceleration and speed; The hybrid electric vehicle quasi-static energy consumption model is constructed based on the vehicle longitudinal dynamics, and takes the vehicle speed and acceleration as input to update and calculate the vehicle power system state and travel energy consumption; The key elements of the energy management strategy model in step S500 include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model; In step S600, the method for training and fusing to obtain the final energy management strategy model comprises: training and fusing to obtain the final energy management strategy model includes: initializing the "driving behavior-comprehensive energy consumption" simulation model, initializing the energy management strategy model and its evaluation model, generating energy management strategy training data, interactive training of the energy management strategy, saving the energy management strategy, and evaluating and online and applying the energy management strategy.

2. The method of claim 1, wherein the method is a method of fusion of multi-vehicle motion interaction perception for networked hybrid electric vehicle energy management. In step S100, the method for constructing the LSTM network to construct the timing driving information encoder comprises: performing feature coding on the observation of the timing driving information of each vehicle: （1） wherein, and are the LSTM network unit hidden state and cell state, respectively, is the information observation.

3. The method of claim 1, wherein the method further comprises: In step S200, the method for constructing the multi-vehicle interaction relationship topology graph within the self-vehicle perception range comprises: modeling each vehicle in the multi-vehicle system as a node of the interaction relationship topology graph, and taking the feature vector of the observation of the timing driving information of each vehicle node as its attribute vector; In the multi-vehicle system, the interaction relationship between vehicles is modeled as the edge of the interaction relationship topology graph, and the edge vector is defined to describe the importance of the interaction relationship between vehicles; the complete undirected graph is used to model the interaction relationship of the multi-vehicle system within the self-vehicle perception range, and the key elements of the model are defined as follows: (1) The node of the interaction relationship topology graph and the attribute vector of the node: Vehicles , , defining as the node of the graph network , defining the feature vector of the time-series travel information observation as the attribute vector of the node ; The set of all vehicle node attributes is denoted by { , ,…, } ; the graph network with the th vehicle as the center node is denoted by ; (2) The edge of the interaction relationship topology graph and the edge vector: The interaction relationships between vehicles are used as a graph network G The edge, Let edge set, edge vector Used to describe vehicles The importance of interactions with other vehicles.

4. The method of claim 1, wherein the method further comprises: The multi-vehicle interaction perception model also includes: (1) LSTM encoding layer: the time series driving information encoder established in step S100, the input of which is each attribute vector of the multi-vehicle interaction relationship topology graph , and the output is , ,…, ; the encoder shares parameters between the multi-vehicles; (2) a random hard self-attention layer, which takes a vector composed of node attribute sets in a graph network as an input sequence, and the length of the input sequence is determined according to the number of nodes in the real-time multi-vehicle system, rather than a fixed value; ;​​​​ Utilizing learnable weight matrices , , , generate query matrix , key matrix , value matrix , compute attention score matrix and : （2） wherein, column-wise normalization, and for describing the importance of the interaction relationship between each vehicle within the perception range of the ego vehicle, is the edge set of the multi-vehicle interaction relationship topology established in step one. In this layer, the binary attention score matrix between vehicles is learned by hard self-attention The multi-vehicle interaction relationship topology is cropped, the effective vehicle-vehicle interaction relationship is extracted, the Gumble-Softmax reparameterization method is introduced, and the random sampling output is obtained under the differentiable premise : （3） where, is the Gumble-Softmax function for column-wise mapping with column vector as input, whose output formula is as follows （4） wherein, is an attention score, is a temperature coefficient, Gumbel noise term , ; (3) a deterministic soft self-attention layer, input of which is , , value matrix , and output of which is a multi-vehicle motion interaction correlation representation ; First, based on attention weights Remove image Disconnect unrelated vehicle nodes and unrelated edges: If No. line, number If a column element has a value of 0, then the vehicle is identified. and If there is no interaction relationship, then the two vehicles are considered to have an interaction relationship. This identifies the vehicle nodes with valid interaction relationships and obtains the multi-vehicle interaction subgraph after initial reduction. ; After that, the multi-vehicle interaction sub-graph of the primary reduction On this basis, the importance of the effective interaction relationship is calculated by using soft self-attention: the normalized attention score matrix is calculated to represent the importance of the effective interaction relationship; （5） Finally, the driving state information of the surrounding vehicles is selectively extracted to construct a multi-vehicle motion interaction correlation representation : （6） Wherein K is the key matrix, and V is the value matrix.

5. The method of claim 1, wherein the method further comprises: The "driving behavior-comprehensive energy consumption" simulation model of the hybrid electric vehicle in the step S400 includes the following specific components: Vehicle dynamics model: calculate the required drive / brake power of the vehicle according to the expected acceleration output by the driving behavior model and the current vehicle speed : （7） wherein, is the demand drive / braking force, is the vehicle speed, , is the total vehicle kerb mass, , is the acceleration, , is the rolling resistance coefficient, is the road gradient, is the air resistance coefficient, is the vehicle frontal area , is the acceleration due to gravity; Engine fuel consumption model: according to the control action of the energy management strategy, the instantaneous fuel consumption of the engine at the current time is calculated: （8） wherein, is the fuel consumption per unit time, is the engine output torque, is the engine speed, is the engine specific fuel consumption table, , which is obtained by engine bench test, the given control action should not exceed the engine external characteristic and the maximum speed limit, otherwise it is an infeasible control action; Drive motor / generator power consumption model: the required drive motor / generator power at the current time is calculated according to the powertrain configuration given the required drive / brake power of the whole vehicle, engine speed and torque : （9） wherein, is the motor efficiency characteristic table obtained by bench test, and if the rotational speed and torque of the driving motor / generator exceed its external characteristic at a given control action, the control action is infeasible. Battery internal resistance model: the power battery power provides / recoveries all drive motor / generator of the hybrid power system drive / generation power, thereby according to the power battery power , the current of the power battery is calculated , state of charge SoC change rate: （10） Wherein, Voc is the open circuit voltage of the battery pack, R is the internal resistance of the battery pack, C is the capacity of the battery pack, if the current breaks the maximum value under the given control action, The calculation is a virtual number, the battery charging and discharging power breaks the power limit, which is an unfeasible control action.

6. The method of claim 1, wherein the method further comprises: The key elements of the energy management strategy model summarized in the step S500 include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model; Wherein, the implementation method of the key elements is as follows: (1) Define state space : Define state vector for describing vehicle power system state , observation of multi-vehicle system motion state , The vector formed after cascading, , and the elements in the state vector are Min-Max normalized, the vehicle power system state describes the observation of the vehicle power system state, constraints, including: power battery state of charge , the degree of deviation of the state of charge from the expected stable value , whole vehicle demand driving / braking power and torque , demand acceleration , vehicle speed , external characteristic constraints of power system components, power battery charging and discharging power limit, travel time; Here, the constraint state representation of the power system and components is also added to the state space to help strategy learning and avoid unsafe control actions; the observation of the multi-vehicle system motion state is the time series driving information observation of the multi-vehicle system described in step one; When the SoC in the state vector is below a specified lower value or above a specified upper value, or the travel time reaches a maximum value the state is considered to be a termination state; (2) Define the action space : The action amount is determined according to the power system configuration and its control degrees of freedom, including: engine speed , engine torque as the action vector , ; (3) reward function ; First, the engine's instantaneous fuel consumption The weighted sum of the deviations of the power battery SoC is used as the reward function: ;in, and As a weighting factor, and Conditional value: This value is true if and only if the engine efficiency is higher than a set threshold. ,otherwise ; Next, Perform Min-Max normalization and set limits: ,in With preset boundary values , For input Cut off, due to the influence of the amount of motion, Unable to determine precisely, a smaller reward threshold is chosen as... ; Define the trigger event and the reward function. Adjustments are made: Define a constant penalty term. If and only if the number of times the engine start-stop or infeasible control action event occurs is 1, hour, Define constants When the entire process is complete and the SoC remains within a stable range If the trip is terminated early, or the entire trip is completed but the SoC exceeds the stability range, ; (4) Energy management strategy model : The structure comprises: multi-vehicle interaction perception model and full connection feedforward neural network, parameter set is , the model inputs multi-vehicle system observation , outputs multi-vehicle interaction related representation related to the ego vehicle , and then and power system state are cascaded and input into a three-layer full connection feedforward neural network, the number of output layer neurons is equal to twice the dimension of the action vector, and the mean value of each action quantity and variance are output respectively; in the training process, the action vector is obtained through reparameterization sampling: , wherein is Gaussian noise, is element multiplication; (5) Energy management strategy evaluation model : Four fully connected neural networks with the same structure are established as the energy management strategy evaluation model, and the parameter sets are , , , ; the number of input layer neurons of each network is equal to the sum of the dimensions of the state vector and the action vector, and the number of output layer neurons is 1; the hidden layer has 4 layers, and the number of neurons decreases layer by layer; the output layer activation function is a linear activation function, and the other layers are linear rectifier functions ReLU.

7. The method of claim 1, wherein the method further comprises: The method for training and fusing to obtain the final energy management strategy model also includes: (1) The strategy model parameters are initialized by using the Xavier initialization method , the energy management strategy evaluation model parameters , are initialized and assigned values , , the update rate , the discount rate , the entropy coefficient , the maximum number of updates , and the number of updates ; (2) "Driving behavior-comprehensive energy consumption" simulation environment initialization: configure the vehicle flow in the road network, the average speed of each vehicle is 72 km / h, and the speed adjustment coefficient is subject to a normal distribution with a mean of 1 and a variance of 0.2; the initial state of the battery of the controlled vehicle power system is ; the termination state flag ; initialize the storage space to store the training data, and the sample capacity of the training data is 2e5; record the initialized simulation environment as , that is, the model calculates the model state and reward value at the next moment according to the current model state and control action; (3) For the controlled vehicle, obtain its state vector at the current time ; (4) Obtain the action vector controlled by the vehicle power system and perform the action in the simulation model: ; (5) judging whether it is a termination state: if not, storing the data tuple as a sample in the storage space , updating the current time environment state ; otherwise, , storing the data tuple as a sample in the storage space , and executing step (2) once; (6) When the storage space If the number of samples in the storage space is greater than or equal to 100, step (7) is performed, otherwise step (3) is performed. (7) From storage space 256 samples were randomly selected from the middle: Calculate the update target of the evaluation model: （11） (8) Update the Q network parameters using gradient descent , : （12） (9) Adopting gradient descent update strategy model parameters : （13） (10) update , : ; (11) updating the number of parameter updates: ; if , then return to step (3), otherwise perform step (12); (12) saving energy management strategy model and parameters thereof , complete the fusion of multi-vehicle interactive perception energy management strategy training.

8. The method of claim 1, wherein the method further comprises: The method for evaluating and implementing the energy management strategy online comprises the following steps: obtaining the state vector of the controlled vehicle and the surrounding multi-vehicle system in each control cycle , calculating the mean value of the action vector as the control action , sending the control action to the power system of the controlled vehicle and executing the control action to complete the online application of the strategy.

Images