Networked hybrid electric vehicle energy management method fusing multi-vehicle motion interactive perception

Abstract

The invention discloses a networked hybrid electric vehicle energy management method fusing multi-vehicle motion interactive perception, and the method comprises the steps: collecting multi-vehicle system time sequence driving information in a certain time window in a self-vehicle perception range, constructing a time sequence driving information encoder based on an LSTM network, carrying out the feature coding of the time sequence driving information of each vehicle, and carrying out the feature coding of the time sequence driving information of each vehicle; obtaining a time sequence driving information feature vector; establishing a multi-vehicle interaction relation topological graph in a self-vehicle sensing range in combination with the completely undirected graph; cutting a multi-vehicle interaction relation topological graph, and constructing a multi-vehicle interaction perception model and a hybrid electric vehicle'driving behavior-comprehensive energy consumption 'simulation model; based on the model, obtaining an energy management strategy based on maximum entropy deep reinforcement learning; a driving behavior-comprehensive energy consumption simulation model, an energy management strategy model and an evaluation model of the energy management strategy model are initialized, and the energy management strategy model fusing multi-vehicle interactive perception is trained through a reinforcement learning SoftActor-Critc algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile energy management, and in particular to an energy management method for networked hybrid electric vehicles integrating multi-vehicle motion interactive perception. Background Art

[0002] Hybrid vehicles, with their energy efficiency, environmental friendliness, and long driving range, are currently a mainstream technology solution for new energy vehicles. Ensuring the real-time coordination and efficient operation of multiple power sources in hybrid systems through energy management is key to achieving energy conservation, emission reduction, and performance improvements. Currently, with the increase in on-board computing power and the rapid development of artificial intelligence technology, intelligent optimization algorithms such as deep reinforcement learning have been gradually applied to hybrid vehicle energy management, addressing the weak adaptability and suboptimality of traditional energy management algorithms. However, urban road traffic usually has characteristics such as dense traffic flow and variable driving conditions, which brings huge challenges to the real-time and dynamic energy management of hybrid power systems. For example: (1) The interaction of multiple vehicles in dynamic and dense urban roads makes the driving scene of hybrid vehicles more complex, and the driving conditions are highly uncertain, resulting in the stability of energy management optimization effects that needs to be improved; (2) Traditional optimization-based energy management strategies are difficult to establish a mapping relationship between complex multi-vehicle motion interactions and efficient energy management strategies, and the degree of utilization of networked traffic information by energy management strategies needs to be improved; (3) Ordinary feedforward neural network-based intelligent energy management strategies usually only support fixed-dimensional state information input, and are difficult to apply to time-varying characteristics such as the uncertain number of other vehicles in a multi-vehicle system and the uncertain relative motion relationship between vehicles. Therefore, for hybrid vehicles, how to design an energy management strategy with multi-vehicle motion interaction perception capabilities, strong adaptability to high-dynamic driving conditions on urban roads, and stable optimization performance is a technical problem that needs to be solved to achieve deep energy saving of networked hybrid vehicles. Summary of the invention

[0003] The present invention proposes an energy management method for a networked hybrid electric vehicle that integrates multi-vehicle motion interactive perception to solve the technical problems in the background technology.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for energy management of a connected hybrid electric vehicle integrating multi-vehicle motion interaction perception includes the following steps:

[0006] S100: Collecting time-series driving information of multiple vehicles within a certain time window within the vehicle's perception range, constructing a time-series driving information encoder based on an LSTM network, performing feature encoding on the time-series driving information of each vehicle, and outputting its time-series driving information feature vector;

[0007] S200. Use the time-sequential driving information feature vector as a node, and combine a complete undirected graph to establish a multi-vehicle interaction relationship topology graph within the self-vehicle perception range;

[0008] S300. Crop the multi-vehicle interaction relationship topology graph to construct a multi-vehicle interaction perception model;

[0009] S400. Construct a hybrid vehicle driving behavior-comprehensive energy consumption simulation model;

[0010] S500. Based on the multi-vehicle interaction perception model and the driving behavior-comprehensive energy consumption simulation model, construct an energy management strategy model based on maximum entropy deep reinforcement learning;

[0011] S600. Initialize the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model based on the reinforcement learning SoftActor-Critc algorithm, and train and fuse them to obtain the final energy management strategy model.

[0012] Furthermore, the method for constructing a time-sequential driving information encoder using an LSTM network in step S100 of the present invention includes:

[0013] Perform feature encoding on the time-sequential driving information observations of each vehicle:

[0014]

[0015] Among them, and are the hidden state and cell state of the LSTM network unit respectively, is the information observation.

[0016] Furthermore, the method for the multi-vehicle interaction relationship topology graph within the self-vehicle perception range in step S200 of the present invention is as follows:

[0017] Model each vehicle in the multi-vehicle system as a node of the interaction relationship topology graph, and use the feature encoding vector of the time-sequential driving information observation of each vehicle node as its attribute vector;

[0018] Model the interaction relationship between vehicles in the multi-vehicle system as an edge of the interaction relationship topology graph, and define an edge vector to describe the importance of the interaction relationship between vehicles; Use a complete undirected graph to model the interaction relationship of the multi-vehicle system within the self-vehicle perception range, and the key elements of the model are defined as follows:

[0019] (1) Nodes of the interaction relationship topology graph and the attribute vectors of the nodes:

[0020] Define vehicle i (i = 1, 2..., N) as a node of the graph network G, and define the feature encoding vector of the time-sequential driving information observation as the attribute vector of the node

[0021] The set of all vehicle node attributes is denoted as The graph centered on the i-th vehicle is denoted as G i ;

[0022] (2) Edges and edge vectors of the interaction relationship topology graph:

[0023] The interaction relationships between vehicles serve as the edges of graph G, and E = {w1, w2, …, w N} is the edge set notation, and the edge vector w i is used to describe the importance of the interaction relationship between vehicle i and other vehicles.

[0024] Furthermore, the method for constructing the multi-vehicle interaction perception model in step S300 of the present invention includes:

[0025] Construct a self-attention network to calculate the importance of multi-vehicle interaction relationships;

[0026] Adopt the hard attention and reparameterization methods to prune the multi-vehicle interaction relationship topology graph to obtain a reduced multi-vehicle interaction relationship subgraph;

[0027] Adopt the soft self-attention method to extract the motion interaction correlation representations of other vehicles related to each node vehicle;

[0028] Among them, the multi-vehicle interaction perception model further includes:

[0029] (1) LSTM encoding layer: The sequential driving information encoder established in step S100, whose input is the attribute vectors of each node of the multi-vehicle interaction relationship topology graph and the output is The encoder shares parameters among multiple vehicles;

[0030] (2) Random hard self-attention layer, which uses the vector formed by the node attribute set in graph G as the input sequence, and the input sequence length N is determined according to the number of nodes in the real-time multi-vehicle system rather than a fixed value; the output is the attention score matrix A attn , value matrix V, and binary attention score matrix A hard ;

[0031] Generate the query matrix Q using the learnable weight matrix = W qry X, key matrix K = W q X, value matrix V = W k X, and calculate the attention score matrices A v and A′ attn through dot product: attn :

[0032]

[0033] Among them, softmax(·) is normalized column by column, and A attn and A′ attn are used to describe the importance of the interaction relationship between vehicles within the perception range of the ego vehicle. A′ attn is the edge set of the multi-vehicle interaction relationship topology graph established in step one;

[0034] At this layer, a binary attention score matrix A between vehicles is learned through hard self-attention, hard the multi-vehicle interaction relationship topology graph is pruned to extract effective vehicle-vehicle interaction relationships, and the Gumble-Softmax reparameterization method is introduced to obtain a random sampling output A under the premise of differentiability hard :

[0035]

[0036] Among them, gum(·) is the Gumble-Softmax function mapped column by column, taking a column vector as input, and its output formula is as follows

[0037]

[0038] Among them, α i is the attention score, τ is the temperature coefficient, and the Gumbel noise term g i =-log(-log(u i )), where u i ~U(0,1);

[0039] (3) Deterministic soft self-attention layer, whose input is A attn , A hard , value matrix V, and the output is the multi-vehicle motion interaction correlation representation

[0040] First of all, according to the attention weight A hard , irrelevant vehicle nodes in graph G are removed and irrelevant edges are disconnected: if the element value of the i-th row and j-th column of A hard is 0, it is determined that there is no interaction relationship between vehicle i and j, otherwise it is considered that there is an interaction relationship between the two vehicles, so as to clarify the vehicle nodes with effective interaction relationships and obtain a multi-vehicle interaction subgraph G after primary reduction sub ;

[0041] After that, based on the multi-vehicle interaction subgraph G sub after primary reduction, soft self-attention is used to calculate the importance of effective interaction relationships: calculate the normalized attention score matrix A soft to represent the importance of effective interaction relationships;

[0042]

[0043] Finally, selectively extract the driving state information of surrounding vehicles and construct a multi-vehicle motion interaction correlation representation B ′ :

[0044] B′ = Attention(Q qry , K, V) = VA soft (6)

[0045] where K is the key matrix and V is the value matrix.

[0046] Furthermore, the "driving behavior - comprehensive energy consumption" simulation model of the hybrid vehicle in step S400 of the present invention includes: an urban road network model, a driving behavior model, and a quasi-static energy consumption model of the hybrid vehicle;

[0047] Among them, the urban road network model: construct an urban road network based on the urban traffic simulation platform SUMO;

[0048] The driving behavior model: construct a driving behavior model of the vehicle based on the following models preset in the SUMO system: a car-following model, a lane-changing model, and an intersection model. It is a driving behavior decision-making unit that takes the surrounding traffic state of the vehicle as input and outputs the vehicle acceleration and vehicle speed;

[0049] Construct a quasi-static energy consumption model of the hybrid vehicle based on the vehicle longitudinal dynamics. Taking the vehicle speed and acceleration as input, update and calculate the vehicle power system state and travel energy consumption. The specific composition is as follows:

[0050] The vehicle dynamics model: calculate the required driving / braking power P of the whole vehicle according to the desired acceleration and current vehicle speed output by the driving behavior model req :

[0051] P req = F req v, F req = F j + F f + F i + F ω , F j = ma acc , F f = mgf cosθ road ,

[0052] F i = mg sinθ road , F ω = C D A front v 2 / 21.15 (7)

[0053] where, F reqis the demand driving / braking force, v is the vehicle speed (m / s), m is the curb weight of the vehicle (kg), a acc is the acceleration (m / s 2 ), f is the rolling resistance coefficient, θ road is the road gradient, C D is the air resistance coefficient, A front is the frontal area of the vehicle (m 2 ), g is the acceleration due to gravity;

[0054] Engine fuel consumption model: According to the control actions (engine speed, engine torque) of the energy management strategy, calculate the instantaneous fuel consumption of the engine at the current moment:

[0055] m f = T eng n eng f eng (T eng , n eng ) / (3.6×10 6 )(8)

[0056] where, m f is the fuel consumption per unit time, T eng is the engine output torque, n eng is the engine speed, f eng (T eng , n eng ) is the specific fuel consumption table of the engine (g / kW·h), obtained through engine bench tests. The given control actions should not exceed the engine's external characteristics and maximum speed limit, otherwise they are infeasible control actions;

[0057] Drive motor / generator power consumption model: When the vehicle's demand driving / braking power, engine speed, and torque are given, calculate the power P mot / gen required by the drive motor / generator at the current moment according to the powertrain configuration:

[0058] P mot / gen = P req η -sgn(Preq) (9)

[0059] where, η is the motor efficiency characteristic table obtained through bench tests. When the given control actions are applied, if the speed and torque of the drive motor / generator exceed its external characteristics, it is an infeasible control action;

[0060] Battery internal resistance model: The power of the power battery provides / recovers the drive / generation power of all drive motors / generators in the hybrid system. Thus, based on the power P batt of the power battery, calculate the current I batt of the power battery and the rate of change of the state of charge SoC:

[0061]

[0062] Among them, U oc is the open-circuit voltage of the battery pack, R is the internal resistance of the battery pack, and Q batt is the capacity of the battery pack. Under a given control action, if the current breaks through the maximum value, I batt is calculated as an imaginary number, or the charging and discharging power of the battery breaks through the power limit, all of which are infeasible control actions.

[0063] Furthermore, the key elements of the energy management strategy method summarized in step S500 of the present invention include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model;

[0064] Among them, the implementation methods of the key elements are defined as follows:

[0065] (1) Define the state space S: Define the state vector s as the observable quantities {o1, o2,..., o power} that describe the state o of the vehicle power system and the motion state of the multi-vehicle system N cascaded to form a vector, s ∈ S, and perform Min-Max normalization on each element in the state vector. The observable quantities and constraints of the vehicle power system state describe the vehicle power system state, including but not limited to the state of charge SoC of the power battery, the degree ΔSoC of the state of charge deviating from the expected steady value, the vehicle demand drive / brake power P and torque T req , demand acceleration acc, vehicle speed v, external characteristic constraints of power system components, charging and discharging power limits of power batteries, and travel time; req

[0066] Here, the constraint state representation of the power system and components is also added to the state space to assist in policy learning and avoid unsafe control actions; the observable quantities of the multi-vehicle system motion state are the observable quantities of the multi-vehicle system time-series driving information described in step S100;

[0067] When the SoC in the state vector is lower than the specified lower limit value or higher than the specified upper limit value, or when the travel time reaches the maximum value T, this state is considered a termination state;

[0068] (2) Define the action space A: The action quantity is determined according to the power system configuration and its control degrees of freedom, including but not limited to taking the engine speed T eng and engine torque W eng as the action vector a = [T eng , W eng , a ∈ A;

[0069] (3) Reward function r;

[0070] First, take the weighted sum of the engine's instantaneous fuel consumption \(m\) eng and the deviation degree of the power battery SoC as the reward function: \(r = -\kappa m\) eng -\(\lambda\Delta SoC\) 2 , where \(r\leq0\); among them, \(\kappa\) and \(\lambda\) are weight factors, and \(\kappa\) is a conditional value: \(\kappa = 0.6\) if and only if the engine operating efficiency is higher than the set threshold, otherwise \(\kappa = 1\);

[0071] Next, perform Min - Max normalization and limit on \(r\): where \(clip(x,x\) min ,x\) max ) truncates the input \(x\) with the preset boundary values \(x\) min , \(x\) max . Due to the influence of the action amount, \(r\) min cannot be accurately determined, so a smaller reward threshold is selected as \(r\) min ;

[0072] Define trigger events to adjust the reward function \(r\): Define a constant penalty term \(\rho = 0.4\). When and only when engine start - stop and infeasible control action events (the occurrence times is \(n\)) occur, \(r = r - n\rho\); Define a constant \(bonus = 8\). When the trip ends and the SoC is still within the stable range, \(r = r + bonus\). If the trip is terminated in advance or the trip ends but the SoC exceeds the stable range, \(r = r - bonus\);

[0073] (4) Energy management strategy model \(a\sim\pi\) θ (\(\cdot|s\)): Its structure is composed of a multi - vehicle interaction perception model and a fully - connected feed - forward neural network. The parameter set is \(\theta\). The model inputs the multi - vehicle system observables \(\{o_1,o_2,\cdots,o\}\) N into the multi - vehicle interaction perception model and outputs the multi - vehicle interaction correlation representation \(b'_1\) related to the ego - vehicle. Then, cascade \(b'_1\) with the power system state \(o\) power and input it into a three - layer fully - connected feed - forward neural network. The number of neurons in the output layer is equal to twice the dimension of the action vector, and the mean \(\mu\) θ and variance \(\sigma\) θ of each action amount are output respectively; During the training process, the action vector is obtained through re - parameterized sampling: \(a\) θ (\(s,\xi\))=\(\tanh(\mu\) θ ,\(\sigma\) θ \odot\xi)\), where \(\xi\) is Gaussian noise and \(\odot\) is element - wise multiplication;

[0074] (5) Energy management strategy evaluation model \(Q\): Establish four fully - connected neural networks with the same structure as the energy management strategy evaluation model. The parameter sets are \(\Phi_1\), \(\Phi_2\), \(\Phi\) targ1 , \(\Phi\) targ2; The number of neurons in each network input layer is equal to the sum of the dimensions of the state vector and the action vector, the number of neurons in the output layer is 1; there are 4 hidden layers, and the number of neurons decreases layer by layer; the activation function of the output layer is a linear activation function, and the other layers are the rectified linear unit ReLU.

[0075] Further, in step S600 of the present invention, the method for training and fusing to obtain the final energy management strategy model includes: initializing the "driving behavior - comprehensive energy consumption" simulation model, initializing the energy management strategy model and its evaluation model, generating energy management strategy training data, interactively training the energy management strategy, saving the energy management strategy, evaluating and online applying the energy management strategy.

[0076] Further, the method for training and fusing to obtain the final energy management strategy model of the present invention further includes:

[0077] (1) Initialize the policy model parameters θ, the energy management strategy evaluation model parameters Φ1, Φ2 using the Xavier initialization method, and assign Φ targ1 = Φ1, Φ targ2 = Φ2, the update rate τ = 0.005, the discount rate γ = 0.99, the entropy coefficient α = 0.01, the maximum number of updates N max , the number of updates k = 0;

[0078] (2) Initialize the "driving behavior - comprehensive energy consumption" simulation environment: configure the traffic flow in the road network, the average speed of each vehicle is 72 km / h, and the speed adjustment coefficient follows a normal distribution with a mean of 1 and a variance of 0.2; the initial SoC of the battery of the controlled vehicle's power system = 0.5; the termination state flag d = 0; initialize the storage space to store training data, the sample size of the training data is 2e5; denote the initialized simulation environment as s′, r = HEV(a, s), that is, the model calculates the next moment's model state and reward value according to the current model state and control action;

[0079] (3) For the controlled vehicle, obtain its state vector s at the current moment;

[0080] (4) Obtain the action vector a ~ π θ (·|s) of the controlled vehicle's power system control, and execute the action in the simulation model: s′, r = HEV(a, s);

[0081] (5) Determine whether s’ is a termination state: if not, store the data tuple (s, a, r, s’, d) as a sample in the storage space Update the environmental state s = s’ at the current moment; otherwise, d = 1, store the data tuple (s, a, r, s’, d) as a sample in the storage space and execute step (2) once;

[0082] (6) When the sample size in the storage space is greater than or equal to 100, step (7) is executed; otherwise, step (3) is executed;

[0083] (7) Randomly sample 256 samples from the storage space : B = {(s, a, r, s')}, and calculate the evaluation model update target:

[0084]

[0086] (8) Update the Q-network parameters Φ1, Φ2 using gradient descent:

[0087]

[0088] (9) Update the policy model parameter θ using gradient descent:

[0089]

[0090] (10) Update Φ targ1 、Φ targ2 : Φ targi = τΦ targi +(1 - τ)τΦ i , i = 1, 2;

[0091] (11) Update the parameter update times: k = k + 1; if k ≤ N max , then return to step (3), otherwise execute step (12);

[0092] (12) Save the energy management policy model a ~ π θ (·|s) and its parameter θ, and complete the training of the energy management policy that integrates multi-vehicle interaction perception.

[0093] Furthermore, the implementation method of the energy management policy evaluation and online application of the present invention includes: in each control cycle, obtain the state vector s of the controlled vehicle and the multi-vehicle system around it, calculate π θ (·|s) to obtain the mean value μ θ of the action vector as the control action a, and send it to the power system of the controlled vehicle to execute the control action, thereby completing the online application of the policy.

[0094] As can be seen from the above technical solutions, the present invention realizes the energy management of hybrid electric vehicles with multi-vehicle motion interaction perception, supports the controlled vehicle to extract the motion information representation of other vehicles related to the energy-saving optimization of its hybrid system in real time, and can improve the optimization performance and working condition adaptability of the energy management of hybrid electric vehicles in a multi-vehicle system on high-dynamic and dense urban roads. Description of the Drawings

[0095] Figure 1 This is the flowchart of the energy management method for a connected hybrid electric vehicle that integrates multi-vehicle motion interaction perception;

[0096] Figure 2 This is the topological diagram of the multi-vehicle interaction relationship;

[0097] Figure 3 This is the schematic diagram of the multi-vehicle interaction perception model;

[0098] Figure 4 This is the schematic diagram of the network structure of the stochastic energy management strategy that integrates multi-vehicle interaction perception. Specific implementation manner

[0099] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0100] As Figure 1 shown, the energy management method for a connected hybrid electric vehicle that integrates multi-vehicle motion interaction perception described in this embodiment specifically includes the following steps:

[0101] S100. Within the perception range of the host vehicle, collect the sequential driving information of the multi-vehicle system within a certain time window, construct a sequential driving information encoder based on the LSTM network, and perform feature encoding on the sequential driving information of each vehicle to output its sequential driving information feature vector;

[0102] S200. Let the sequential driving information feature vector be a node, and combine a complete undirected graph to establish a topological graph of the multi-vehicle interaction relationship within the perception range of the host vehicle;

[0103] S300. Prune the topological graph of the multi-vehicle interaction relationship to construct a multi-vehicle interaction perception model;

[0104] S400. Construct a hybrid electric vehicle driving behavior-comprehensive energy consumption simulation model;

[0105] S500. Based on the multi-vehicle interaction perception model and the driving behavior-comprehensive energy consumption simulation model, construct an energy management strategy model based on maximum entropy deep reinforcement learning;

[0106] S600. Initialize the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model based on the reinforcement learning SoftActor-Critc algorithm, and train and integrate to obtain the final energy management strategy model.

[0107] The following will explain each step in detail:

[0108] S100. Collect the sequential driving information of the multi-vehicle system within a certain time window within the self-vehicle perception range. Construct a sequential driving information encoder based on the LSTM network, and perform feature encoding on the sequential driving information of each vehicle to output its sequential driving information feature vector;

[0109] The self-vehicle perception range is determined by the real-time transmission range of the vehicle networking communication data of the self-vehicle and surrounding vehicles. At time t, when the number of vehicles within the self-vehicle perception range is N, the sequential driving information observation of vehicle i (i = 1, 2,..., N) is which is used to describe the driving state observations such as vehicle position, longitudinal and lateral speeds, and heading angle. At time t, collect the sequential driving information observations of the multi-vehicle system within the self-vehicle perception range within the historical time window T:

[0110] The method for constructing a sequential driving information encoder using the LSTM network includes:

[0111] Perform feature encoding on the sequential driving information observations of each vehicle:

[0112]

[0113] Among them, and are the hidden state and cell state of the LSTM network unit respectively.

[0114] S200. Let the sequential driving information feature vector be the node, and combine a complete undirected graph to establish a multi-vehicle interaction relationship topology graph within the self-vehicle perception range;

[0115] The method for establishing a multi-vehicle interaction relationship topology graph using a complete undirected graph includes:

[0116] Model each vehicle in the multi-vehicle system as a node of the interaction relationship topology graph, and use the feature encoding vector of the sequential driving information observation of each vehicle node as its attribute vector;

[0117] Model the interaction relationship between vehicles in the multi-vehicle system as an edge of the interaction relationship topology graph, and define an edge vector to describe the importance of the interaction relationship between vehicles;

[0118] As Figure 2 shown, use a complete undirected graph to model the interaction relationship of the multi-vehicle system within the self-vehicle perception range. The key elements of the model are defined as follows:

[0119] (1) Nodes of the interaction relationship topology graph and the attribute vectors of the nodes:

[0120] Define vehicle i (i = 1, 2..., N) as a node of the graph network G.

[0121] Define the feature encoding vector of the time-series driving information observation as the attribute vector of the node

[0122] The set of all vehicle node attributes is denoted as The graph centered on the i-th vehicle is denoted as G i 。

[0123] (2) Edges and edge vectors of the interaction relationship topology graph:

[0124] The interaction relationship between vehicles is used as the edge of graph G, and E = {w1, w2, …, w N} is the edge set notation, and the edge vector w i is used to describe the importance of the interaction relationship between vehicle i and other vehicles.

[0125] S300. Crop the multi-vehicle interaction relationship topology graph to construct a multi-vehicle interaction perception model;

[0126] The method for constructing the multi-vehicle interaction perception model includes: constructing a self-attention network to calculate the importance of the multi-vehicle interaction relationship; using the hard attention and reparameterization method to crop the multi-vehicle interaction relationship topology graph to obtain a reduced multi-vehicle interaction relationship sub-graph; thus, using the soft self-attention method to extract the other vehicle motion interaction correlation representations related to each node vehicle.

[0127] As Figure 3 shown, the multi-vehicle interaction perception model further includes:

[0128] (1) LSTM encoding layer: The time-series driving information encoder established in step S100. Its input is each attribute vector of the multi-vehicle interaction relationship topology graph and the output is The encoder shares parameters among multiple vehicles.

[0129] (2) Stochastic hard self-attention layer. It uses the vector constituted by the node attribute set in graph G as the input sequence, and the length N of the input sequence can be determined according to the number of nodes in the real-time multi-vehicle system rather than a fixed value; the output is the attention score matrix A attn , value matrix V, and binary attention score matrix A hard .

[0130] First, use the learnable weight matrix to generate the query matrix Q qry = W q X, key matrix K = W k X, value matrix V = W v X, and calculate the attention score matrix A attn and A′ attn through dot product:

[0131]

[0132] Among them, softmax(·) normalizes by column, and A attn and A′ attn are used to describe the importance of the interaction relationships between vehicles within the perception range of the ego vehicle. A′ attn is the edge set of the multi-vehicle interaction relationship topology graph established in Step 1.

[0133] Next, in this layer, a binary attention score matrix A hard (0 or 1, indicating whether there is an effective interaction between two vehicles) is learned through hard self-attention to prune the multi-vehicle interaction relationship topology graph and extract effective vehicle-vehicle interaction relationships. Specifically, the Gumble-Softmax reparameterization method is introduced to obtain a random sampling output A hard :

[0134]

[0135] Among them, gum(·) is the Gumble-Softmax function mapped by column, taking a column vector as input, and its output is shown as follows.

[0136]

[0137] Among them, α i is the attention score, τ is the temperature coefficient, and the Gumbel noise term g i =-log(-log(u i )), u i ~U(0,1).

[0138] (3) Deterministic soft self-attention layer. Its input is A attn 、A hard 、the value matrix V, and the output is the multi-vehicle motion interaction correlation representation

[0139] First of all, according to the attention weight A hard , irrelevant vehicle nodes in graph G are removed and irrelevant edges are disconnected: if the element value of the i-th row and j-th column of A hard is 0, it is determined that there is no interaction relationship between vehicle i and j, otherwise it is considered that there is an interaction relationship between the two vehicles, so as to clarify the vehicle nodes with effective interaction relationships and obtain a multi-vehicle interaction subgraph G sub .

[0140] After that, based on the multi-vehicle interaction subgraph G sub after primary reduction, soft self-attention is used to calculate the importance of effective interaction relationships: calculate the normalized attention score matrix Asoft , which is used to characterize the importance of the effective interaction relationship.

[0141]

[0142] Finally, selectively extract the driving state information of surrounding vehicles and construct a multi-vehicle motion interaction correlation representation B ′ :

[0143] B′ = Attention(Q qry , K, V) = VA soft (6)

[0144] S400. Build a simulation model of "driving behavior - comprehensive energy consumption" for hybrid vehicles;

[0145] The simulation model of "driving behavior - comprehensive energy consumption" for hybrid vehicles includes: an urban road network model, a driving behavior model, and a quasi-static energy consumption model for hybrid vehicles;

[0146] (1) Urban road network model: Build an urban road network based on the urban traffic simulation platform SUMO.

[0147] (2) Driving behavior model: Based on the car-following model, lane-changing model, and intersection model preset in the SUMO system, build a driving behavior model for vehicles, which is a driving behavior decision-making unit. Taking the traffic state around the vehicle as the input, it outputs the vehicle acceleration and vehicle speed.

[0148] (3) Build a quasi-static energy consumption model for hybrid vehicles based on vehicle longitudinal dynamics. Taking the vehicle speed and acceleration as the input, update and calculate the state of the vehicle power system and the energy consumption of the journey. The specific composition is as follows.

[0149] (3.1) Vehicle dynamics model. According to the expected acceleration and current vehicle speed output by the driving behavior model, calculate the required driving / braking power P of the whole vehicle req :

[0150] P req = F req v, F req = F j + F f + F i + F ω , F j = ma acc , F f = mgf cosθ road ,

[0151] F i = mg sinθ road , F ω = C D A frontv 2 / 21.15 (7)

[0152] Among them, F req is the demand drive / braking force, v is the vehicle speed (m / s), m is the curb weight of the whole vehicle (kg), a acc is the acceleration (m / s 2 ), f is the rolling resistance coefficient, θ road is the road gradient, C D is the air resistance coefficient, A front is the frontal area of the vehicle (m 2 ), and g is the acceleration due to gravity.

[0153] (3.2) Engine fuel consumption model. According to the control actions (engine speed, engine torque) of the energy management strategy, calculate the engine instantaneous fuel consumption at the current moment:

[0154] m f = T eng n eng f eng (T e g , n eng ) / (3.6×10 6 )(8)

[0155] Among them, m f is the fuel consumption per unit time, T eng is the engine output torque, n eng is the engine speed, f eng (T eng , n eng ) is the engine specific fuel consumption table (g / kW·h), obtained through engine bench tests. The given control actions should not exceed the engine external characteristics and maximum speed limit, otherwise they are infeasible control actions.

[0156] (3.3) Drive motor / generator power consumption model. When the vehicle demand drive / brake power, engine speed and torque are given, calculate the drive motor / generator power P mo t / gen required at the current moment according to the powertrain configuration:

[0157]

[0158] Among them, η is the motor efficiency characteristic table obtained through bench tests. When the given control actions are taken, if the speed and torque of the drive motor / generator exceed its external characteristics, they are infeasible control actions.

[0159] (3.4) Battery internal resistance model. The power battery provides / recovers the driving / generating power of all drive motors / generators in the hybrid power system, and thus calculates the current I of the power battery according to the power P of the power battery batt , and calculates the rate of change of the state of charge SoC: batt

[0160]

[0161] where U oc is the open-circuit voltage of the battery pack, R is the internal resistance of the battery pack, and Q batt is the capacity of the battery pack. Under a given control action, if the current breaks through the maximum value, I batt is calculated as an imaginary number, or the charging / discharging power of the battery breaks through the power limit, all are infeasible control actions.

[0162] S500. Based on the multi-vehicle interaction perception model and the "driving behavior - comprehensive energy consumption" simulation model, an energy management strategy model based on maximum entropy deep reinforcement learning is constructed

[0163] The key elements of the energy management strategy model include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model;

[0164] Among them, the implementation methods for defining the key elements are as follows:

[0165] (1) Define the state space S. Define the state vector s as the observed variables {o1, o2,..., o power} that describe the state of the vehicle power system o N} after cascading, s ∈ S, and perform Min-Max normalization on each element in the state vector. The state of the vehicle power system describes the observed variables, constraints, etc. of the vehicle power system, including but not limited to the state of charge SoC of the power battery, the degree of deviation of the state of charge from the expected steady value ΔSoC, the driving / braking power P req and torque T req of the whole vehicle demand, the demand acceleration acc, the vehicle speed v, the external characteristic constraints of the power system components, the charging / discharging power limit of the power battery, the travel time, etc.; In particular, the constraint state representation of the power system and its components should be added to the state space here to help policy learning and avoid unsafe control actions. The observed variables of the motion state of the multi-vehicle system are the observed variables of the multi-vehicle system time-series driving information described in step S100.

[0166] When the SoC in the state vector is lower than the lower limit (0.2) or higher than the upper limit (0.9), or the travel time reaches the maximum value T, this state is considered a termination state.

[0167] (2) Define the action space A.

[0168] The action quantity is determined according to the configuration of the power system and its control degrees of freedom, including but not limited to taking the engine speed T eng , the engine torque W eng as the action vector a = [T eng , W eng , a ∈ A.

[0169] (3) Reward function r.

[0170] First, take the weighted sum of the engine instantaneous fuel consumption m eng and the deviation degree of the power battery SoC as the reward function: r = -km eng -λΔSoC 2 , r ≤ 0. Among them, κ and λ are weighting factors, and κ is a conditional value: κ = 0.6 when and only when the engine operating efficiency is higher than the set threshold, otherwise κ = 1.

[0171] Next, perform Min - Max normalization and limit on r: where clip(x, x min , x max ) truncates the input x with the preset boundary values x min , x max . Due to being affected by the action quantity, r min cannot be accurately determined, and a smaller reward threshold (less than 90% of the above r) can be selected as r min .

[0172] Finally, define trigger events to adjust the reward function r: Define a constant penalty term ρ = 0.4. When and only when events such as engine start - stop and infeasible control actions occur (the number of occurrences is n), r = r - nρ; Define a constant bonus = 8. When the trip ends and the SoC is still within the stable range, r = r + bonus. If the trip terminates early or the trip ends but the SoC exceeds the stable range, r = r - bonus.

[0173] (4) Energy management strategy model a ∼ π θ (·|s). Its structure is composed of the multi - vehicle interaction perception model and the fully - connected feed - forward neural network described in step three. As Figure 4 shown, the parameter set is θ. The model inputs the multi - vehicle system observations {o1, o2, …, o N} into the multi - vehicle interaction perception model, outputs the multi - vehicle interaction correlation representation b′1 related to the ego - vehicle, and then cascades b′1 with the power system state o power and inputs them into a three - layer fully - connected feed - forward neural network. The number of neurons in the output layer is equal to twice the dimension of the action vector, and the means μ θ and variances σ of each action quantity are respectively outputθ During the training process, the action vector is obtained by reparameterized sampling: a θ (s, ξ) = tanh(μ θ , σ θ ⊙ ξ), where ξ is Gaussian noise and ⊙ is element-wise multiplication.

[0174] (5) Energy management strategy evaluation model Q. Four fully connected neural networks with the same structure are established as the energy management strategy evaluation models, and the parameter sets are Φ1, Φ2, Φ targ1 , Φ targ2 respectively. The number of neurons in the input layer of each network is equal to the sum of the dimensions of the state vector and the action vector, and the number of neurons in the output layer is 1; there are 4 hidden layers, and the number of neurons decreases layer by layer; the activation function of the output layer is a linear activation function, and the other layers are rectified linear unit functions ReLU.

[0175] S600. Initialize the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model based on the reinforcement learning SoftActor-Critc algorithm, and train and fuse them to obtain the final energy management strategy model.

[0176] The method for training and fusing the energy management strategy model with multi-vehicle interactive perception based on the reinforcement learning SoftActor-Critc (SAC) algorithm further includes: initializing the "driving behavior-comprehensive energy consumption" simulation model, the energy management strategy model and its evaluation model, initializing the "driving behavior-comprehensive energy consumption" simulation model, initializing the energy management strategy model and its evaluation model, generating energy management strategy training data, performing interactive training on the energy management strategy, saving the energy management strategy, evaluating and online applying the energy management strategy. The specific implementation method is as follows:

[0177] (1) Initialize the parameters θ of the policy model, the parameters Φ1, Φ2 of the energy management strategy evaluation model using the Xavier initialization method, and assign Φ targ1 = Φ1, Φ targ2 = Φ2, the update rate τ = 0.005, the discount rate γ = 0.99, the entropy coefficient α = 0.01, the maximum number of updates N max , and the number of updates k = 0;

[0178] (2) Initialize the "driving behavior-comprehensive energy consumption" simulation environment. Configure the traffic flow in the road network, the average speed of each vehicle is 72 km / h, and the speed adjustment coefficient follows a normal distribution (mean = 1, variance 0.2); the initial SoC of the power system battery of the controlled vehicle is 0.5; the termination state flag d = 0; initialize the storage space To store training data, the sample size of the training data is 2e5; Denote the initialized simulation environment as s′, r = HEV(a, s), that is, the model calculates the model state and reward value at the next moment according to the current model state and control action.

[0179] (3) For the controlled vehicle, obtain its state vector s at the current moment;

[0180] (4) Obtain the action vector a ~ π θ (·|s) of the power system control of the controlled vehicle, and execute the action in the simulation model: s′, r = HEV(a, s);

[0181] (5) Judge whether s’ is a terminal state: If not, store the data tuple (s, a, r, s’, d) as a sample in the storage space Update the environmental state s = s’ at the current moment; Otherwise, d = 1, store the data tuple (s, a, r, s’, d) as a sample in the storage space And execute step (2) once;

[0182] (6) When the sample size in the storage space is greater than or equal to 100, execute step (7), otherwise execute step (3);

[0183] (7) Randomly sample 256 samples from the storage space : B = {(s, a, r, s’)}, calculate the evaluation model update target:

[0184]

[0185] (8) Update the Q-network parameters Φ1, Φ2 using gradient descent:

[0186]

[0187] (9) Update the policy model parameter θ using gradient descent:

[0188]

[0189] (10) Update Φ targ1 、Φ targ2 : Φ targi = τΦ targi +(1 - τ)Φ i , i = 1, 2;

[0190] (11) Update the parameter update times: k = k + 1; If k ≤ N max , then return to step (3), otherwise execute step (12);

[0191] (12) Save the energy management strategy models a to π θ (·|s) and its parameter θ to complete the training of the energy management strategy that fuses multi-vehicle interactive perception.

[0192] Implementation method of policy evaluation and online application: In each control cycle, obtain the state vector s of the controlled vehicle and its surrounding multi-vehicle system, and calculate π θ (·|s) to obtain the mean μ of the action vector θ As the control action a, send it to the power system of the controlled vehicle and execute the control action to complete the online application of the policy.

[0193] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive Solid State Disk (SSD)), etc.

[0194] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element. Each embodiment in this specification is described in a related manner, and the same or similar parts among the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

[0195] The above embodiments are only used to illustrate the technical solutions of the present invention, not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A networked hybrid vehicle energy management method integrating multi-vehicle motion interaction perception, characterized in that It includes the following steps: S100. Within the perception range of the host vehicle, collect the time-series driving information of the multi-vehicle system within a certain time window, construct a time-series driving information encoder based on the LSTM network, perform feature encoding on the time-series driving information of each vehicle, and output its time-series driving information feature vector; S200. Let the time-series driving information feature vector be a node, and combine a complete undirected graph to establish a multi-vehicle interaction relationship topology graph within the perception range of the host vehicle; S300. Prune the multi-vehicle interaction relationship topology graph to construct a multi-vehicle interaction perception model; S400. Construct a hybrid vehicle driving behavior-comprehensive energy consumption simulation model; S500. Based on the multi-vehicle interaction perception model and the driving behavior-comprehensive energy consumption simulation model, construct an energy management strategy model based on maximum entropy deep reinforcement learning; S600. Initialize the driving behavior-comprehensive energy consumption simulation model and the energy management strategy model based on the reinforcement learning SoftActor-Critc algorithm, and train and fuse them to obtain the final energy management strategy model.

2. The energy management method of the connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 1, wherein The method for constructing a time-series driving information encoder by the LSTM network in step S100 includes: Perform feature encoding on the observed values of the time-series driving information of each vehicle: Among them, and are the hidden state and cell state of the LSTM network unit respectively, is the information observation quantity.

3. The energy management method for a connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 1, wherein The method for the multi-vehicle interaction relationship topology graph within the perception range of the host vehicle in step S200 is: Model each vehicle in the multi-vehicle system as a node of the interaction relationship topology graph, and use the feature encoding vector of the observed values of the time-series driving information of each vehicle node as its attribute vector; Model the interaction relationship between vehicles in the multi-vehicle system as the edge of the interaction relationship topology graph, and define an edge vector to describe the importance of the interaction relationship between vehicles; Use a complete undirected graph to model the interaction relationship of the multi-vehicle system within the perception range of the host vehicle. The key elements of the model are defined as follows: (1) Nodes of the interaction relationship topology graph and attribute vectors of the nodes: Define vehicle i (i = 1, 2..., N) as the node of graph network G, and define the feature encoding vector of the time-series driving information observation as the attribute vector of the node The set of all vehicle node attributes is denoted as The graph centered on the i-th vehicle is denoted as G i ; (2) Edges of the interaction relationship topology graph and edge vectors: The interaction relationship between vehicles is used as the edge of graph G, and E = {w1, w2, …, w N} is denoted as the edge set. The edge vector w i is used to describe the importance of the interaction relationship between vehicle i and other vehicles.

4. The energy management method for a connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 1, characterized in that, The method for constructing a multi-vehicle interaction perception model in step S300 includes: Construct a self-attention network to calculate the importance of multi-vehicle interactions; Use the hard attention and reparameterization method to prune the multi-vehicle interaction relationship topology graph to obtain a reduced multi-vehicle interaction relationship subgraph; Use the soft self-attention method to extract the motion interaction correlation representations of other vehicles related to each node vehicle; Among them, the multi-vehicle interaction perception model also includes: (1) LSTM Encoding Layer: The temporal driving information encoder established in step S100, whose input is each attribute vector of the multi-vehicle interaction relationship topology graph The output is The encoder shares parameters among multiple vehicles; (2) Random hard self-attention layer, which uses the vector formed by the set of node attributes in graph G as the input sequence, and the length N of the input sequence is determined according to the number of nodes in the real-time multi-vehicle system rather than a fixed value; A attn , value matrix V, binary attention score matrix A hard ; Using a learnable weight matrix Generate query matrix Q qry = W q X, key matrix K = W k X, value matrix V = W v X, calculate the attention score matrix A through dot product attn and A' attn : Among them, softmax(·) is normalized column by column, and A attn and A' attn are used to describe the importance of the interaction relationship between vehicles within the perception range of the ego vehicle. A' attn is the edge set of the multi-vehicle interaction relationship topology graph established in the first step; In this layer, a binary attention score matrix A between vehicles is learned through hard self-attention hard , the multi-vehicle interaction relationship topology graph is cropped to extract effective vehicle-vehicle interaction relationships, and the Gumble-Softmax reparameterization method is introduced to obtain a random sampling output A on the premise of differentiability hard : Among them, gum(·) is the Gumble-Softmax function mapped by columns, taking a column vector as the input, and its output formula is as follows where α i is the attention score, τ is the temperature coefficient, and the Gumbel noise term g i = -log(-log(u i )), where u i ~ U(0, 1); (3) Deterministic soft self-attention layer, with input A attn 、A hard 、value matrix V, and output is the multi-vehicle motion interaction correlation representation First, based on the attention weight A hard , remove the irrelevant vehicle nodes in graph G and disconnect the irrelevant edges: If the element value of the hard i-th row and j-th column of A is 0, it is determined that there is no interaction relationship between vehicles i and j, otherwise it is considered that there is an interaction relationship between the two vehicles, so as to clarify the vehicle nodes with effective interaction relationships and obtain the multi-vehicle interaction sub-graph G sub that has undergone primary reduction; After that, on the basis of the multi-vehicle interaction subgraph G of the primary reduction sub Soft self-attention is used to calculate the importance of effective interaction relationships: calculate the normalized attention score matrix A soft , which is used to represent the importance of effective interaction relationships; Finally, selectively extract the driving state information of surrounding vehicles to construct a multi-vehicle motion interaction correlation representation B′: B′ = Attention(Q qry , K, V) = VA soft (6) Where K is the key matrix and V is the value matrix.

5. The energy management method for a connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 1, characterized in that, The "driving behavior-comprehensive energy consumption" simulation model of the hybrid vehicle in step S400 includes: an urban road network model, a driving behavior model, and a quasi-static energy consumption model of the hybrid vehicle; Among them, the urban road network model: Construct an urban road network based on the urban traffic simulation platform SUMO; The driving behavior model: Based on the following-following model, lane-changing model, and intersection model preset in the SUMO system, construct a driving behavior model of the vehicle, which is a driving behavior decision-making unit, taking the traffic state around the vehicle as the input, and outputting the vehicle acceleration and vehicle speed; Build a quasi-static energy consumption model for a hybrid electric vehicle based on vehicle longitudinal dynamics, with vehicle speed and acceleration as inputs, and update the calculation of the vehicle power system state and travel energy consumption. The specific composition is as follows: Vehicle dynamics model: Calculate the required driving / braking power P of the vehicle based on the desired acceleration and current vehicle speed output by the driving behavior model req : P req = F req v, F req = F j + F f + F i + F ω , F j = ma acc , F f = mgf cosθ road , F i = mg sinθ road , F ω = C D A front v 2 / 21.15 (7) Among them, F req is the demand driving / braking force, v is the vehicle speed (m / s), m is the curb weight of the whole vehicle (kg), a acc is the acceleration (m / s 2 ), f is the rolling resistance coefficient, θ road is the road gradient, C D is the air resistance coefficient, A front is the frontal area of the vehicle (m 2 ), and g is the acceleration due to gravity; Engine fuel consumption model: Calculate the instantaneous fuel consumption of the engine at the current moment according to the control actions (engine speed, engine torque) of the energy management strategy: m f = T eng n eng f eng (T eng , n eng ) / (3.6×10 6 ) (8) where m f is the fuel consumption per unit time, T eng is the engine output torque, n eng is the engine speed, f eng (T eng , n eng ) is the specific fuel consumption map of the engine (g / kW·h), obtained through engine bench tests. The given control actions should not exceed the engine's external characteristics and maximum speed limit, otherwise they are infeasible control actions; Drive motor / generator power consumption model: When the required driving / braking power of the vehicle, engine speed, and torque are given, calculate the power P of the drive motor / generator required at the current moment according to the powertrain configuration mot / gen : Among them, η is the motor efficiency characteristic table obtained through bench tests. When a given control action is applied, if the speed and torque of the drive motor / generator exceed its external characteristics, it is an infeasible control action; Battery internal resistance model: The power battery provides / recovers the driving / generating power of all drive motors / generators in the hybrid power system, and thus calculates the current I of the power battery according to the power P of the power battery batt , and calculates the current I of the power battery batt , state of charge SoC change rate: Among them, U oc is the open-circuit voltage of the battery pack, R is the internal resistance of the battery pack, and Q batt is the capacity of the battery pack. Under a given control action, if the current breaks through the maximum value, I batt is calculated as an imaginary number, or the charging and discharging power of the battery breaks through the power limit, all of which are infeasible control actions.

6. The networked hybrid vehicle energy management method integrating multi-vehicle motion interaction perception according to claim 1, characterized in that, The key elements summarized in step S500 of the energy management strategy model include: state space, action space, reward function, energy management strategy model, and energy management strategy evaluation model; Among them, the implementation method of the key elements is defined as follows: (1) Define the state space S: Define the state vector s as the observable quantities {o1, o2, …, o} of the vehicle power system state o power and the motion state of the multi-vehicle system N} after cascading. s ∈ S, and perform Min-Max normalization on each element in the state vector. The vehicle power system state describes the observable quantities and constraints of the vehicle power system state, including but not limited to the state of charge SoC of the power battery, the degree of deviation of the state of charge from the expected steady value ΔSoC, the vehicle demand drive / brake power P req and torque T req , the demand acceleration acc, vehicle speed v, the external characteristic constraints of the power system components, the charge and discharge power limit of the power battery, and travel time; Here, the constraint state characterization of the power system and components is added to the state space to help strategy learning and avoid unsafe control actions; the observable quantity of the multi-vehicle system motion state is the observable quantity of the multi-vehicle system time-series driving information described in step one; When the SoC in the state vector is lower than the specified lower limit value or higher than the specified upper limit value, or when the travel time reaches the maximum value T, this state is considered a termination state; (2) Define the action space A: The action quantity is determined according to the configuration of the dynamic system and its control degrees of freedom, including but not limited to the engine speed T eng , the engine torque W eng as the action vector a = [T eng , W eng , a ∈ A; (3) Reward function r; First, take the weighted sum of the instantaneous fuel consumption m of the engine eng and the deviation degree of the SoC of the power battery as the reward function: r = -κm eng -λΔSoC 2 , r ≤ 0; where κ and λ are weight factors, and κ is a conditional value: κ = 0.6 if and only if the engine working efficiency is higher than the set threshold, otherwise κ = 1; Next, perform Min-Max normalization and limit on r: where clip(x, x min , x max ) truncates the input x with the preset boundary values x min , x max . Due to the influence of the action amount, r min cannot be accurately determined, and a smaller reward threshold is selected as r min ; Define trigger events to adjust the reward function r: Define a constant penalty term ρ = 0.

4. When and only when engine start / stop and infeasible control action events (the number of occurrences is n) occur, r = r - nρ; Define a constant bonus = 8. When the travel ends and the SoC is still within the stable range, r = r + bonus. If the travel terminates early or the travel ends but the SoC exceeds the stable range, r = r - bonus; (4) Energy management strategy model a~π θ (·|s): Its structure is composed of a multi-vehicle interaction perception model and a fully connected feedforward neural network. The parameter set is θ. The model inputs the multi-vehicle system observables {o1, o2, …, o N} into the multi-vehicle interaction perception model, outputs the multi-vehicle interaction correlation representation b′1 related to the ego vehicle, and then cascades b′1 with the power system state o power and inputs them into a three-layer fully connected feedforward neural network. The number of neurons in the output layer is equal to twice the dimension of the action vector, and the means μ θ and variances σ θ of each action quantity are respectively output; during the training process, the action vector is obtained through reparameterized sampling: where ξ is Gaussian noise and ⊙ is element-wise multiplication; (5) Energy management strategy evaluation model Q: Four fully connected neural networks with the same structure are established as the energy management strategy evaluation model, and the parameter sets are Φ1, Φ2, Φ targ1 , Φ targ2 respectively; the number of neurons in the input layer of each network is equal to the sum of the dimensions of the state vector and the action vector, and the number of neurons in the output layer is 1; there are 4 hidden layers, and the number of neurons decreases layer by layer; the activation function of the output layer is a linear activation function, and the other layers are rectified linear unit ReLU.

7. The energy management method for a connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 1, wherein In step S600, the training and fusion to obtain the final energy management strategy model includes: initialization of the "driving behavior - comprehensive energy consumption" simulation model, initialization of the energy management strategy model and its evaluation model, generation of energy management strategy training data, interactive training of the energy management strategy, saving of the energy management strategy, evaluation and online and application of the energy management strategy.

8. The energy management method for a connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 7, characterized in that The method for training and fusion to obtain the final energy management strategy model also includes: (1) Initialize the parameters θ of the policy model, the parameters Φ1 and Φ2 of the energy management strategy evaluation model using the Xavier initialization method, and assign Φ targ1 = Φ1, Φ targ2 = Φ2, the update rate τ = 0.005, the discount rate γ = 0.99, the entropy coefficient α = 0.01, the maximum number of updates N max , the number of updates k = 0; (2)Initialization of the "driving behavior - comprehensive energy consumption" simulation environment: Configure the traffic flow within the road network, with the average vehicle speed being 72 km / h and the vehicle speed adjustment coefficient following a normal distribution with a mean of 1 and a variance of 0.2; the initial SoC of the power system battery of the controlled vehicle is 0.5; the termination state flag d = 0; initialize the storage space to store the training data, with the training data sample size being 2e5; denote the initialized simulation environment as s′, r = HEV(a, s), that is, the model calculates the next - moment model state and reward value based on the current model state and control action; (3) For the controlled vehicle, obtain its state vector s at the current moment; (4) Obtain the action vectors a~π for the control of the power system of the vehicle to be controlled θ (·|s), and execute the action in the simulation model: s′, r = HEV(a, s); (5) Determine whether s’ is a terminal state: If not, store the data tuple (s, a, r, s’, d) as a sample in the storage space Update the environmental state s = s’ at the current moment; otherwise, d = 1, and store the data tuple (s, a, r, s’, d) as a sample in the storage space And execute step (2) once; When the sample size in the storage space is greater than or equal to 100, step (7) is executed; otherwise, step (3) is executed. (7) Randomly sample 256 samples from the storage space : B = {(s, a, r, s')}, and calculate the update objective of the evaluation model: (8) Update the Q-network parameters Φ1 and Φ2 using gradient descent: (9) Update the policy model parameters θ using gradient descent: (10) Update Φ targ1 , Φ targ2 : Φ targi = τΦ targi + (1 - τ)Φ i , i = 1, 2; (11) Update parameter Update times: k = k + 1; If k ≤ N max , then return to step (3), otherwise execute step (12); (12) Save the energy management strategy models a to π θ (·|s) and its parameter θ to complete the training of the energy management strategy that fuses multi-vehicle interaction perception.

9. The energy management method for a connected hybrid vehicle integrating multi-vehicle motion interaction perception according to claim 7, characterized in that, The implementation method for evaluating and online applying an energy management strategy includes: within each control cycle, obtaining the state vector s of the vehicle under control and the multi-vehicle system around it, and calculating π θ (·|s) to obtain the mean value μ of the action vector θ as the control action a, sending it to the power system of the vehicle under control and executing the control action to complete the online application of the strategy.

Images