Vehicle control method, vehicle, and computer-readable storage medium

Abstract

The application discloses a vehicle control method, a vehicle and a computer readable storage medium, and relates to the technical field of vehicle control. The method comprises the following steps: acquiring vehicle state data and environment perception data of a vehicle; determining target demand power of the vehicle based on the vehicle state data and the environment perception data; analyzing the vehicle state data and the target demand power by using a target power distribution network to obtain a power distribution result, wherein the power distribution result is used to represent first power to be output by an engine of the vehicle and second power to be output by a battery of the vehicle, and the target power distribution network is a deep neural network obtained through reinforcement learning training; and generating a vehicle control instruction based on the power distribution result. The application solves the technical problem that the adaptive capability of the vehicle energy management strategy is poor in the prior art, thereby affecting the economy of the vehicle.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and more specifically, to a vehicle control method, a vehicle, and a computer-readable storage medium. Background Technology

[0002] Hybrid electric vehicles, as an important technological route bridging traditional fuel vehicles and pure electric vehicles, have energy management strategies that directly determine the vehicle's fuel economy, power responsiveness, and battery life.

[0003] Current mainstream energy management methods are mostly based on rule-based control or optimization control (such as the equivalent fuel consumption minimization strategy). These methods rely on pre-set expert experience or global optimization models and have a certain degree of effectiveness under fixed operating conditions. However, when faced with frequently changing driving intentions, complex and ever-changing traffic environments, battery state drift, and nonlinear and time-varying factors such as temperature fluctuations in real-world driving conditions, traditional methods generally suffer from insufficient adaptive capabilities and rigid strategies.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a vehicle control method, a vehicle, and a computer-readable storage medium to at least address the technical problem in the related art where the vehicle energy management strategy has poor adaptive capability, thereby affecting vehicle economy.

[0006] According to one aspect of the present invention, a vehicle control method is provided, comprising: acquiring vehicle state data and environmental perception data; determining a target power demand of the vehicle based on the vehicle state data and environmental perception data; analyzing the vehicle state data and target power demand using a target power allocation network to obtain a power allocation result, wherein the power allocation result is used to characterize a first power to be output by the vehicle's engine and a second power to be output by the vehicle's battery, and the target power allocation network is a deep neural network trained by reinforcement learning; and generating vehicle control commands based on the power allocation result.

[0007] Optionally, the vehicle control method further includes: using a target power evaluation network corresponding to the target power allocation network to perform an expected evaluation of the power allocation result and obtain an expected value; in response to the expected value being greater than the expected threshold, performing a feasibility verification of the power allocation result and obtaining a verification result; in response to the verification result indicating that the power allocation result does not meet the feasibility conditions, updating the power allocation result based on the feasibility conditions and obtaining an updated power allocation result.

[0008] Optionally, vehicle status data includes: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, and transmission system temperature; environmental perception data includes: road slope and road curvature. Based on the vehicle status data and environmental perception data, the target power demand of the vehicle is determined, including: determining the initial power demand based on the accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, road slope, and road curvature; predicting mechanical loss power based on the transmission system temperature; and determining the target power demand based on the initial power demand and mechanical loss power.

[0009] Optionally, the vehicle control method further includes: creating an initial power allocation network and a first copy network, wherein the initial power allocation network and the first copy network have the same network structure and network parameters; creating an initial power evaluation network and a second copy network, wherein the initial power evaluation network and the second copy network have the same network structure and network parameters; and using training data, the first copy network, and the second copy network, performing reinforcement learning training on the initial power allocation network and the initial power evaluation network to obtain a target power allocation network.

[0010] Optionally, the vehicle control method further includes: generating multiple sets of empirical data in a simulation environment, wherein any set of empirical data includes: first data, second data, third data, and fourth data, the first data being used to describe the vehicle state and power demand generated at the current simulation time step, the second data being used to characterize the first power allocation result obtained by the initial power allocation network after analyzing the first data, the third data being used to characterize the reward value of the first power allocation result, and the fourth data being used to describe the updated vehicle state and updated power demand obtained based on the first power allocation result; and randomly selecting a preset number of data groups from the multiple sets of empirical data to form training data.

[0011] Optionally, the initial power allocation network and the initial power evaluation network are trained using the training data, the first copy network, and the second copy network to obtain the target power allocation network. This includes: updating the network parameters of the initial power evaluation network using the training data, the first copy network, and the second copy network to obtain the target power evaluation network; and updating the network parameters of the initial power allocation network using the training data and the target power evaluation network to obtain the target power allocation network.

[0012] Optionally, the vehicle control method further includes: in response to the network parameter update count of the initial power evaluation network reaching a preset update count, updating the network parameters of the second replica network according to the current network parameters of the updated initial power evaluation network to obtain the updated second replica network; and in response to the network parameter update count of the initial power allocation network reaching a preset update count, updating the network parameters of the first replica network according to the current network parameters of the updated initial power allocation network to obtain the updated first replica network.

[0013] Optionally, the vehicle control method further includes: superimposing exploration noise onto the second data to obtain data to be processed; and calculating the reward value of the data to be processed to obtain the third data.

[0014] According to another aspect of the present invention, a vehicle control system is also provided, comprising: an acquisition module for acquiring vehicle state data and environmental perception data; a determination module for determining a target power demand of the vehicle based on the vehicle state data and environmental perception data; an analysis module for analyzing the vehicle state data and target power demand using a target power allocation network to obtain a power allocation result, wherein the power allocation result is used to characterize a first power to be output by the vehicle's engine and a second power to be output by the vehicle's battery, and the target power allocation network is a deep neural network trained by reinforcement learning; and a generation module for generating vehicle control commands based on the power allocation result.

[0015] According to another aspect of the present invention, a vehicle is also provided, comprising: a memory storing an executable program; and a processor for running the executable program, wherein the executable program executes the vehicle control method described in any of the preceding embodiments.

[0016] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the vehicle control method described in any of the above.

[0017] This invention provides a vehicle control method, comprising: acquiring vehicle state data and environmental perception data; determining the target power demand of the vehicle based on the vehicle state data and environmental perception data; analyzing the vehicle state data and target power demand using a target power allocation network to obtain a power allocation result, wherein the power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery, and the target power allocation network is a deep neural network trained by reinforcement learning; and generating vehicle control commands based on the power allocation result. This invention first acquires vehicle status data and environmental perception data to comprehensively characterize the vehicle's current operating conditions, providing a high-dimensional and dynamic input basis for subsequent energy allocation. Second, based on the vehicle status data and environmental perception data, it determines the target power demand, ensuring that power allocation is always driven by actual driving needs. Furthermore, a target power allocation network trained through reinforcement learning serves as the core control unit. By analyzing the vehicle status data and target power demand, it generates power allocation results to determine the first power output from the engine and the second power output from the vehicle's battery. This power allocation result highly matches the vehicle's operating conditions, significantly improving the adaptability of the energy allocation management strategy to varying operating conditions. Finally, based on the power allocation results, vehicle control commands are generated, achieving effective vehicle energy management and improving overall vehicle fuel economy. In summary, this invention achieves the technical effect of improving the adaptability of vehicle energy management strategies to varying operating conditions, thus solving the technical problem in related technologies where poor adaptability of vehicle energy management strategies affects vehicle economy. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a flowchart of a vehicle control method according to one embodiment of the present invention;

[0020] Figure 2 This is a network structure diagram of an intelligent agent for power distribution according to one embodiment of the present invention;

[0021] Figure 3 This is a flowchart illustrating the training process of an agent according to one embodiment of the present invention;

[0022] Figure 4 This is a structural block diagram of a vehicle control system according to one embodiment of the present invention;

[0023] Figure 5This is a schematic diagram of a vehicle according to one embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] According to an embodiment of the present invention, an embodiment of a vehicle control method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] This invention provides a vehicle control method. Figure 1 This is a flowchart of a vehicle control method according to one embodiment of the present invention, such as... Figure 1 As shown, the vehicle control method includes the following steps:

[0028] Step S101: Obtain vehicle status data and environmental perception data.

[0029] Step S102: Determine the target power requirement of the vehicle based on vehicle status data and environmental perception data;

[0030] Step S103: Analyze the vehicle state data and target power demand using the target power allocation network to obtain the power allocation result. The power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery. The target power allocation network is a deep neural network trained by reinforcement learning.

[0031] Step S104: Based on the power allocation results, generate vehicle control commands.

[0032] The aforementioned acquisition of vehicle status data and environmental perception data refers to multi-dimensional data reflecting the vehicle's operating status and external environmental conditions, collected in real time through an onboard sensor network. This multi-dimensional data serves as the input basis for subsequent energy management decisions.

[0033] The aforementioned vehicle status data includes, but is not limited to: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, transmission system temperature, engine output power, engine coolant temperature, power battery output power, power battery surface temperature, and battery state of charge.

[0034] The aforementioned environmental perception data includes, but is not limited to, road slope and road curvature.

[0035] In one optional embodiment, vehicle status data is collected collaboratively with the vehicle controller via an onboard sensor network. Specifically, the accelerator pedal and brake pedal openings are monitored in real-time by potentiometers or Hall effect sensors; gear position information is acquired by a transmission position sensor and transmitted via a CAN bus; the vehicle speed change rate is derived from wheel speed sensor signals through differential calculation; the transmission system temperature is measured by temperature probes mounted on the transmission or differential housing; engine output power and coolant temperature are calculated and directly acquired by a crankshaft speed sensor, an intake manifold pressure sensor, and an engine coolant temperature sensor, respectively; and the power battery's output power, surface temperature, and state of charge are estimated in real-time using the battery management system's built-in current sensor, voltage sampling circuit, and temperature array, combined with ampere-hour integration and open-circuit voltage lookup table algorithms, and then uniformly uploaded to the energy management control unit via the CAN bus.

[0036] In one optional embodiment, environmental perception data can be acquired in real time through an onboard multi-sensor fusion system. Specifically, this includes: measuring the vehicle's longitudinal and lateral acceleration and pitch angle using a high-precision inertial measurement unit; combining this with position and elevation information provided by a global navigation satellite system; and deriving the current road slope through coordinate transformation and differential calculations. Simultaneously, the system collects road texture and geometric features from a front-facing camera or LiDAR; extracts the three-dimensional curvature change of the road centerline using a point cloud fitting algorithm; and then outputs continuous and smooth curvature values ​​through filtering and interpolation. Furthermore, the system can receive high-precision map preset information from roadside units or cloud map services via the vehicle-to-everything (V2X) network, enabling multi-source data complementarity and redundancy verification, thereby ensuring the accuracy and robustness of environmental perception under complex operating conditions.

[0037] The target power requirement mentioned above refers to the total drive power required by the vehicle in the next control cycle. The target power requirement serves as a constraint boundary for vehicle energy distribution.

[0038] In one optional embodiment, the target power demand is not obtained by a linear mapping of throttle opening, but rather by integrating the driver's accelerator pedal change rate, target vehicle speed error, gradient resistance, air resistance, rolling resistance, and auxiliary load demands such as air conditioning, and jointly deducing it from a physical driving model and an intent recognition model. Optionally, the driver's intent is quantified by the derivative value and rate of change of the accelerator pedal opening. If the rate of change of the pedal exceeds a threshold and continues to rise, it is determined to be a "rapid acceleration" intent, and a dynamic compensation term will be added to the target power to improve responsiveness. If a vehicle decelerating or a speed limit sign is detected ahead, predictive power reduction is triggered based on forward perception data to reduce energy consumption. At the same time, the power of auxiliary loads such as air conditioning, heated seats, and in-vehicle electronic devices is also included in the total demand in real time to ensure that energy management covers the entire vehicle's power needs.

[0039] In one optional embodiment, the calculation of the target power demand employs a hierarchical modeling structure: the bottom layer is a physical model based on vehicle dynamics equations, calculating the base power required to maintain the current vehicle speed; the middle layer is a driver intent recognition module, which classifies pedal sequences into intents (e.g., smooth, moderate, rapid) using a lightweight long short-term memory network; the top layer is a comprehensive decision-making module, which dynamically weights the power demand based on intent category and environmental slope factor to output the final target power demand. This step ensures that the energy management strategy is always guided by real driving needs, rather than preset values ​​under fixed operating conditions, providing an accurate target benchmark for subsequent intelligent allocation.

[0040] The aforementioned target power allocation network refers to an end-to-end power allocation network model obtained through reinforcement learning training.

[0041] In one optional embodiment, a deep neural network constructed based on the Deep Deterministic Policy Gradient (DDPG) algorithm is used as the target power allocation network. The target power allocation network directly maps from high-dimensional state inputs to continuous power allocation outputs, achieving coordinated optimization of engine and battery power.

[0042] Optionally, the target power distribution network takes vehicle state data (such as engine output power, engine coolant temperature, power battery output power, power battery surface temperature, and battery state of charge) and target power demand as input, and after processing through two fully connected hidden layers, outputs two continuous values: the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery.

[0043] The above-mentioned generation of vehicle control commands based on power allocation results refers to converting the power allocation results into control signals that can be executed by the vehicle power system, driving the engine controller and the battery management system to execute in coordination.

[0044] Optionally, during the control command generation process, dynamic power coupling verification is implemented: if the battery temperature is too high, the discharge power is forcibly reduced and the engine power ratio is increased; if the battery charge is below a threshold (e.g., 20%), the charging priority mode is triggered, and the motor regeneration power is limited to ensure the charge is restored; if the engine is currently in a start-stop state, the engine is started in advance and the warm-up power is reserved to avoid frequent start-stop affecting smoothness.

[0045] This invention provides a vehicle control method, comprising: acquiring vehicle state data and environmental perception data; determining the target power demand of the vehicle based on the vehicle state data and environmental perception data; analyzing the vehicle state data and target power demand using a target power allocation network to obtain a power allocation result, wherein the power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery, and the target power allocation network is a deep neural network trained by reinforcement learning; and generating vehicle control commands based on the power allocation result. This invention first acquires vehicle status data and environmental perception data to comprehensively characterize the vehicle's current operating conditions, providing a high-dimensional and dynamic input basis for subsequent energy allocation. Second, based on the vehicle status data and environmental perception data, it determines the target power demand, ensuring that power allocation is always driven by actual driving needs. Furthermore, a target power allocation network trained through reinforcement learning serves as the core control unit. By analyzing the vehicle status data and target power demand, it generates power allocation results to determine the first power output from the engine and the second power output from the vehicle's battery. This power allocation result highly matches the vehicle's operating conditions, significantly improving the adaptability of the energy allocation management strategy to varying operating conditions. Finally, based on the power allocation results, vehicle control commands are generated, achieving effective vehicle energy management and improving overall vehicle fuel economy. In summary, this invention achieves the technical effect of improving the adaptability of vehicle energy management strategies to varying operating conditions, thus solving the technical problem in related technologies where poor adaptability of vehicle energy management strategies affects vehicle economy.

[0046] The vehicle control method in the embodiments of this application will be further described below.

[0047] Optionally, the vehicle control method further includes:

[0048] Step S1031: Using the target power evaluation network corresponding to the target power allocation network, perform an expected evaluation of the power allocation result to obtain the expected value;

[0049] Step S1032: In response to the expected value being greater than the expected threshold, a feasibility check is performed on the power allocation result to obtain the check result;

[0050] Step S1033: In response to the verification result indicating that the power allocation result does not meet the feasibility conditions, the power allocation result is updated based on the feasibility conditions to obtain the updated power allocation result.

[0051] The aforementioned target power evaluation network is an assessment network matched with the target power allocation network. The role of the target power evaluation network is to quantify and score the expected long-term returns of the power allocation results output by the target power allocation network, thereby determining whether the current allocation strategy is reasonable.

[0052] In one alternative embodiment, the target power evaluation network is modeled using a deep neural network. The input is the input and output of the target power allocation network, and the output is a scalar value (i.e., the Q value), which represents the expected value of the cumulative reward in the future after the vehicle performs the power allocation action in the current state, i.e., the "expected value".

[0053] Optionally, this expected value comprehensively reflects the combined benefits of multi-objective optimization, including fuel economy, battery degradation, driving smoothness, and battery state of charge maintenance.

[0054] In one optional embodiment, the target power evaluation network employs a three-layer fully connected neural network structure, with the hidden layer activation function using an Exponential Linear Unit (ELU) to ensure that the gradient remains non-zero in the negative range, preventing neuron "death" and improving evaluation stability. The closer the Q-value of the target power evaluation network output is to 0 (or higher), the better the power allocation strategy. This process requires no external database or lookup table; it is entirely evaluated online end-to-end by the neural network, possessing real-time performance and adaptability.

[0055] Optionally, when the expected value exceeds the preset "expectation threshold", it is determined that the power allocation result has a high potential for comprehensive benefit under the reinforcement learning evaluation framework, and the subsequent engineering physical constraint verification process will be triggered.

[0056] Optionally, the "expected threshold" is a system-calibrated dynamic threshold value. Its initial value is obtained by statistically analyzing the Q-value distribution of the optimal strategy under a large number of working conditions in offline simulation, and it can be adaptively adjusted according to the training phase to balance exploration and utilization.

[0057] Furthermore, after triggering the verification, it is verified whether the power distribution results meet the hard constraints of the vehicle's powertrain and battery management system. The verification content includes, but is not limited to: whether the engine output power is within the allowable operating range (e.g., 1.5kW to 45kW), whether the battery charging and discharging power exceeds its maximum allowable power curve (dynamically determined by looking up a table based on the battery's state of charge and temperature), and whether the total power meets the vehicle's requirements (power conservation), etc.

[0058] For example, if the battery discharge power in the power allocation result is 8kW, although the Q value of the power allocation result is high, the allocated battery discharge power is close to the maximum discharge capacity boundary of the battery at high temperature. At this time, the verification result is "infeasible", triggering the subsequent correction mechanism.

[0059] When the feasibility verification fails, the original allocation scheme is not discarded directly, but a "constraint-based power correction mechanism" is activated to dynamically adjust the original result.

[0060] The aforementioned feasibility conditions refer to the hard constraints on the vehicle's powertrain and battery management systems.

[0061] In one optional embodiment, the type of constraint violation is first identified (e.g., battery power exceeding limits, engine power being too low, etc.). Then, adjustments are made sequentially according to constraint priority to obtain an updated power allocation result. For example, if the battery discharge power exceeds the limit, the battery power is proportionally reduced and the engine power is increased accordingly without reducing the overall vehicle power requirement, while ensuring the engine power remains within its high-efficiency range. If the engine power has reached its upper limit, battery power limiting operation is forcibly initiated, and excess demand is smoothly compensated through energy recovery strategies (e.g., regenerative braking pre-allocation). All adjustments are solved quickly using linear programming or quadratic programming, with computation time controlled within 5ms, meeting real-time control requirements.

[0062] Optionally, the updated power allocation results can be evaluated a second time until a feasible solution that satisfies both physical constraints and has high expected value is obtained.

[0063] The above steps effectively resolve the typical contradiction of reinforcement learning strategies being "theoretically optimal but not feasible in engineering" in real systems, and achieve a smooth transition from "data-driven decision-making" to "safe and controllable execution," providing a highly robust solution for energy management under complex operating conditions.

[0064] Optionally, vehicle status data includes: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, and transmission system temperature; environmental perception data includes: road gradient and road curvature. Based on the vehicle status data and environmental perception data, the target power requirement of the vehicle is determined, including:

[0065] Step S1021: Determine the initial power requirement based on the accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, road gradient, and road curvature.

[0066] Step S1022: Predict mechanical loss power based on transmission system temperature;

[0067] Step S1023: Determine the target power demand based on the initial power demand and mechanical loss power.

[0068] The aforementioned accelerator pedal opening refers to the percentage of physical displacement when the driver presses the accelerator pedal, detected by a potentiometer or magnetoresistive sensor, and used to characterize the driver's immediate intention to demand power output.

[0069] The aforementioned brake pedal opening reflects the driver's intention to apply braking torque to the braking system. It is obtained by a pressure sensor or displacement sensor, and its value is directly related to the potential for brake energy recovery.

[0070] The aforementioned vehicle gear position refers to the current gear status of the transmission system, provided by the transmission control unit. It is used to determine the current driving mode and gear ratio setting of the vehicle, which directly affects the power transmission efficiency and power demand.

[0071] The aforementioned vehicle speed change rate is a digital expression of the vehicle's instantaneous acceleration, obtained by differential calculation of the wheel speed sensor signal. It is used to identify the driver's rapid acceleration, rapid deceleration, or cruise stability behavior and is an important dynamic indicator for predicting fluctuations in power demand.

[0072] The aforementioned transmission system temperature refers to the thermal state of key transmission components such as transmission oil, differential, or drive shaft. It is measured by an embedded temperature sensor and used to evaluate the nonlinear characteristics of mechanical friction loss as a function of temperature. It is an important input for calculating energy loss compensation.

[0073] The aforementioned road gradient refers to the angle of inclination of the road section in front of the vehicle relative to the horizontal plane. It is calculated by the inertial measurement unit in combination with high-precision map preload data. Positive values ​​indicate uphill and negative values ​​indicate downhill, which directly affects the power required for the vehicle to overcome the component of gravity.

[0074] The aforementioned road curvature refers to the degree of curvature of the road's longitudinal direction, usually expressed as the change in steering angle per unit distance (unit: 1 / m). It is obtained by fusing forward-looking cameras, millimeter-wave radar, and high-precision maps to predict the additional power demand of vehicles in curves due to increased lateral inertia and tire rolling resistance.

[0075] In one alternative embodiment, a multi-input, nonlinearly mapped power decision function is constructed to calculate the total wheel-end power (i.e., initial demand power) required by the vehicle to maintain the desired motion state under the current driving intention and road conditions.

[0076] Optionally, the power decision function can be an analytical expression based on a physical model or a lightweight neural network. When obtaining the initial power demand using a lightweight neural network, the accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, road gradient, and road curvature are used as inputs to the lightweight neural network. This lightweight neural network is trained based on historical vehicle data.

[0077] Optionally, predicting mechanical loss power based on transmission system temperature refers to establishing a predictive model between transmission system temperature and internal friction loss, and using the feedback value from temperature sensors to estimate in real time the power consumed by mechanical movements such as gear meshing, bearing rolling, and lubricating oil shearing under the current operating conditions.

[0078] In one optional embodiment, the predictive model is constructed based on laboratory bench calibration data. This model is used to analyze the transmission system temperature and predict mechanical power loss. For example, when the transmission system temperature is low (e.g., during cold starts, below 30°C), the lubricating oil viscosity is high and the oil film thickness is large, leading to a significant increase in internal shear resistance, and the mechanical power loss may reach 8%–12% of the rated output power; when the temperature rises to the normal operating range (60°C–90°C), the lubricating oil fluidity improves, and the power loss stabilizes at 3%–5%.

[0079] Furthermore, the target demand power is determined based on the initial demand power and mechanical loss power. In an optional embodiment, the target demand power is the sum of the initial demand power and the mechanical loss power.

[0080] It's easy to understand that an energy management system must ensure that the total power provided by the power source (engine or battery) not only meets driving needs but also compensates for energy lost due to reduced transmission efficiency; otherwise, the actual vehicle speed will deviate from the driver's intention. For example, when a vehicle is driving uphill in a cold mountainous area, the initial power demand increases significantly due to the combined effects of the gradient and low-temperature resistance. If the predicted compensation for mechanical power loss is ignored and power is allocated only according to the initial power demand, it will lead to excessively rapid battery discharge or frequent engine intervention, resulting in abnormal energy consumption and reduced driving smoothness. The precise generation of the target power demand enables the energy allocation strategy to have "system-level closed-loop compensation" capabilities, ensuring that the effective power output to the wheels is highly consistent with the driver's intention.

[0081] In the above steps, the thermal state of the transmission system is incorporated as a dynamic compensation factor into the power decision closed loop, which realizes high-precision, low-latency, and adaptive prediction of the vehicle's real power demand. This significantly improves the fuel economy and power response consistency of the energy management strategy under complex operating conditions, and provides core decision-making basis for the efficient operation of the hybrid system in extreme environments and dynamic driving scenarios.

[0082] Optionally, the vehicle control method further includes:

[0083] Step S1051: Create an initial power allocation network and a first replica network, wherein the initial power allocation network and the first replica network have the same network structure and network parameters;

[0084] Step S1052: Create an initial power evaluation network and a second replica network, wherein the initial power evaluation network and the second replica network have the same network structure and network parameters;

[0085] Step S1053: Using the training data, the first copy network, and the second copy network, perform reinforcement learning training on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network.

[0086] The input layer of the aforementioned initial power allocation network receives a multi-dimensional state vector consisting of vehicle state data and target power demand. Nonlinear features are extracted through two fully connected hidden layers. The output layer uses a tanh activation function to constrain the output value within the physically feasible range and combines an adaptive scaling factor to map it to the power range that the actual system can execute.

[0087] The aforementioned first replica network is an exact copy of the initial power allocation network. Its network structure, connection method, activation function, and parameter initialization method are completely identical to the initial network. However, it does not participate in gradient updates during training and is only used as the target network for calculating stable target values ​​in subsequent reinforcement learning. The existence of the replica network is to solve the "moving target" problem in reinforcement learning (i.e., if the initial power allocation network is directly used to predict the action of the next state, the estimated target Q value will oscillate violently with the real-time changes of the main network, thereby destroying the learning stability). By setting the parameters of the replica network as a mirror image of the initial network and gradually synchronizing them through a slow soft update mechanism, the generation of the target result can have temporal smoothness and low variance characteristics, significantly improving training convergence. In the embodiments of this application, the parameters of the first replica network are completely identical to those of the initial network in the early stage of training. In each training cycle, it only receives small parameter increments from the initial network through soft updates. Its update frequency is much lower than that of the main network, ensuring that its output always lags behind the current policy, forming a stable reference benchmark.

[0088] The aforementioned initial power evaluation network is used to assess the long-term cumulative reward (i.e., Q-value) obtained after performing a certain power allocation action under the current vehicle state and target power demand. The initial power evaluation network also consists of an input layer, two hidden layers (using the ELU activation function), and a single output unit, and its design goal is to approximate the true Bellman optimal Q-function. During training, the initial power evaluation network updates its parameters by minimizing the mean squared error between the predicted Q-value and the target Q-value.

[0089] The aforementioned second copy network is a mirror image of the initial power evaluation network. Its structure and parameters are identical to the power evaluation network, but it also does not participate in direct gradient updates; it serves only as the target network to calculate the target Q-value. The parameter updates of the second copy network employ a soft update strategy, meaning that after each update of the main network, its parameters are brought closer to the main network at a small update rate (e.g., 0.005). This delayed synchronization method maintains the correlation between the target network and the main network while effectively suppressing high-frequency oscillations during training. In this embodiment, the initial power evaluation network and the second copy network together constitute a dual-network evaluation system, ensuring that Q-value estimation possesses both learning capabilities and temporal consistency.

[0090] In one alternative embodiment, the initial power allocation network and the initial power evaluation network are jointly trained through experience replay and policy gradient optimization, ultimately obtaining a "target power allocation network" that can continuously output the optimal power allocation scheme under complex and dynamic hybrid driving conditions.

[0091] Optionally, the training data comes from the real-time interaction between the initial power allocation network and the virtual vehicle environment. Based on the training data, the first copy network, and the second copy network, reinforcement learning is performed on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network.

[0092] In the above steps, by constructing a dual-network architecture, autonomous learning and continuous optimization of the hybrid vehicle energy management strategy are achieved, breaking away from the limitations of traditional expert rules that rely on manual parameter tuning.

[0093] Optionally, the vehicle control method further includes:

[0094] Step S1054: In the simulation environment, multiple sets of empirical data are generated. Each set of empirical data includes: first data, second data, third data, and fourth data. The first data is used to describe the vehicle state and power demand generated at the current simulation time step. The second data is used to characterize the first power allocation result obtained by the initial power allocation network after analyzing the first data. The third data is used to characterize the reward value of the first power allocation result. The fourth data is used to describe the updated vehicle state and updated power demand obtained based on the first power allocation result.

[0095] Step S1055: Randomly select a preset number of data groups from multiple sets of empirical data to form training data.

[0096] The aforementioned simulation environment refers to the hybrid electric vehicle dynamic simulation model system. This simulation environment can accurately reproduce the dynamic response characteristics of the engine, power battery, transmission system and the whole vehicle, and has high temporal resolution and physical consistency. It can simulate the energy flow and state evolution process under real road conditions.

[0097] The aforementioned first data serves as the environmental observation input for each simulation time step, consisting of six-dimensional vehicle operating parameters generated by the simulation environment. Specifically, these parameters include: target power demand, engine output power, engine coolant temperature, battery output power, battery temperature, and battery state of charge. It should be noted that the operating parameters generated by the simulation environment conform to the actual vehicle operating conditions and parameter constraints, and are not arbitrarily generated.

[0098] The second data mentioned above is the deterministic action value output by the initial power distribution network after receiving the first data, which is a combination of continuous control quantities of the engine target output power and the battery target output power.

[0099] The third data mentioned above is a scalar feedback signal calculated by the simulation environment based on the reward function, which is used to measure the quality of the power allocation result.

[0100] In one alternative embodiment, the above-mentioned six-dimensional vehicle operating parameters are generated in a simulation environment based on preset kinematic constraint equations.

[0101] In one optional embodiment, the reward function is set by comprehensively considering four optimization objectives: fuel consumption cost, battery aging and damage cost, penalty for battery state of charge deviating from the target range, and smoothness penalty for engine start-stop frequency. The reward value is calculated in the form of a negative logarithm, so that the optimization objective is aligned with the minimization direction of the neural network loss function; that is, the higher the reward, the better the strategy.

[0102] The fourth data mentioned above refers to the vehicle state (including engine output power, engine coolant temperature, power battery output power, power battery temperature, and power battery state of charge) and required power obtained by iterative calculation in the simulation environment after the action corresponding to the second data is executed.

[0103] In one optional embodiment, the simulation environment generates first data, the initial power allocation network analyzes the first data to obtain a first power allocation result, the simulation environment evaluates the first power allocation result with a reward value, outputs third data, and, based on the second data, calculates the vehicle state and required power at the next moment after executing the action corresponding to the second data. Through multiple interactions between the initial power allocation network and the simulation environment, multiple sets of empirical data are generated and stored in an experience replay buffer.

[0104] Furthermore, from the multiple sets of empirical data placed in the empirical replay buffer, 64 to 256 sets of empirical data are randomly selected according to a uniform distribution to form an independent and identically distributed minimum training batch, which is used as the training data. This random sampling process ensures that the experience at each time step is selected with equal probability.

[0105] In the above steps, multidimensional empirical data containing complete state-action-feedback-transition relationships is continuously generated in the simulation environment, and training data is constructed by combining random sampling mechanism to provide basic data for subsequent network parameter updates.

[0106] Optionally, using the training data, the first copy network, and the second copy network, reinforcement learning is performed on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network, including:

[0107] Step S10531: Using the training data, the first copy network, and the second copy network, update the network parameters of the initial power evaluation network to obtain the target power evaluation network.

[0108] Step S10532: Using the training data and the target power evaluation network, update the network parameters of the initial power allocation network to obtain the target power allocation network.

[0109] During the process of updating the network parameters of the initial power evaluation network using training data, the initial power evaluation network is responsible for evaluating the expected value generated by the current power allocation strategy in real time, while the first and second replica networks are used together to calculate the target Q-value (i.e., the target expected value). During training, the Adam optimizer combined with stochastic gradient descent is used to backpropagate and update all weight parameters of the initial power evaluation network with a fixed learning rate, thereby gradually narrowing the gap between the predicted Q-value (i.e., the expected value output by the initial power evaluation network) and the target expected value. This process, through continuous iteration, enables the power evaluation network to more accurately evaluate the long-term performance of different power allocation strategies, providing a reliable value benchmark for strategy optimization.

[0110] Furthermore, using the training data and the target power evaluation network, the network parameters of the initial power allocation network are updated. The optimization objective of this update process is to enable the target power evaluation network to generate the maximum expected value in response to the output of the initial power allocation network. The parameter update direction is calculated through backpropagation, and the parameters are updated using a preset learning rate.

[0111] Optionally, to improve training efficiency and stability, multiple rounds of reinforcement learning training are performed on the initial power allocation network and the initial power evaluation network. In each round of training, the initial power evaluation network and the initial power allocation network are updated alternately, so that the power allocation strategy gradually approaches the global optimum in continuous optimization, thereby achieving a multi-objective synergistic improvement in fuel economy, battery life and driving smoothness.

[0112] Optionally, the vehicle control method further includes:

[0113] Step S1056: In response to the network parameter update count of the initial power evaluation network reaching the preset update count, the network parameters of the second replica network are updated according to the current network parameters of the updated initial power evaluation network to obtain the updated second replica network.

[0114] Step S1057: In response to the network parameter update count of the initial power allocation network reaching the preset update count, the network parameters of the first replica network are updated according to the current network parameters of the updated initial power allocation network to obtain the updated first replica network.

[0115] In one alternative embodiment, the network parameters of the second copy network are not updated synchronously with the reinforcement learning training process, but are only synchronized once by soft update after the initial power evaluation network has completed a preset number of parameter updates (such as every 4 steps or every 100 steps).

[0116] Soft update refers to linearly mixing the parameters of the main network (such as the initial power evaluation network) into the target network (such as the second copy network) at a small update rate. This mechanism allows the parameters of the target network to smoothly follow the slow evolution of the main network, maintaining the relative stability of the target value without lagging completely and causing policy learning to stagnate.

[0117] Optionally, after each parameter update of the initial power evaluation network, an update counter is incremented. When the counter reaches a preset threshold (e.g., 100 times), a soft update process is triggered, proportionally copying the weight parameters of the current initial power evaluation network to the second copy network to complete parameter synchronization. This process is executed asynchronously in the background, without affecting the online training and environment interaction of the initial power evaluation network, ensuring the continuity of the training process.

[0118] Furthermore, the network parameters of the first replica network are also synchronized using a soft update method. After the initial power allocation network completes a preset number of parameter updates, the latest weight parameters of the current initial power allocation network are linearly superimposed onto the first replica network at a soft update ratio (e.g., 0.005), thus achieving a gradual migration of parameters.

[0119] In the above steps, by introducing a soft update mechanism, the training oscillation and divergence problems caused by self-feedback coupling in the continuous action space of deep reinforcement learning are significantly suppressed.

[0120] Optionally, the vehicle control method further includes:

[0121] Step S10541: Add exploration noise to the second data to obtain the data to be processed;

[0122] Step S10542: Calculate the reward value of the data to be processed to obtain the third data.

[0123] The aforementioned exploration noise refers to a random disturbance signal with temporal correlation and mean regression characteristics generated using the Ornstein-Uhlenbeck (OU) process. Unlike independent and identically distributed Gaussian white noise, OU noise exhibits memory in the time dimension, meaning that the current noise value is affected by the previous moment, showing a smooth and continuous fluctuation trajectory, which is highly consistent with the inertial characteristics of the vehicle's power system.

[0124] In one alternative embodiment, the second data output by the initial power allocation network is linearly superimposed with the OU noise to form a perturbed action vector, i.e., the data to be processed.

[0125] Furthermore, the simulation environment calculates the reward value corresponding to the data to be processed, thereby obtaining the third data.

[0126] In the above steps, by superimposing OU exploration noise with physical inertial matching characteristics on the data output by the initial power allocation network, the exploration efficiency and stability of the reinforcement learning policy in the continuous action space are significantly improved.

[0127] Optionally, Figure 2 This is a network structure diagram of an intelligent agent for power distribution according to one embodiment of the present invention, such as... Figure 2 As shown, this agent is a DDPG-based agent. The DDPG-based agent uses an Actor-Critic architecture, whose core comprises two complementary neural networks: the decision-generating network Actor (i.e., the initial power allocation network) is responsible for formulating the control policy, while the value evaluation network Critic (i.e., the initial power evaluation network) is responsible for accurately evaluating the decision's effectiveness. Both the Critic and Actor networks contain an input layer, two hidden layers, and an output layer. The optimal action-value function... Modeling using deep neural networks is called a Critic network. A Critic network is responsible for evaluating the merits of actions taken in a given state; its input includes the current state. and actions The output is a Q-value estimate for that state-action pair. The DDPG algorithm directly uses a deep neural network to evaluate the optimal policy function. The modeling used is called an Actor network. An Actor network is used to directly output an action value, and its input is the current state. The output is a deterministic action. ,action This is the input to the Critic network. The Critic network is defined as... The weights of a neural network are defined as follows: Actor network is defined as The weights of a neural network are defined as follows: .

[0128] In both the Critic and Actor networks, the hidden layers use ELU as the preferred activation function, which is defined as follows:

[0129] (1)

[0130] Where x is the input of the neuron (the output after the weighted sum of the previous layer). It is a hyperparameter (usually set to 1.0 or a positive number close to 1, such as 0.01) that controls the saturation negative value that ELU approximates when the input is negative infinity.

[0131] In the positive interval (x>0), ELU behaves exactly like ReLU: linear and unsaturated, with a gradient that is always 1. This retains the advantages of ReLU in mitigating gradient vanishing and being computationally efficient. In the negative interval (x≤0), ELU is not a flat line (like LeakyReLU) or a zero line (like ReLU), but a smooth exponential curve that starts from the origin and smoothly decreases as the input becomes more negative, gradually approaching -1. After the output layer is processed by the tanh function and adaptive scaling, the output control quantity can be automatically adjusted according to the set extreme values ​​of the control quantity.

[0132] Optionally, the target network of the Critic network is defined as... (i.e., the second copy network), the neural network weight parameters are: Define the target network of the Actor network as... The neural network weight parameters are The Critic network and Actor network have the same structure as their target network.

[0133] Optionally, Figure 3 This is a flowchart illustrating the training process of an agent according to one embodiment of the present invention, such as... Figure 3 As shown, agent training includes: creating an initial power allocation network and a first copy network; creating an initial power evaluation network and a second copy network; constructing an experience pool; and updating network parameters based on the experience pool.

[0134] The core objective of Critic network updates is to optimize the estimation capability of the action-value function. Through parameter tuning, the network's Q-value predictions for each state-action pair gradually approach their theoretical true values. Using the target network... and To calculate the target Q value :

[0135] (2)

[0136] in, It is a discount factor, with a value between 0 and 1, used to adjust the weight of future rewards in the current decision. This is the reward value for the next moment.

[0137] The Critic network uses mean squared error as the loss function to minimize the predicted Q-value (i.e., ) and the target Q value (i.e. The difference between ) is expressed by the loss function as follows:

[0138] (3)

[0139] Finally, the parameters of the Critic network are updated using the stochastic gradient descent optimization algorithm, as shown below:

[0140] (4)

[0141] (5)

[0142] in, This is the difference between the predicted Q value and the target Q value.

[0143] The core mechanism of Actor network updates is to optimize network parameters through the policy gradient method, so that when the actions generated by Actor are used as input to the Critic network, they can guide the Critic network to maximize the action-value function estimate of the output. The loss function for Actor network weight updates is calculated as follows:

[0144] (6)

[0145] Based on the chain rule of differentiation, the stochastic gradient ascent method is used to adjust the weights of the Actor network. An update will be performed, specifically as follows:

[0146] (7)

[0147] (8)

[0148] in, is the learning rate of the Actor network.

[0149] Furthermore, a soft update method is used to update the target network, specifically as follows:

[0150] (9)

[0151] (10)

[0152] Among them, the neural network parameters of the target network of the Critic network are: The neural network parameters of the target network of the Actor network are: , The learning rate of the target network ( ).

[0153] Furthermore, to achieve effective exploration of the continuous action space, an exploration mechanism decoupled from the target strategy is adopted. Exploration noise with temporal correlation is generated through an OU process. This noise is then linearly superimposed with the deterministic strategy output by the Actor network to construct an exploration behavior strategy (i.e., the data to be processed) that combines inertial memory and random perturbation. The OU noise, with its smoothness and autocorrelation properties, significantly enhances the algorithm's exploratory capabilities. The OU process is specifically represented as follows:

[0154] (11)

[0155] in, This is OU noise.

[0156] Furthermore, the OU process exhibits time-dependent characteristics, and its stochastic differential equation can be expressed as:

[0157] (12)

[0158] in, , The mean (set to 0). The variance is (which gradually decreases to 0 during the training process). It is a Wiener process. In the early stages of training, a large variance is used to increase exploration in order to find the global optimum, while in the later stages, a small variance is used to improve utilization in order to stably converge to the optimum, thus balancing exploration and utilization.

[0159] Compared to independent Gaussian noise, OU noise can explore a wider action space and is more likely to find better strategies. By adjusting its variance and other parameters, OU noise can also balance exploration and exploitation, increasing the exploratory nature of the algorithm in the early stages of training and gradually focusing on exploiting known strategies in the later stages. Secondly, the regression properties of OU noise endow it with excellent stability. During training, OU noise can maintain the continuity of actions and avoid violent jitter, thus ensuring smooth and stable training. This characteristic is crucial for improving the convergence of the DDPG algorithm. Finally, OU noise is particularly suitable for inertial systems, especially when the time discretization granularity is small. Its smoothness and autocorrelation characteristics are well-suited to the action variation characteristics of inertial systems, thus significantly improving exploration efficiency.

[0160] Furthermore, in an optional embodiment, the input to the Actor network (i.e., the environmental state space) is composed of vehicle operating parameters, driver behavior patterns, and external environmental states. The speed trajectory of the driving condition can effectively characterize the driver behavior patterns and the stochastic characteristics of the driving environment. Careful consideration is needed when selecting state variables: the selected state variables should be easily obtainable and fully reflect environmental information, while avoiding excessively high dimensionality, thereby reducing the complexity of solving the optimization problem. Based on the above principles, the state variables are selected as follows: ,in, Used to reflect the speed trajectory characteristics under driving conditions. Used to reflect the engine's operating status. Used to reflect engine coolant temperature Used to reflect the battery's output power The temperature of the power battery is used to reflect its temperature, while the state of charge (SOC) reflects its energy storage status. In another optional embodiment, [the following is omitted as the text is incomplete and requires further context]. Replace it with the vehicle's target power requirement as the state variable.

[0161] Optionally, the output of the Actor network is the predicted engine output power and battery output power.

[0162] Furthermore, in the simulation environment, the reward function is designed as follows when calculating the reward value:

[0163] (13)

[0164] in, These are the weighting coefficients. Used to characterize fuel consumption (i.e., the fuel consumption of a vehicle when the vehicle is controlled according to the predicted engine output power). Used to characterize aging damage in power batteries Used to characterize the degree to which the battery's state of charge (SOC) deviates from the baseline, representing the user's satisfaction with the vehicle's battery level. The number of times the engine starts indicates the smoothness of driving.

[0165] It should be noted that Deep Deterministic Policy Gradient (DDPG) is a reinforcement learning method specifically designed to solve continuous control tasks. It integrates the following three key characteristics: First, the algorithm does not require prior knowledge of the dynamic model of the environment; instead, it optimizes the policy progressively through real-time interaction with the environment. Second, DDPG can learn from historical experience data from any policy, not just data generated by the current policy. This characteristic benefits from experience replay technology, effectively improving the stability and efficiency of the learning process. Third, unlike learning a value function, DDPG directly learns a mapping function from state to action, i.e., the policy function, enabling the agent to accurately explore and execute the optimal action within a continuous action space. DDPG deterministically outputs a single optimal action, allowing it to handle complex continuous action spaces without the need for discretization, thus avoiding performance degradation that might result from discretization. Given these advantages, the DDPG algorithm is adopted for energy allocation control.

[0166] In one optional embodiment, the DDPG agent training algorithm includes: constructing Actor and Critic networks, randomly initializing neural network weight parameters; cloning the target network; constructing an experience pool (using randomly generated actions to interact with the environment, storing the experience data during the interaction process into the initial experience pool); and updating network parameters.

[0167] Furthermore, in deep reinforcement learning, the choice of the number of hidden layer neurons is a core element in building an efficient model, profoundly affecting several key aspects of the model. First, this choice directly determines the model's representational power and complexity; too few or too many neurons can lead to a decline in learning performance. The former results in underfitting due to insufficient information capture, while the latter leads to overfitting and loss of generalization ability. Second, the number of neurons also relates to learning efficiency and training convergence. An appropriate number can accelerate the learning process and ensure the model steadily converges to the global optimum. In addition, a reasonable neuron configuration is crucial for improving the model's generalization performance, enabling the agent to make informed decisions in unknown environments. Finally, considering the limited computational resources, the choice of the number of neurons must also balance the model's computational efficiency and practicality, ensuring excellent performance even under resource constraints. Therefore, in practical applications of deep reinforcement learning, these factors must be comprehensively considered, and fine-tuning of parameters is necessary to determine the optimal number of hidden layer neurons to build a learning model that is both efficient and has strong generalization ability. Simultaneously, the choice of hidden layer activation functions is equally crucial for building efficient neural network models. It not only determines the model's nonlinear representation capability, but also profoundly affects gradient propagation, training stability, and generalization ability.

[0168] Meanwhile, a suitable activation function can introduce non-linearity into the neural network, enabling the model to learn complex features. It also ensures effective gradient propagation, avoiding gradient vanishing or exploding, and guaranteeing training stability. Furthermore, a suitable activation function can promote model generalization and improve its adaptability to different environments.

[0169] It's important to note that the standard ReLU activation function has a strictly zero gradient when the input is negative. If a neuron outputs negative values ​​for most of the training data, its weights may fail to update due to the zero gradient, causing the neuron to "die" and permanently lose its function. ELU's solution is that, in the negative region, while the derivative of ELU is small, it is never zero. This means that even with a large negative input, the neuron will still receive a small gradient, and its weights can still be updated, thus maintaining its activity. Furthermore, ReLU's output is always greater than or equal to 0, and its mean is greater than 0. This non-zero centering characteristic can cause subsequent layers to update gradients with all positive or all negative values ​​during backpropagation, resulting in a zigzag optimization path and reduced convergence efficiency. ELU's solution is that, since it can output negative values ​​in the negative region, the mean of ELU's output is closer to zero, making the gradient update direction more natural, helping to accelerate training convergence and typically reducing the number of iterations. Further, ReLU is not differentiable at the origin, although in practice a subgradient (such as 0 or 1) is usually specified. While Leaky ReLU is continuous, its left and right derivatives at the origin are not equal, resulting in an inflection point. ELU, on the other hand, solves this by using an exponential function in its negative region, which is continuous and differentiable across the entire real domain, and its derivative is also continuous. This smoothness provides a more stable and less noisy gradient signal for optimization processes such as gradient descent, making it particularly beneficial for handling noisy data or scenarios requiring high numerical stability.

[0170] In one alternative embodiment, the DDPG algorithm updates the weights of the neural network based on experience data acquired through interaction with the environment. The DDPG algorithm employs experience replay and target network techniques to improve the convergence and stability of the neural network. First, as the foundation of the experience replay mechanism, the initial experience pool is crucial in the DDPG algorithm. It provides the agent with rich training samples, breaks the temporal correlation between experiences, and stabilizes the training process. These samples help the Critic and Actor networks better approximate the policy and value functions, accelerating algorithm convergence and improving stability and exploration capabilities. The DDPG agent updates the Critic and Actor networks by randomly sampling N experience data points from the experience pool to form a loss function. Common experience pool filling methods include: random exploration strategies, efficiently collecting environmental feedback through the agent's random actions; demonstration data guidance, injecting expert decision trajectories as high-quality learning samples; pre-trained policy initialization, generating a basic experience distribution using prior knowledge; data augmentation techniques, improving data diversity through state-action space transformation; and hybrid policy integration, combining the advantages of multiple methods to achieve a balance between experience coverage and quality.

[0171] Secondly, the target network technique is crucial for improving the stability and convergence of the DDPG algorithm. The target network is used to estimate the target Q-value and the target policy, and its parameters are gradually updated from the main network through a soft update method to ensure stability. This technique reduces the "moving target" problem during training by fixing the target network parameters for a period of time, making the target value more stable and beneficial for training convergence. Simultaneously, using the target network for estimation reduces estimation errors, further improving the algorithm's stability and convergence speed. This soft update method maintains the correlation between the target network and the main network while avoiding training instability caused by directly copying parameters.

[0172] Furthermore, in the DDPG algorithm, the neural network learning rate is a key hyperparameter, directly affecting training efficiency, stability, and final performance. The learning rate determines the step size of network weight updates, controlling the adjustment magnitude of the Actor and Critic networks. A higher learning rate can accelerate convergence but may also lead to instability in the training process; a lower learning rate, while stable, will slow down the convergence speed. An appropriate learning rate can ensure smooth model training, avoid extreme cases, and achieve better performance. In an optional embodiment, a policy network delayed update mechanism is introduced. By setting the weight update frequency of the Critic network to k times that of the Actor network, the volatility of the action-value function estimation process is effectively suppressed, thereby improving the convergence of the neural network. This policy network delayed update mechanism has the following advantages: First, it helps to avoid overfitting, reduce training oscillations, and ensure the stability of the training process; second, it can fully utilize the evaluation capability of the Critic network, improve learning efficiency, and ensure the accuracy of policy gradient estimation, which helps the algorithm converge; finally, it can also balance the relationship between exploration and exploitation, enabling the algorithm to achieve better performance in reinforcement learning tasks.

[0173] In another alternative embodiment, vehicle driving information is uploaded to the cloud, where the agent is continuously trained to make its control strategy more closely resemble the driver's driving habits.

[0174] This invention also provides a vehicle control system for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0175] According to another aspect of the present invention, a vehicle control system is also provided. Figure 4 This is a structural block diagram of a vehicle control system according to one embodiment of the present invention, such as... Figure 4 As shown, the vehicle control system 400 includes: an acquisition module 401 for acquiring vehicle state data and environmental perception data; a determination module 402 for determining the target power demand of the vehicle based on the vehicle state data and environmental perception data; an analysis module 403 for analyzing the vehicle state data and target power demand using a target power allocation network to obtain a power allocation result, wherein the power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery, and the target power allocation network is a deep neural network trained by reinforcement learning; and a generation module 404 for generating vehicle control commands based on the power allocation result.

[0176] Optionally, the analysis module 403 is further configured to: use the target power evaluation network corresponding to the target power allocation network to perform an expected evaluation of the power allocation result and obtain an expected value; in response to the expected value being greater than the expected threshold, perform a feasibility verification of the power allocation result and obtain a verification result; in response to the verification result indicating that the power allocation result does not meet the feasibility conditions, update the power allocation result based on the feasibility conditions and obtain an updated power allocation result.

[0177] Optionally, the vehicle status data includes: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, and transmission system temperature; the environmental perception data includes: road slope and road curvature; the determination module 402 is also used to: determine the initial power demand based on the accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, road slope, and road curvature; predict the mechanical loss power based on the transmission system temperature; and determine the target power demand based on the initial power demand and the mechanical loss power.

[0178] Optionally, the vehicle control system 400 further includes a training module (not shown in the figure), which is used to: create an initial power allocation network and a first copy network, wherein the initial power allocation network and the first copy network have the same network structure and network parameters; create an initial power evaluation network and a second copy network, wherein the initial power evaluation network and the second copy network have the same network structure and network parameters; and use the training data, the first copy network, and the second copy network to perform reinforcement learning training on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network.

[0179] Optionally, the training module is also used to: generate multiple sets of empirical data in a simulation environment, wherein any set of empirical data includes: first data, second data, third data, and fourth data, the first data is used to describe the vehicle state and power demand generated at the current simulation time step, the second data is used to characterize the first power allocation result obtained by the initial power allocation network after analyzing the first data, the third data is used to characterize the reward value of the first power allocation result, and the fourth data is used to describe the updated vehicle state and updated power demand obtained based on the first power allocation result; and randomly select a preset number of data groups from the multiple sets of empirical data to form training data.

[0180] Optionally, the training module is also used to: update the network parameters of the initial power evaluation network using training data, the first copy network, and the second copy network to obtain the target power evaluation network; and update the network parameters of the initial power allocation network using training data and the target power evaluation network to obtain the target power allocation network.

[0181] Optionally, the training module is further configured to: in response to the network parameter update count of the initial power evaluation network reaching a preset update count, update the network parameters of the second replica network according to the current network parameters of the updated initial power evaluation network to obtain the updated second replica network; and in response to the network parameter update count of the initial power allocation network reaching a preset update count, update the network parameters of the first replica network according to the current network parameters of the updated initial power allocation network to obtain the updated first replica network.

[0182] Optionally, the training module is also used to: superimpose exploration noise onto the second data to obtain the data to be processed; and calculate the reward value of the data to be processed to obtain the third data.

[0183] According to another aspect of the present invention, a vehicle is also provided. Optionally, Figure 5 This is a schematic diagram of a vehicle according to one embodiment of the present invention, such as... Figure 5 As shown, the vehicle 500 may include a memory 510 and a processor 520, wherein the memory 510 is used to store an executable program; the processor 520 is used to run the program stored in the memory 510, and the program executes the vehicle control method in any of the above-mentioned claims of this application when it runs.

[0184] Optionally, in this embodiment, the executable program performs the following steps when it runs:

[0185] Step S101: Obtain vehicle status data and environmental perception data.

[0186] Step S102: Determine the target power requirement of the vehicle based on vehicle status data and environmental perception data;

[0187] Step S103: Analyze the vehicle state data and target power demand using the target power allocation network to obtain the power allocation result. The power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery. The target power allocation network is a deep neural network trained by reinforcement learning.

[0188] Step S104: Based on the power allocation results, generate vehicle control commands.

[0189] Optionally, the executable program performs the following steps when it runs: using a target power evaluation network corresponding to the target power allocation network, it performs an expected evaluation of the power allocation result to obtain an expected value; in response to the expected value being greater than the expected threshold, it performs a feasibility check on the power allocation result to obtain a check result; in response to the check result indicating that the power allocation result does not meet the feasibility conditions, it updates the power allocation result based on the feasibility conditions to obtain an updated power allocation result.

[0190] Optionally, the vehicle status data includes: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, and transmission system temperature; the environmental perception data includes: road slope and road curvature. When the above executable program runs, it performs the following steps: determining the initial power demand based on the accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, road slope, and road curvature; predicting the mechanical loss power based on the transmission system temperature; and determining the target power demand based on the initial power demand and the mechanical loss power.

[0191] Optionally, the executable program executes the following steps during runtime: creating an initial power allocation network and a first copy network, wherein the initial power allocation network and the first copy network have the same network structure and network parameters; creating an initial power evaluation network and a second copy network, wherein the initial power evaluation network and the second copy network have the same network structure and network parameters; and using training data, the first copy network, and the second copy network, performing reinforcement learning training on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network.

[0192] Optionally, the executable program executes the following steps during runtime: In a simulation environment, multiple sets of empirical data are generated, wherein any set of empirical data includes: first data, second data, third data, and fourth data. The first data is used to describe the vehicle state and power demand generated at the current simulation time step. The second data is used to characterize the first power allocation result obtained by the initial power allocation network after analyzing the first data. The third data is used to characterize the reward value of the first power allocation result. The fourth data is used to describe the updated vehicle state and updated power demand obtained based on the first power allocation result. A preset number of data groups are randomly selected from the multiple sets of empirical data to form training data.

[0193] Optionally, the executable program performs the following steps when it runs: updating the network parameters of the initial power evaluation network using training data, the first copy network, and the second copy network to obtain the target power evaluation network; and updating the network parameters of the initial power allocation network using training data and the target power evaluation network to obtain the target power allocation network.

[0194] Optionally, the executable program performs the following steps when it runs: in response to the network parameter update count of the initial power evaluation network reaching a preset update count, the network parameters of the second replica network are updated according to the current network parameters of the updated initial power evaluation network to obtain the updated second replica network; in response to the network parameter update count of the initial power allocation network reaching a preset update count, the network parameters of the first replica network are updated according to the current network parameters of the updated initial power allocation network to obtain the updated first replica network.

[0195] Optionally, the executable program performs the following steps when it runs: superimposing exploration noise on the second data to obtain the data to be processed; calculating the reward value of the data to be processed to obtain the third data.

[0196] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the vehicle control method described in any of the above.

[0197] Optionally, in this embodiment, the executable program can be configured to store an executable program for performing the following steps:

[0198] Step S101: Obtain vehicle status data and environmental perception data.

[0199] Step S102: Determine the target power requirement of the vehicle based on vehicle status data and environmental perception data;

[0200] Step S103: Analyze the vehicle state data and target power demand using the target power allocation network to obtain the power allocation result. The power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery. The target power allocation network is a deep neural network trained by reinforcement learning.

[0201] Step S104: Based on the power allocation results, generate vehicle control commands.

[0202] Optionally, the executable program can be configured to store an executable program for performing the following steps: using a target power evaluation network corresponding to the target power allocation network to perform an expected evaluation of the power allocation result and obtain an expected value; in response to the expected value being greater than the expected threshold, performing a feasibility check on the power allocation result and obtaining a check result; in response to the check result indicating that the power allocation result does not meet the feasibility conditions, updating the power allocation result based on the feasibility conditions and obtaining an updated power allocation result.

[0203] Optionally, the vehicle status data includes: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, and transmission system temperature; the environmental perception data includes: road slope and road curvature. The aforementioned executable program can be configured to store an executable program for performing the following steps: determining the initial power demand based on the accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, road slope, and road curvature; predicting mechanical loss power based on the transmission system temperature; and determining the target power demand based on the initial power demand and mechanical loss power.

[0204] Optionally, the executable program described above can be configured to store an executable program for performing the following steps: creating an initial power allocation network and a first copy network, wherein the initial power allocation network and the first copy network have the same network structure and network parameters; creating an initial power evaluation network and a second copy network, wherein the initial power evaluation network and the second copy network have the same network structure and network parameters; and using training data, the first copy network, and the second copy network, performing reinforcement learning training on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network.

[0205] Optionally, the executable program can be configured to store an executable program for performing the following steps: In a simulation environment, multiple sets of empirical data are generated, wherein any set of empirical data includes: first data, second data, third data, and fourth data. The first data is used to describe the vehicle state and power demand generated at the current simulation time step. The second data is used to characterize the first power allocation result obtained by the initial power allocation network after analyzing the first data. The third data is used to characterize the reward value of the first power allocation result. The fourth data is used to describe the updated vehicle state and updated power demand obtained based on the first power allocation result. A preset number of data groups are randomly selected from the multiple sets of empirical data to form training data.

[0206] Optionally, the executable program can be configured to store an executable program for performing the following steps: updating the network parameters of the initial power evaluation network using training data, the first copy network, and the second copy network to obtain the target power evaluation network; and updating the network parameters of the initial power allocation network using training data and the target power evaluation network to obtain the target power allocation network.

[0207] Optionally, the executable program can be configured to store an executable program for performing the following steps: in response to the network parameter update count of the initial power evaluation network reaching a preset update count, updating the network parameters of the second replica network according to the current network parameters of the updated initial power evaluation network to obtain the updated second replica network; in response to the network parameter update count of the initial power allocation network reaching a preset update count, updating the network parameters of the first replica network according to the current network parameters of the updated initial power allocation network to obtain the updated first replica network.

[0208] Optionally, the executable program can be configured to store an executable program for performing the following steps: superimposing exploration noise on the second data to obtain data to be processed; calculating the reward value of the data to be processed to obtain the third data.

[0209] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0210] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0211] In the embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The system embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0214] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0215] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A vehicle control method, characterized in that, include: Acquire vehicle status data and environmental perception data; Based on the vehicle status data and the environmental perception data, the target power requirement of the vehicle is determined. The vehicle state data and the target power demand are analyzed using a target power allocation network to obtain a power allocation result. The power allocation result is used to characterize the first power to be output by the vehicle's engine and the second power to be output by the vehicle's battery. The target power allocation network is a deep neural network trained by reinforcement learning. Based on the power allocation results, vehicle control commands are generated.

2. The vehicle control method according to claim 1, characterized in that, The vehicle control method further includes: Using the target power evaluation network corresponding to the target power allocation network, the power allocation result is evaluated to obtain the expected value; In response to the expected value being greater than the expected threshold, a feasibility check is performed on the power allocation result to obtain the check result; In response to the verification result indicating that the power allocation result does not meet the feasibility conditions, the power allocation result is updated based on the feasibility conditions to obtain the updated power allocation result.

3. The vehicle control method according to claim 1, characterized in that, The vehicle status data includes: accelerator pedal opening, brake pedal opening, vehicle gear, vehicle speed change rate, and transmission system temperature. The environmental perception data includes: road slope and road curvature. Based on the vehicle status data and the environmental perception data, the target power requirement of the vehicle is determined, including: The initial power requirement is determined based on the accelerator pedal opening, the brake pedal opening, the vehicle gear, the vehicle speed change rate, the road gradient, and the road curvature. Based on the temperature of the transmission system, predict the mechanical loss power; The target power demand is determined based on the initial power demand and the mechanical loss power.

4. The vehicle control method according to claim 1, characterized in that, The vehicle control method further includes: An initial power allocation network and a first replica network are created, wherein the initial power allocation network and the first replica network have the same network structure and network parameters; An initial power evaluation network and a second replica network are created, wherein the initial power evaluation network and the second replica network have the same network structure and network parameters; Using training data, the first replica network, and the second replica network, reinforcement learning is performed on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network.

5. The vehicle control method according to claim 4, characterized in that, The vehicle control method further includes: In the simulation environment, multiple sets of empirical data are generated. Each set of empirical data includes: first data, second data, third data, and fourth data. The first data is used to describe the vehicle state and power demand generated at the current simulation time step. The second data is used to characterize the first power allocation result obtained by the initial power allocation network after analyzing the first data. The third data is used to characterize the reward value of the first power allocation result. The fourth data is used to describe the updated vehicle state and updated power demand obtained based on the first power allocation result. A predetermined number of data groups are randomly selected from the multiple sets of empirical data to form the training data.

6. The vehicle control method according to claim 5, characterized in that, Using the training data, the first replica network, and the second replica network, reinforcement learning is performed on the initial power allocation network and the initial power evaluation network to obtain the target power allocation network, including: Using the training data, the first replica network, and the second replica network, the network parameters of the initial power evaluation network are updated to obtain the target power evaluation network; Using the training data and the target power evaluation network, the network parameters of the initial power allocation network are updated to obtain the target power allocation network.

7. The vehicle control method according to claim 6, characterized in that, The vehicle control method further includes: In response to the network parameter update count of the initial power evaluation network reaching a preset update count, the network parameters of the second replica network are updated according to the current network parameters of the updated initial power evaluation network to obtain the updated second replica network; In response to the network parameter update count of the initial power allocation network reaching the preset update count, the network parameters of the first replica network are updated according to the current network parameters of the updated initial power allocation network to obtain the updated first replica network.

8. The vehicle control method according to claim 5, characterized in that, The vehicle control method further includes: The second data is superimposed with exploration noise to obtain the data to be processed; The reward value of the data to be processed is calculated to obtain the third data.

9. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the executable program, wherein the executable program, when running, performs the vehicle control method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the vehicle control method according to any one of claims 1 to 8.

Images