Integrated Energy Efficiency Optimization Control Method and System for Intelligent Connected New Energy Vehicles

Abstract

This invention provides an integrated energy efficiency optimization control method and system for intelligent connected new energy vehicles, comprising: acquiring road condition information to form a traffic information representation vector; acquiring energy management and thermal management system data to form a low-dimensional representation vector of the energy management and thermal management system states and fusing it into data scaling normalization and vector concatenation; performing reinforcement learning training on a whole vehicle model; updating the neural network parameters of the energy and thermal management integrated control system after reinforcement learning training; establishing a neural network model for energy and thermal management integrated control through data scaling normalization, vector concatenation, and updating the neural network parameters of the energy and thermal management integrated control system; generating control signals for the thermal management system and the energy management system; and transmitting the control signals to the thermal management system components and the energy management system components respectively for action execution. This invention solves the problem of large energy losses caused by the difficulty in coordinating and optimizing the heating management and capacity management of existing vehicles.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to an integrated energy efficiency optimization control method and system for intelligent connected new energy vehicles. Background Technology

[0002] Intelligentization and electrification are two major development directions for the future automotive industry, with hybrid electric vehicles equipped with intelligent connected systems expected to become mainstream. In terms of automotive energy conservation, the powertrain control strategy of intelligent connected hybrid electric vehicles is key to improving overall vehicle energy efficiency. However, most current energy control systems often overlook the energy consumption of thermal management. Nevertheless, the thermal management system of hybrid electric vehicles is a core module in modern automotive product development, encompassing a comprehensive system engineering project including oil cooling, air conditioning, HVAC, turbocharging and intercooling, and cooling of the engine compartment and passenger compartment. Furthermore, the energy consumption of the components involved in thermal management cannot be ignored. The thermal management system requires stable and efficient optimal control of a complex system, maintaining the temperature of various components of the hybrid electric vehicle within set values ​​and ranges with minimal error.

[0003] It is evident that energy management and thermal management are both crucial for energy conservation and emission reduction in hybrid electric vehicles (HEVs). However, current research typically treats them as two independent systems controlled separately, neglecting their highly coupled relationship. Therefore, coordinating the control of these two tasks holds promise for further improving the fuel economy of HEVs. However, traditional optimal control algorithms have encountered bottlenecks in terms of efficiency and optimality when facing high-dimensional control problems such as coordinating thermal and energy management. Currently, only research exists on applying reinforcement learning to HEV energy management, but no research has been found on using reinforcement learning to achieve coordinated control of thermal and energy management. Therefore, there is an urgent need to study integrated control systems for HEV thermal and energy management, and multi-task deep reinforcement learning control methods are well-suited for this problem. The advantage of multi-task deep reinforcement learning lies in its ability to handle not only the high-dimensional state variables involved in energy and thermal management but also to consider multi-source real-time traffic information. Furthermore, it can simultaneously output action variables involved in multiple tasks, achieving coordinated control of thermal and energy management.

[0004] For the single task of thermal management, the use of deep reinforcement learning for the control system of hybrid electric vehicle (HEV) thermal management is still in the research stage. The thermal management system not only needs to control the temperature of various components of the HEV within a set range, but also needs to ensure multiple vehicle evaluation indicators, including powertrain reliability, passenger comfort, thermal economy, and engine emission characteristics. However, HEV thermal management requires the control of many system parameters, and the modeling of each part is complex, making it a typical hybrid nonlinear system. It is difficult to achieve optimal control of complex systems using model predictive control or other modeling methods. Therefore, how to use reinforcement learning to control the operating temperature of components such as the engine, motor, and battery, while ensuring cabin comfort, is an urgent problem to be solved in HEV thermal management control. On the other hand, the energy management of HEVs needs to achieve better fuel economy and reduce energy consumption by rationally distributing the output power of various components. Summary of the Invention

[0005] This invention provides an integrated energy efficiency optimization control method and system for intelligent connected new energy vehicles, which solves the problem of large energy loss caused by the difficulty in coordinating and optimizing the heating management and capacity management of existing vehicles.

[0006] This invention provides an integrated energy efficiency optimization control method for intelligent connected new energy vehicles, comprising:

[0007] Obtaining road condition information constitutes a representation vector of traffic information, and obtaining energy management and thermal management system data constitutes a low-dimensional representation vector of the state of energy management and thermal management system.

[0008] The representation vector of the traffic information and the low-dimensional representation vector of the state of the energy management and thermal management system are fused into data scale normalization and vector concatenation;

[0009] The neural network parameters of the energy and thermal management integrated control system are updated after reinforcement learning training by acquiring a whole vehicle model.

[0010] A neural network model for integrated energy and thermal management control is established by using data scaling normalization, vector concatenation, and neural network parameter updates for the integrated energy and thermal management control system.

[0011] The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution.

[0012] According to the integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention, the acquisition of road condition information to form a representation vector of traffic information specifically includes:

[0013] Acquire long-term and short-term road condition information provided by intelligent connected roadside equipment and vehicle-side systems to form a representation vector of traffic information;

[0014] The traffic information representation vector includes: distance to the next traffic light, the status and countdown of the next traffic light, remaining driving distance, and road traffic situation.

[0015] According to the integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention, the acquisition of energy management and thermal management system data constitutes a low-dimensional representation vector of the state of the energy management and thermal management system, specifically including:

[0016] Perform state variable design and control variable design;

[0017] The design of the state variables includes the design of the environment and vehicle system, engine cooling system, passenger compartment system, motor system, battery system, and intelligent connected transportation environment;

[0018] The control variable design includes: engine radiator fan speed, engine coolant pump speed, electronic thermostat opening, compressor displacement, evaporator fan speed, battery cooling fan speed, engine output power, motor output power, and motor coolant pump speed, which are set as control variables.

[0019] According to the integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention, the step of acquiring a whole vehicle model for reinforcement learning training, and updating the neural network parameters of the energy and thermal management integrated control system after reinforcement learning training, specifically includes:

[0020] Based on the acquired vehicle model, reinforcement learning training is performed through a pre-set reinforcement learning training framework;

[0021] After training, the neural network parameters of the energy and thermal management integrated control system are updated.

[0022] According to the integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention, the method for establishing a neural network model for integrated energy and thermal management control through data scaling normalization, vector cascading, and neural network parameter updates of the integrated energy and thermal management control system specifically includes:

[0023] During the training process of the energy management and thermal management collaborative control system based on the reinforcement learning training framework, before the offline training process begins, the capacity of the replay buffer is initialized, and the transformation tuples sampled from the interaction process of the reinforcement learning training framework based on the exploration are stored in the replay buffer for parameter updates. The process is repeated in the replay buffer, iterating and recording new transformations.

[0024] When the replay buffer is full, the oldest experience is discarded and replaced by a new experience. For each round, a small batch of multiple transformation tuples are randomly sampled from the replay buffer to update the parameters once.

[0025] The parameters updated by the evaluation network with training mechanism and the weights of the target network are inherited from the corresponding evaluation network with soft updates, thus establishing a neural network model for integrated control of energy and thermal management.

[0026] According to the present invention, an integrated energy efficiency optimization control method for intelligent connected new energy vehicles generates control signals for a thermal management system and an energy management system through a neural network model, and transmits the control signals to the thermal management system components and the energy management system components respectively for action execution, specifically including:

[0027] The neural network model outputs control signals for the thermal management system, which are then transmitted to the thermal management actuator.

[0028] The neural network model outputs the optimal engine or battery power allocation result to the dynamic coordination control module, and outputs the given value determined by the dynamic coordination control module to the energy management actuator.

[0029] The corresponding actions are performed through the thermal management actuator and the energy management actuator.

[0030] This invention also provides an integrated energy efficiency optimization control system for intelligent connected new energy vehicles, the system comprising:

[0031] The data acquisition module is used to acquire road condition information to form a representation vector of traffic information, and to acquire energy management and thermal management system data to form a low-dimensional representation vector of the state of energy management and thermal management system.

[0032] The data fusion module is used to fuse the representation vector of the traffic information and the low-dimensional representation vector of the energy management and thermal management system status into data scale normalization and vector concatenation;

[0033] The training module is used to perform reinforcement learning training by acquiring a whole vehicle model, and to update the neural network parameters of the energy and thermal management integrated control system after reinforcement learning training;

[0034] The model generation module is used to establish a neural network model for the integrated energy and thermal management control system through data scaling normalization, vector concatenation, and neural network parameter updates of the integrated energy and thermal management control system.

[0035] The control module is used to generate control signals for the thermal management system and the energy management system through the neural network model, and transmit the control signals to the thermal management system components and the energy management system components respectively for action execution.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the integrated energy efficiency optimization control method for intelligent connected new energy vehicles as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the integrated energy efficiency optimization control method for intelligent connected new energy vehicles as described above.

[0038] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the integrated energy efficiency optimization control method for intelligent connected new energy vehicles as described above.

[0039] This invention provides an integrated energy efficiency optimization control method and system for intelligent connected new energy vehicles. It uses system state parameters and real-time traffic information provided by the intelligent network as input variables. With the goals of optimizing system fuel consumption, energy consumption of various components, and cabin comfort, it outputs the on / off thresholds or action ranges of key electronic control components and the power settings of power components such as the engine and drive motor through interactive reinforcement learning between the intelligent agent and the environment. By comprehensively considering the coordinated control of thermal management and energy management in hybrid vehicles, it autonomously controls the output power of the engine and drive motor, as well as the state of thermal management components such as the fan and thermostat, to achieve the lowest possible energy consumption cost while ensuring that components such as the engine, drive motor, and power battery operate within their optimal temperature range. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts of an integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention;

[0042] Figure 2 This is the second flowchart of an integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention;

[0043] Figure 3 This is the third flowchart of an integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the present invention;

[0044] Figure 4 This is a schematic diagram of the overall framework of the hybrid electric vehicle thermal management and energy management coordinated control system provided by the present invention;

[0045] Figure 5 This is a hardware and software architecture diagram of the hybrid electric vehicle thermal management and energy management system provided by the present invention;

[0046] Figure 6 This is a schematic diagram of the signal transmission process in the online application of the energy management and thermal management system provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the offline training process of the energy and thermal management integrated control system based on reinforcement learning provided by the present invention;

[0048] Figure 8 This is a schematic diagram of the module connection of an integrated energy efficiency optimization control system for intelligent connected new energy vehicles provided by the present invention;

[0049] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0050] Figure label:

[0051] 110: Data acquisition module; 120: Data fusion module; 130: Training module; 140: Model generation module; 150: Control module;

[0052] 910: Processor; 920: Communication interface; 930: Memory; 940: Communication bus. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] The following is combined with Figures 1-3 This invention describes an integrated energy efficiency optimization control method for intelligent connected new energy vehicles, comprising:

[0055] Obtaining road condition information constitutes a representation vector of traffic information, and obtaining energy management and thermal management system data constitutes a low-dimensional representation vector of the state of energy management and thermal management system.

[0056] The representation vector of the traffic information and the low-dimensional representation vector of the state of the energy management and thermal management system are fused into data scale normalization and vector concatenation;

[0057] The neural network parameters of the energy and thermal management integrated control system are updated after reinforcement learning training by acquiring a whole vehicle model.

[0058] A neural network model for integrated energy and thermal management control is established by using data scaling normalization, vector concatenation, and neural network parameter updates for the integrated energy and thermal management control system.

[0059] The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution.

[0060] This invention explains the composition and working principle of the engine and its control system, the drive motor and its control system, the air conditioning thermal management control system, the power battery and its control system, the temperature sensing system, and the VCU controller.

[0061] The engine and its control system consist of components such as the engine, radiator and fan, water pump, thermostat, and HVAC heater. The radiator, as the main heat exchange component, facilitates convective heat exchange between the cylinder block, coolant, and engine compartment environment. The water pump provides pressure and head to each loop within the system, ensuring coolant circulation. The thermostat is a three-way valve that divides the system into two loops: a large loop and a small loop. When the engine outlet coolant temperature is lower than the thermostat's opening temperature, the large loop closes, and coolant flows through the bypass to the water pump and back to the engine; during this process, the cooling system does not exchange heat with the external environment. When the engine outlet coolant temperature is higher than the thermostat's maximum lift temperature, the small loop closes, and all system coolant flows through the radiator and back to the engine; at this point, the system is at its maximum heat exchange capacity. When the coolant temperature is within the thermostat's adjustment range, both the large and small loops open simultaneously. The thermostat adjusts the valve opening based on the engine outlet coolant temperature, thereby controlling the coolant flow ratio between the large and small loops.

[0062] The air conditioning thermal management control system consists of a compressor, expansion valve, evaporator, and condenser. The high-temperature, high-pressure gas from the compressor outlet is condensed and dissipated in the condenser, becoming a high-temperature, high-pressure liquid. The refrigerant then flows through the expansion valve, where it is throttled and depressurized, transforming into a low-temperature gas-liquid mixture. This low-temperature refrigerant mixture evaporates and absorbs heat in the evaporator, and the superheated vapor from the evaporator outlet returns to the compressor, forming a flow cycle. In the entire refrigeration cycle, the vapor compression process in the compressor is a power-consuming process, the evaporator and condenser processes are phase-change heat-generating processes, and the expansion valve process is a throttling and depressurization process. In summer, the air conditioning system provides low-temperature cooled air to the passenger compartment and cools the power battery; in winter, the heating, ventilation, and air conditioning (HVAC) heater utilizes waste heat from the engine cooling cycle to provide high-temperature heated air to the passenger compartment.

[0063] The drive motor and its control system consist of an electric motor, a cooling water pump, a two-way valve, a radiator, and a fan. The motor's auxiliary power output and charging process do not operate under full-condition conditions. When the motor is not in operation, there is no need to regulate the motor temperature. The opening and closing of the thermal management system is controlled by the two-way valve.

[0064] The power battery and its control system employ an air-cooled thermal management system, utilizing air conditioning for cooling and equipped with an independent fan to regulate airflow. The temperature sensing system consists of temperature sensors for the passenger compartment, engine, motor, and battery, responsible for collecting the temperature data from these three components and transmitting the signals to the VCU controller. The VCU controller, based on feedback signals from the passenger compartment temperature and the operating temperatures of the motor and battery, controls the speed of the motor cooling water pump and the battery cooling fan, respectively.

[0065] The thermal management system controller is integrated into the vehicle's VCU (the core electronic control unit for vehicle control decisions). Air temperature sensors and coolant temperature sensors are located inside the passenger compartment and at the engine outlet, respectively. These temperature sensors feed back system status signals to the controller. The VCU outputs control variable signals for actuators such as the water pump, cooling fan, and compressor, achieving integrated control of energy management and thermal management.

[0066] The VCU controller, or Energy Management and Thermal Management Integrated Control Unit, operates based on signals transmitted from various management systems.

[0067] The acquired road condition information forms the representation vector of traffic information, specifically including:

[0068] Acquire long-term and short-term road condition information provided by intelligent connected roadside equipment and vehicle-side systems to form a representation vector of traffic information;

[0069] The traffic information representation vector includes: distance to the next traffic light, the status and countdown of the next traffic light, remaining driving distance, and road traffic situation.

[0070] Data from the energy management and thermal management system is used to construct a low-dimensional representation vector of the system's state, specifically including:

[0071] Perform state variable design and control variable design;

[0072] The design of the state variables includes the design of the environment and vehicle system, engine cooling system, passenger compartment system, motor system, battery system, and intelligent connected transportation environment;

[0073] The control variable design includes: engine radiator fan speed, engine coolant pump speed, electronic thermostat opening, compressor displacement, evaporator fan speed, battery cooling fan speed, engine output power, motor output power, and motor coolant pump speed, which are set as control variables.

[0074] The state variable design in this invention includes:

[0075] For environmental and vehicle systems, the ambient temperature T env (°C), vehicle speed v car (m / s), vehicle acceleration a car (m / s 2 Introduce state variables;

[0076] For the engine cooling system, the engine speed ω ICE (rpm), engine torque T ICE (Nm), the difference between the engine outlet coolant temperature measured and the reference value △T c (°C), the difference between the measured cylinder temperature and the reference value △T w (°C) Introduce state variables;

[0077] For the crew cabin, the difference ΔT between the detected cabin temperature and the reference value is calculated. cab (°C), cabin airflow (m³) cab Variables such as (kg / s) are introduced as state variables;

[0078] For the motor system, the difference ΔT between the motor temperature and the reference value is... MG Motor speed ω MG (rpm) and motor torque T MG (Nm) Introduce state variables;

[0079] For the battery system, the difference ΔT between the battery temperature and the reference value is... bat (°C) and battery SOC (%) introduce state variables.

[0080] For intelligent connected traffic environments, the distance d from the next traffic light will be considered. tl The status and countdown of the next traffic light C tl Remaining driving range θ GIS Road traffic situation S ITS Introduce state variables.

[0081] Design control variables, and set the engine radiator fan speed N. fan1 Engine cooling water pump speed (N) pump1 Electronic thermostat opening θ thmst Compressor displacement V com Evaporator fan speed Nfan2 Battery cooling fan speed N fan3 Engine output power P ICE Motor output power P MG Motor cooling water pump speed (N) pump2 Set as a control variable.

[0082] The system employs reinforcement learning training by acquiring a complete vehicle model. Following this training, the neural network parameters of the energy and thermal management integrated control system are updated. Specifically, this includes:

[0083] Based on the acquired vehicle model, reinforcement learning training is performed through a pre-set reinforcement learning training framework;

[0084] After training, the neural network parameters of the energy and thermal management integrated control system are updated.

[0085] refer to Figures 4-6 This invention can employ various reinforcement learning training frameworks. An example using the classic TD3 algorithm is provided below; other algorithm frameworks, such as DQN, A3C, PPO, and SAC, can also be used.

[0086] For standard Q-learning, its value function Q π (s t ,a t ) refers to state-action combination (s t ,a t The Q value of ) and subsequent state-action combination (s t+1 ,a t+1 The fundamental relationship between the Q values ​​is achieved by updating them using time-difference based on the Bellman equation:

[0087]

[0088] To handle tasks with continuous control characteristics, the strategy π, denoted by θ, can be... θ The gradient of the expected return is used for updating. Therefore, the actor-critic method is adopted, and the policy is updated within the actor network using the deterministic policy gradient (DPG) algorithm. The DPG update mechanism is as follows:

[0089]

[0090] For using the neural network-based function approximator Q... w (s t ,a t In multiple updates, the fixed target value y can be obtained from a network using time-difference updates, where one of the frozen target networks Q...w′ (s t ,a t The following formula is used:

[0091]

[0092] It should be noted that action s t+1 From the target action network π θ′ The weights of the target network can be generated and updated periodically to precisely match the weights of the evaluation network. Alternatively, an off-policy type soft update mechanism can be used to update the weights, which is achieved by randomly sampling mini-batch transition sequences from the experience replay buffer.

[0093] Compared to classic model-free DQL algorithms, DDPG combines DQL with DPG, utilizing an actor-critic framework to construct a continuous action space. Furthermore, a state-of-the-art algorithm evolved from DDPG, namely twin delayed deep deterministic policy gradient (TD3), offers faster convergence and a more stable training process compared to traditional DDPG. Its improvements primarily stem from three aspects: employing a sheared DQL mechanism, TD3 learns two Q-functions instead of one; smoothing the target policy by adding noise; and a policy update delay mechanism, where TD3 updates the policy slower than the Q-function. Therefore, this patent utilizes TD3 to train the energy management and thermal management agent of the power system. While outputting engine power (PICE) and motor power (PMG), it also outputs the operating states of each thermal management controlled unit, such as engine radiator fan speed (Nfan1), and expands the TD3 agent's state space to acquire multivariate traffic and terrain information processed by the spatial-temporal data processing (STDP) framework in real time. Once the training process is complete, the network can download the data to the powertrain controller for online application.

[0094] The framework of learning algorithms, such as Figure 7 As shown, the actor network is designed to output actions. Furthermore, a randomly initialized actor network with exploratory noise is used, through a... t =π θ (s tThe engine control command is generated using N(0,δ), where θ is the parameter of the evaluation network. Gaussian noise N(0,δ) is added to the deterministic policy to achieve a balance between exploration and exploitation. This iterative process is repeated to obtain various transition sequences, with exploration noise added to the actions. The evaluation actor network is the policy model that generates continuous output commands for energy management and thermal management. Two evaluation critic networks calculate the Q-value based on the inputs of the state and actions. Two evaluation critic networks are created simultaneously to compute the Q-value in parallel to reduce overestimation problems; these are denoted as θ, δ ... and The smaller of the two evaluation critic network outputs will be used to update the parameters of both evaluation critic networks by minimizing a time-difference-based loss function. Here, γ represents the discount factor, and... and Defined as the Q-value calculated by two target critic networks, such as... Figure 7 As shown. Additionally... This represents the action generated by the target actor network after adding sheared random noise, used to smooth state-action value estimation.

[0095]

[0096]

[0097]

[0098] For each transition sequence e t =(s t ,a t ,r t ,s t+1 ), s t It evaluates the output of the actor network, and the action of the network output is a. t =π θ (s t ). The obtained action π θ (s t The Q-values ​​are calculated by inputting into two evaluation critic networks, and are expressed as follows: and Here, w1 and w2 are the parameters for evaluating critic network I and critic network II, respectively. Furthermore, the Q-value Q generated using the deterministic policy gradient and the evaluation critic network I is used. w1 (s t ,π θ (s tThe target network is used to update the weights of the evaluation actor network. It's important to note that the target network is updated every d steps, meaning that the target network's weight updates are slower than the evaluation network's.

[0099]

[0100] In the training process of the TD3-based energy management and thermal management collaborative control system, before the offline training begins, the capacity of the replay buffer D is initialized to M. Transformation tuples sampled from the TD3 interaction process based on the exploration-based strategy are stored in the replay buffer for parameter updates. The agent iterates repeatedly in the replay buffer, recording new transformations. When the replay buffer is full, the oldest experience is discarded and replaced by new experience. For each round, a small batch of M transformation tuples are randomly sampled from the replay buffer to update their parameters once. Furthermore, the parameters updated by the evaluation network with the training mechanism and the weights of the target network are inherited from the corresponding evaluation network with soft updates, where σ is the soft update ratio. When the E-step ends, the optimal strategy can be selected and downloaded to the controller for online application.

[0101]

[0102] This invention also designs a reward function. The thermal management aspect of this invention mainly includes the temperatures of the engine, battery, and motor. Therefore, the errors between the actual and given values ​​of the engine outlet water temperature Tc, battery temperature Tbat, and cabin temperature Tcab are included in the reward function. The energy management aspect focuses on overall vehicle fuel economy and battery SOC (state of charge). Therefore, the overall fuel consumption mfuel and battery SOC consumption (SOC0-SOC) are included in the reward function. f The energy consumption of components with low power consumption, such as electronic thermostats, is ignored in the reward function. When defining the reward function, all coefficients of the variables are set as penalty coefficients, i.e., negative values, with penalty weights represented by a and b, as shown in the following formula.

[0103]

[0104] By defining a multi-objective reward function that takes into account energy economy, thermal management comfort, and component lifespan, the reinforcement learning agent is trained offline to obtain a control rate that can achieve optimal vehicle energy efficiency, and the result is stored in the form of a neural network for online retrieval.

[0105] The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution. Specifically, this includes:

[0106] The neural network model outputs control signals for the thermal management system, which are then transmitted to the thermal management actuator.

[0107] The neural network model outputs the optimal engine or battery power allocation result to the dynamic coordination control module, and outputs the given value determined by the dynamic coordination control module to the energy management actuator.

[0108] The corresponding actions are performed through the thermal management actuator and the energy management actuator.

[0109] After the thermal management actuator and the energy management actuator perform their respective actions, they collect data through the energy management and thermal management system to reconstruct a low-dimensional representation vector of the energy management and thermal management system's state, thus completing closed-loop feedback.

[0110] This invention provides an integrated energy efficiency optimization control method for intelligent connected new energy vehicles, using system state parameters and real-time traffic information provided by the intelligent network as input variables. With the goals of optimizing system fuel consumption, energy consumption of various components, and cabin comfort, it utilizes interactive reinforcement learning between the intelligent agent and the environment to output the on / off thresholds or action ranges of key electronic control components, as well as the power settings for power components such as the engine and drive motor. By comprehensively considering the coordinated control of thermal management and energy management in hybrid vehicles, and through autonomous control of the engine and drive motor output power, and the states of thermal management components such as the fan and thermostat, the method enables the vehicle to achieve minimum energy consumption costs while ensuring that components such as the engine, drive motor, and power battery operate within their optimal temperature ranges.

[0111] refer to Figure 8 The present invention also discloses an integrated energy efficiency optimization control system for intelligent connected new energy vehicles, the system comprising:

[0112] The data acquisition module 110 is used to acquire road condition information to form a representation vector of traffic information, and to acquire energy management and thermal management system data to form a low-dimensional representation vector of the state of energy management and thermal management system.

[0113] The data fusion module 120 is used to fuse the representation vector of the traffic information and the low-dimensional representation vector of the energy management and thermal management system status into data scale normalization and vector concatenation.

[0114] Training module 130 is used to perform reinforcement learning training by acquiring a whole vehicle model, and to update the neural network parameters of the energy and thermal management integrated control system after reinforcement learning training;

[0115] The model generation module 140 is used to establish a neural network model for energy and thermal management integrated control through the data scaling normalization, vector concatenation, and neural network parameter update of the energy and thermal management integrated control system.

[0116] The control module 150 is used to generate control signals for the thermal management system and the energy management system through the neural network model, and transmit the control signals to the thermal management system components and the energy management system components respectively for action execution.

[0117] Among them, the data acquisition module acquires long-term and short-term road condition information provided by intelligent connected roadside equipment and vehicle-side systems, forming a representation vector of traffic information;

[0118] The traffic information representation vector includes: distance to the next traffic light, the status and countdown of the next traffic light, remaining driving distance, and road traffic situation.

[0119] Perform state variable design and control variable design;

[0120] The design of the state variables includes the design of the environment and vehicle system, engine cooling system, passenger compartment system, motor system, battery system, and intelligent connected transportation environment;

[0121] The control variable design includes: engine radiator fan speed, engine coolant pump speed, electronic thermostat opening, compressor displacement, evaporator fan speed, battery cooling fan speed, engine output power, motor output power, and motor coolant pump speed, which are set as control variables.

[0122] The training module performs reinforcement learning training based on the acquired vehicle model through a preset reinforcement learning training framework.

[0123] After training, the neural network parameters of the energy and thermal management integrated control system are updated.

[0124] In the training process of the energy management and thermal management collaborative control system based on the reinforcement learning training framework, before the offline training process begins, the model generation module initializes the capacity of the replay buffer and stores the transformation tuples sampled from the interaction process of the reinforcement learning training framework based on the exploration strategy in the replay buffer for parameter updates. The module iterates repeatedly in the replay buffer and records new transformations.

[0125] When the replay buffer is full, the oldest experience is discarded and replaced by a new experience. For each round, a small batch of multiple transformation tuples are randomly sampled from the replay buffer to update the parameters once.

[0126] The parameters updated by the evaluation network with training mechanism and the weights of the target network are inherited from the corresponding evaluation network with soft updates, thus establishing a neural network model for integrated control of energy and thermal management.

[0127] The control module, wherein the neural network model outputs control signals for the thermal management system and transmits them to the thermal management actuator;

[0128] The neural network model outputs the optimal engine or battery power allocation result to the dynamic coordination control module, and outputs the given value determined by the dynamic coordination control module to the energy management actuator.

[0129] The corresponding actions are performed through the thermal management actuator and the energy management actuator.

[0130] This invention provides an integrated energy efficiency optimization control system for intelligent connected new energy vehicles, using system state parameters and real-time traffic information provided by the intelligent network as input variables. Aiming at optimizing system fuel consumption, energy consumption of various components, and cabin comfort, it utilizes interactive reinforcement learning between the intelligent agent and the environment to output the on / off thresholds or action ranges of key electronic control components, as well as the power settings for power components such as the engine and drive motor. By comprehensively considering the coordinated control of thermal management and energy management in hybrid vehicles, and autonomously controlling the output power of the engine and drive motor, as well as the status of thermal management components such as the fan and thermostat, the system achieves minimum energy consumption costs while ensuring that components such as the engine, drive motor, and power battery operate within their optimal temperature ranges.

[0131] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communication interface 920, a memory 930, and a communication bus 940. The processor 910, communication interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions from the memory 930 to execute an integrated energy efficiency optimization control method for intelligent connected new energy vehicles. This method includes: acquiring road condition information to form a representation vector of traffic information; and acquiring energy management and thermal management system data to form a low-dimensional representation vector of the energy management and thermal management system state.

[0132] The representation vector of the traffic information and the low-dimensional representation vector of the state of the energy management and thermal management system are fused into data scale normalization and vector concatenation;

[0133] The neural network parameters of the energy and thermal management integrated control system are updated after reinforcement learning training by acquiring a whole vehicle model.

[0134] A neural network model for integrated energy and thermal management control is established by using data scaling normalization, vector concatenation, and neural network parameter updates for the integrated energy and thermal management control system.

[0135] The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution.

[0136] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute an integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the above methods, the method including: acquiring road condition information to form a representation vector of traffic information, and acquiring energy management and thermal management system data to form a low-dimensional representation vector of the state of energy management and thermal management system;

[0138] The representation vector of the traffic information and the low-dimensional representation vector of the state of the energy management and thermal management system are fused into data scale normalization and vector concatenation;

[0139] The neural network parameters of the energy and thermal management integrated control system are updated after reinforcement learning training by acquiring a whole vehicle model.

[0140] A neural network model for integrated energy and thermal management control is established by using data scaling normalization, vector concatenation, and neural network parameter updates for the integrated energy and thermal management control system.

[0141] The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution.

[0142] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an integrated energy efficiency optimization control method for intelligent connected new energy vehicles provided by the above methods. The method includes: acquiring road condition information to form a representation vector of traffic information, and acquiring energy management and thermal management system data to form a low-dimensional representation vector of the state of the energy management and thermal management system.

[0143] The representation vector of the traffic information and the low-dimensional representation vector of the state of the energy management and thermal management system are fused into data scale normalization and vector concatenation;

[0144] The neural network parameters of the energy and thermal management integrated control system are updated after reinforcement learning training by acquiring a whole vehicle model.

[0145] A neural network model for integrated energy and thermal management control is established by using data scaling normalization, vector concatenation, and neural network parameter updates for the integrated energy and thermal management control system.

[0146] The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0148] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An integrated energy efficiency optimization control method for intelligent connected new energy vehicles, characterized in that, include: Obtaining road condition information constitutes a representation vector of traffic information, and obtaining energy management and thermal management system data constitutes a low-dimensional representation vector of the state of energy management and thermal management system. The representation vector of the traffic information and the low-dimensional representation vector of the state of the energy management and thermal management system are fused into data scale normalization and vector concatenation; The neural network parameters of the energy and thermal management integrated control system are updated after reinforcement learning training by acquiring a whole vehicle model. A neural network model for integrated energy and thermal management control is established by using data scaling normalization, vector concatenation, and neural network parameter updates for the integrated energy and thermal management control system. The neural network model generates control signals for the thermal management system and the energy management system, and transmits the control signals to the thermal management system components and the energy management system components respectively for action execution. Specifically, the establishment of the neural network model for integrated energy and thermal management control through data scaling normalization, vector cascading, and neural network parameter updates of the integrated energy and thermal management control system includes: During the training process of the energy and thermal management integrated control system based on the reinforcement learning training framework, before the offline training process begins, the capacity of the replay buffer is initialized, and the transformation tuples sampled from the interaction process of the reinforcement learning training framework based on the exploration are stored in the replay buffer for parameter updates. The process is repeated in the replay buffer and new transformations are recorded. When the replay buffer is full, the oldest experience is discarded and replaced by a new experience. For each round, a small batch of multiple transformation tuples are randomly sampled from the replay buffer to update the parameters once. The parameters updated by the evaluation network with training mechanism and the weights of the target network are inherited from the corresponding evaluation network with soft updates, thus establishing a neural network model for integrated control of energy and thermal management.

2. The integrated energy efficiency optimization control method for intelligent connected new energy vehicles according to claim 1, characterized in that, The acquired road condition information constitutes a representation vector of traffic information, specifically including: Acquire long-term and short-term road condition information provided by intelligent connected roadside equipment and vehicle-side systems to form a representation vector of traffic information; The traffic information representation vector includes: distance to the next traffic light, the status and countdown of the next traffic light, remaining driving distance, and road traffic situation.

3. The integrated energy efficiency optimization control method for intelligent connected new energy vehicles according to claim 1, characterized in that, The acquired energy management and thermal management system data constitutes a low-dimensional representation vector of the energy management and thermal management system state, specifically including: Perform state variable design and control variable design; The design of the state variables includes the design of the environment and vehicle system, engine cooling system, passenger compartment system, motor system, battery system, and intelligent connected transportation environment; The control variable design includes setting the engine radiator fan speed, engine coolant pump speed, electronic thermostat opening, compressor displacement, evaporator fan speed, battery cooling fan speed, engine output power, motor output power, and motor coolant pump speed as control variables.

4. The integrated energy efficiency optimization control method for intelligent connected new energy vehicles according to claim 1, characterized in that, The process of acquiring a whole vehicle model for reinforcement learning training, followed by updating the neural network parameters of the energy and thermal management integrated control system after reinforcement learning training, specifically includes: Based on the acquired vehicle model, reinforcement learning training is performed through a pre-set reinforcement learning training framework; After training, the neural network parameters of the energy and thermal management integrated control system are updated.

5. The integrated energy efficiency optimization control method for intelligent connected new energy vehicles according to claim 1, characterized in that, The neural network model generates control signals for the thermal management system and the energy management system, and transmits these control signals to the thermal management system components and the energy management system components respectively for execution. Specifically, this includes: The neural network model outputs control signals for the thermal management system, which are then transmitted to the thermal management actuator. The neural network model outputs the optimal engine or battery power allocation result to the dynamic coordination control module, and outputs the given value determined by the dynamic coordination control module to the energy management actuator. The corresponding actions are performed through the thermal management actuator and the energy management actuator.

6. An integrated energy efficiency optimization control system for intelligent connected new energy vehicles, characterized in that, The system includes: The data acquisition module is used to acquire road condition information to form a representation vector of traffic information, and to acquire energy management and thermal management system data to form a low-dimensional representation vector of the state of energy management and thermal management system. The data fusion module is used to fuse the representation vector of the traffic information and the low-dimensional representation vector of the energy management and thermal management system status into data scale normalization and vector concatenation; The training module is used to perform reinforcement learning training by acquiring a whole vehicle model, and to update the neural network parameters of the energy and thermal management integrated control system after reinforcement learning training; The model generation module is used to establish a neural network model for the integrated energy and thermal management control system through data scaling normalization, vector concatenation, and neural network parameter updates of the integrated energy and thermal management control system. The control module is used to generate control signals for the thermal management system and the energy management system through the neural network model, and transmit the control signals to the thermal management system components and the energy management system components respectively for action execution; Specifically, the establishment of the neural network model for integrated energy and thermal management control through data scaling normalization, vector cascading, and neural network parameter updates of the integrated energy and thermal management control system includes: During the training process of the energy and thermal management integrated control system based on the reinforcement learning training framework, before the offline training process begins, the capacity of the replay buffer is initialized, and the transformation tuples sampled from the interaction process of the reinforcement learning training framework based on the exploration are stored in the replay buffer for parameter updates. The process is repeated in the replay buffer and new transformations are recorded. When the replay buffer is full, the oldest experience is discarded and replaced by a new experience. For each round, a small batch of multiple transformation tuples are randomly sampled from the replay buffer to update the parameters once. The parameters updated by the evaluation network with training mechanism and the weights of the target network are inherited from the corresponding evaluation network with soft updates, thus establishing a neural network model for integrated control of energy and thermal management.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the integrated energy efficiency optimization control method for intelligent connected new energy vehicles as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the integrated energy efficiency optimization control method for intelligent connected new energy vehicles as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the integrated energy efficiency optimization control method for intelligent connected new energy vehicles as described in any one of claims 1 to 5.

Images