Vehicle energy management model determination method and device, vehicle energy control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle energy management model determination method, a vehicle energy control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining first sample data which at least comprises first vehicle data and environment data; determining a vehicle energy management simulation model according to the first sample data and the first training model; second sample data are obtained, and the second sample data comprise second vehicle data; a vehicle energy management model is determined according to the second sample data, the vehicle energy management simulation model and the second training model, and the vehicle energy management model is used for determining corresponding vehicle operation control information according to different vehicle data and environment data, so that the internal combustion engine and the motor are coordinated in real time through a reinforcement learning agent, and the vehicle energy management efficiency is improved. And energy waste is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, in particular to a vehicle energy management model determination method, a vehicle energy control method and device, an electronic device and a storage medium. BACKGROUND

[0002] In the field of new energy vehicle energy management, currently, control strategies mainly rely on fixed rules based on expert experience. These control strategies coordinate the work of internal combustion engines and electric motors through predefined logic (such as SOC threshold control) to optimize energy consumption. However, this approach is based on static data and cannot adjust the internal combustion engine and electric motor in real time according to user control instructions to reduce energy consumption. Therefore, how to reduce energy consumption is a problem that needs to be solved urgently. SUMMARY

[0003] Some embodiments of the present application aim to provide a vehicle energy management model determination method, a vehicle energy control method and device, an electronic device and a storage medium. Through the technical solutions of the embodiments of the present application, first sample data is obtained, wherein the first sample data at least includes first vehicle data and environment data; a vehicle energy management simulation model is determined according to the first sample data and a first training model; second sample data is obtained, wherein the second sample data includes second vehicle data; and the vehicle energy management model is determined according to the second sample data, the vehicle energy management simulation model and a second training model. The vehicle energy management model is used to determine corresponding vehicle operation control information according to different vehicle data and environment data. In the embodiments of the present application, a training model is simulated and trained by using vehicle data and environment data to obtain a vehicle energy management simulation model. Then, another training model is trained again according to the vehicle energy management simulation model and vehicle data to obtain a vehicle energy management model. The vehicle energy management model is used to determine corresponding vehicle operation control information according to different vehicle data and environment data. In this way, the internal combustion engine and electric motor are coordinated in real time by the reinforcement learning agent to reduce energy waste.

[0004] In a first aspect, some embodiments of the present application provide a vehicle energy management model determination method, comprising: obtaining first sample data, wherein the first sample data at least includes first vehicle data and environment data; determining a vehicle energy management simulation model according to the first sample data and a first training model; obtaining second sample data, wherein the second sample data includes second vehicle data; determine the vehicle energy management model according to different vehicle data and environment data.

[0005] Some embodiments of the present application obtain a vehicle energy management simulation model by simulating and training a training model with vehicle data and environment data, and then train another training model according to the vehicle energy management simulation model and vehicle data to obtain a vehicle energy management model, which is used to determine corresponding vehicle operation control information according to different vehicle data and environment data. In this way, the internal combustion engine and the electric motor are coordinated in real time by the reinforcement learning agent, and energy waste is reduced.

[0006] Optionally, the method further comprises: performing knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model. Optionally, the performing knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model comprises: performing distillation training on the vehicle energy management model and a target vehicle distillation model with preset training parameters; calculating a third loss function in each training process, wherein the third loss function is determined using a KL loss function; in a case where a third loss value of the third loss function is less than a third preset value, determining the target vehicle distillation model as the vehicle energy distillation model.

[0007] Some embodiments of the present application obtain a vehicle energy distillation model after knowledge distillation of a vehicle energy management model, and the computing power requirement is reduced from 10 TOPS to 5 TOPS, and the memory occupancy is reduced from 100 KB to 50 KB (document 6.4 software and hardware integration scheme), thereby reducing the hardware cost of the vehicle controller.

[0008] Optionally, the determining the vehicle energy management simulation model according to the first sample data and the first training model comprises: training the first training model with the first sample data; calculating a first loss value in each training process according to a first loss function, wherein the first loss function is determined according to fuel consumption, engine start-stop times, and battery power; in a case where the first loss value meets a first preset value, determining a first training model corresponding to the first loss value as the vehicle energy management simulation model.

[0009] According to some embodiments of the present application, a multi-objective reward function, i.e., a first loss function, is established according to the fuel consumption, the number of engine start-stop times, and the battery power, so that the number of engine start-stop times can be controlled, the vibration and noise can be reduced, and the driving comfort can be improved.

[0010] Optionally, the first loss value in each training process is calculated according to the first loss function, wherein the first loss function is determined according to the fuel consumption, the number of engine start-stop times, and the battery power, and includes: determining a first weight corresponding to the fuel consumption, a second weight corresponding to the number of engine start-stop times, and a third weight corresponding to the battery power; calculating a first product of the fuel consumption and the first weight, a second product of the number of engine start-stop times and the second weight, and a third product of the battery power and the third weight, respectively; determining the first loss function according to the sum of the first product, the second product, and the third product.

[0011] Optionally, the vehicle energy management model is determined according to the second sample data and the second training model, and includes: determining training data according to the second sample data and the vehicle energy management simulation model; training a second training model using the training data, wherein the second training model includes a conservative Q learning algorithm; determining a second loss function according to a double Q value mean square error value, an offline training loss value, an Actor loss value, and a Critic loss value; in a case where a second loss value of the second loss function meets a second preset value, determining a second training model corresponding to the second loss value as the vehicle energy management model.

[0012] According to some embodiments of the present application, the second sample data and the vehicle energy management simulation model are comprehensively considered to obtain the training data, the problems of generalization and individualization are solved, the offline reinforcement learning and the data backflow mechanism are used to learn the driving habits, the customized strategy is generated, and the user satisfaction is improved.

[0013] In a second aspect, some embodiments of the present application provide a vehicle energy control method, including: obtaining actual vehicle data and actual environment data of a vehicle; determining running control information corresponding to the vehicle using a vehicle energy management model obtained by the determination method of the vehicle energy management model according to any one of the first aspect, wherein the running control information at least includes vehicle torque data, battery power data, and vehicle speed. Some embodiments of the present application install the trained vehicle energy management model on the controller of the vehicle. The vehicle energy management model can be input into the vehicle energy management model according to real-time collected vehicle data and actual environment data, to obtain the current vehicle corresponding operation control information, for example, multi-objective optimization to reduce the number of start-stop, thereby reducing fuel consumption and maintenance costs.

[0014] In a third aspect, some embodiments of the present application provide a device for determining a vehicle energy management model, comprising: A first obtaining module is configured to obtain first sample data, wherein the first sample data comprises at least first vehicle data and environment data; A first training module is configured to determine a vehicle energy management simulation model according to the first sample data and a first training model; A second obtaining module is configured to obtain second sample data, wherein the second sample data comprises second vehicle data; A second training module is configured to determine the vehicle energy management model according to the second sample data, the vehicle energy management simulation model and a second training model, wherein the vehicle energy management model is configured to determine corresponding vehicle operation control information according to different vehicle data and environment data.

[0015] Some embodiments of the present application simulate and train the training model by using vehicle data and environment data to obtain a vehicle energy management simulation model, and then train another training model according to the vehicle energy management simulation model and vehicle data to obtain a vehicle energy management model. The vehicle energy management model is configured to determine corresponding vehicle operation control information according to different vehicle data and environment data. In this way, the reinforcement learning agent can real-time coordinate the internal combustion engine and the electric motor to reduce energy waste.

[0016] Optionally, the second training module is configured to: Perform knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model. Optionally, the second training module is configured to: Perform distillation training on the vehicle energy management model and a target vehicle distillation model by using preset training parameters; Calculate a third loss function in each training process, wherein the third loss function is determined by using a KL loss function; In a case where a third loss value of the third loss function is less than a third preset value, determine the target vehicle distillation model as the vehicle energy distillation model.

[0017] Some embodiments of the present application obtain a distilled vehicle energy model by distilling a vehicle energy management model, reducing the computing power requirement from 10 TOPS to 5 TOPS, and reducing the memory occupation from 100 KB to 50 KB (document 6.4 software and hardware integration scheme), thereby reducing the hardware cost of the vehicle controller.

[0018] Optionally, the first training module is configured to: train the first training model by using the first sample data; calculate a first loss value in each training process according to a first loss function, wherein the first loss function is determined according to the fuel consumption, the engine start-stop times and the battery power; determine the first training model corresponding to the first loss value as the vehicle energy management simulation model in a case where the first loss value meets a first preset value.

[0019] Some embodiments of the present application establish a multi-objective reward function, i.e., a first loss function, according to the fuel consumption, the engine start-stop times and the battery power, so as to control the engine start-stop times, reduce vibration and noise, and improve the driving comfort.

[0020] Optionally, the first training module is configured to: determine a first weight corresponding to the fuel consumption, a second weight corresponding to the engine start-stop times and a third weight corresponding to the battery power; respectively calculate a first product of the fuel consumption and the first weight, a second product of the engine start-stop times and the second weight, and a third product of the battery power and the third weight; determine the first loss function according to a sum of the first product, the second product and the third product.

[0021] Optionally, the second training module is configured to: determine training data according to the second sample data and the vehicle energy management simulation model; train a second training model by using the training data, wherein the second training model comprises a conservative Q learning algorithm; determine a second loss function according to a double Q value mean square error value, an offline training loss value, an Actor loss value and a Critic loss value; determine the second training model corresponding to the second loss value as the vehicle energy management model in a case where a second loss value of the second loss function meets a second preset value.

[0022] Some embodiments of the present application comprehensively consider the second sample data and the vehicle energy management simulation model to obtain training data, solve the problems of generalization and individualization, and use offline reinforcement learning and a data backflow mechanism to learn driving habits, generate customized strategies, and improve user satisfaction.

[0023] In a fourth aspect, some embodiments of the present application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining a vehicle energy management model according to any of the embodiments of the first aspect.

[0024] In a fifth aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, wherein the program is executable on a processor to implement the method for determining a vehicle energy management model according to any of the embodiments of the first aspect.

[0025] In a sixth aspect, some embodiments of the present application provide a computer program product, including a computer program, wherein the computer program is executable on a processor to implement the method for determining a vehicle energy management model according to any of the embodiments of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the following will briefly introduce the drawings needed to be used in some embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0027] Figure 1 A flowchart of a method for determining a vehicle energy management model according to an embodiment of the present application is shown in FIG. 1; Figure 2 A flowchart of another method for determining a vehicle energy management model according to an embodiment of the present application is shown in FIG. 2; Figure 3 A flowchart of a vehicle energy management simulation model according to an embodiment of the present application is shown in FIG. 3; Figure 4 A flowchart of a vehicle energy management model according to an embodiment of the present application is shown in FIG. 4; Figure 5 A flowchart of a vehicle energy distillation model according to an embodiment of the present application is shown in FIG. 5; Figure 6 A flowchart of a model detection method according to an embodiment of the present application is shown in FIG. 6; Figure 7A flowchart of the vehicle controller performing steps provided by an embodiment of the present application; Figure 8 A flowchart of the vehicle energy management method provided by an embodiment of the present application; Figure 9 A flowchart of the vehicle controller performing steps provided by another embodiment of the present application; Figure 10 A structural diagram of the determination device of the vehicle energy management model provided by an embodiment of the present application; Figure 11 An electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0029] It should be noted that similar reference numerals and letters refer to similar items in the following drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", and the like are merely used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0030] In the field of new energy vehicle energy management, currently mainly relies on the control strategy of fixed rules based on expert experience, these control strategies coordinate the work of internal combustion engine and electric motor through predefined logic (such as SOC threshold control) to optimize energy consumption, but this way is determined based on static data, and cannot be real-time according to the control instruction of the user to the vehicle, and the vehicle can timely coordinate the internal combustion engine and the electric motor according to the control instruction to reduce energy consumption, therefore, how to reduce energy consumption is the problem that needs to be solved at present, in view of this, some embodiments of the application provide a kind of vehicle energy management model determination method, the method comprises obtaining first sample data, wherein first sample data at least includes first vehicle data and environmental data;Determine vehicle energy management simulation model according to first sample data and first training model;Obtain second sample data, wherein the second sample data includes second vehicle data;According to second sample data, vehicle energy management simulation model and second training model, determine vehicle energy management model, vehicle energy management model is used to determine corresponding vehicle operation control information according to different vehicle data and environmental data, in the application embodiment, vehicle data and environmental data are used to simulate training model, obtain vehicle energy management simulation model, and then according to the vehicle energy management simulation model and vehicle data, another training model is trained again, and vehicle energy management model is obtained, which is used to determine corresponding vehicle operation control information according to different vehicle data and environmental data, so that the internal combustion engine and the electric motor are coordinated in real time by reinforcement learning intelligent agent, and energy waste is reduced.

[0031] As Figure 1 shown, the embodiments of the application provide a kind of vehicle energy management model determination method, the method comprises: S101、obtain first sample data, wherein first sample data at least includes first vehicle data and environmental data; Specifically, the application embodiment is applied to terminal equipment, which can be a server or a server. The terminal device obtains first sample data, and the first sample data includes first vehicle data and environmental data. The first vehicle data includes vehicle speed, acceleration, battery SOC and output torque. The environmental data includes slope and traffic condition.

[0032] S102, determine vehicle energy management simulation model according to first sample data and first training model; Specifically, the terminal device pre-processes the first sample data to obtain a training sample set corresponding to the first sample data, including state s, action a, reward r, next state s', and termination flag done.

[0033] The first training model is trained by using the training sample set, in each training process, a loss function value is calculated, and in a case where the loss function value is less than a preset value, a vehicle energy management simulation model is obtained, where the first training model can be a deep Q network (DQN) or an Actor-Critic model.

[0034] S103, acquire second sample data, where the second sample data includes second vehicle data; Specifically, after the terminal device simulates the vehicle energy management simulation model, the terminal device pre-processes the second vehicle data to obtain processed data, for example, a state s, an action a, a reward r, a next state s', and a termination flag done.

[0035] S104, determine a vehicle energy management model according to the second sample data, the vehicle energy management simulation model, and a second training model, where the vehicle energy management model is used to determine corresponding vehicle operation control information according to different vehicle data and environment data.

[0036] Specifically, the terminal device inputs the processed data into the vehicle energy management simulation model to obtain vehicle operation control information, and then trains the second training model according to the processed data and the vehicle operation control information, calculates a loss function value of each training, and in a case where the loss value is less than a preset value, determines the second training model corresponding to the loss value as the vehicle energy management model, so that the vehicle energy management model obtained in this way is used to determine corresponding vehicle operation control information according to different vehicle data and environment data.

[0037] Some embodiments of the present application simulate and train a training model by using vehicle data and environment data to obtain a vehicle energy management simulation model, and then train another training model according to the vehicle energy management simulation model and the vehicle data to obtain a vehicle energy management model, where the vehicle energy management model is used to determine corresponding vehicle operation control information according to different vehicle data and environment data, so that the reinforcement learning agent can real-time coordinate the internal combustion engine and the electric motor to reduce energy waste.

[0038] Another embodiment of the present application further supplements the method for determining the vehicle energy management model provided in the above embodiment.

[0039] The vehicle energy management in the embodiment of the present application includes two parts, namely, a model training process on a terminal device and a vehicle-end deployment two-stage process: • as Figure 2As shown, in the model training stage, data is generated by using the vehicle model and the external environment, an Actor-Critic (double network collaborative mechanism) agent, i.e., a vehicle energy management simulation model, is trained, and then example line reinforcement learning is performed on the vehicle energy management simulation model using second sample data to obtain a vehicle energy management model; and then the vehicle energy management model is distilled and compressed to obtain a vehicle energy distillation model.

[0040] As shown in FIG. 1, the vehicle energy management model is trained in the model training stage. Figure 7 As shown, in the deployment stage, the compiled model (vehicle energy distillation model or vehicle energy management model) is executed by the VCU controller to perform real-time control, and ensure real-time signal transmission, such as speed and acceleration, which are input into the agent module, i.e., the trained model, with a real-time cycle to ensure control responsiveness.

[0041] Optionally, the method further includes: performing knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model. Optionally, performing knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model includes: performing distillation training on the vehicle energy management model and a target vehicle distillation model using preset training parameters; calculating a third loss function in each training process, wherein the third loss function is determined using a KL loss function; in a case where a third loss value of the third loss function is less than a third preset value, determining the target vehicle distillation model as the vehicle energy distillation model.

[0042] As shown in FIG. 1, the vehicle energy management model is trained in the model training stage. Figure 5 As shown in FIG. 1, the vehicle energy management model is trained in the model training stage. A teacher-student network model can be used, i.e., an original strategy (Net-Teacher) generates samples, a small-scale network (Net-Student) learns, the number of hidden layer neurons is reduced, and then knowledge distillation compression is performed, based on the teacher-student network architecture, the loss function is minimized to output differences, the teacher model (vehicle energy management model) is compressed to make the compressed model, i.e., the vehicle energy distillation model, have a size of 50 KB, meet the real-time requirement of the vehicle end, in this way, after the model is compressed, the number of parameters is reduced by 50%, the computing power requirement is reduced to 5 TOPS, and the 50 ms cycle of the vehicle end is met.

[0043] Some embodiments of the present application perform knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model after knowledge distillation, reduce the computing power requirement from 10 TOPS to 5 TOPS, reduce the memory occupancy from 100 KB to 50 KB, and reduce the hardware cost of the vehicle controller.

[0044] Optionally, a vehicle energy management simulation model is determined based on the first sample data and the first training model, including: The first training model is trained using the first sample data; The first loss value is calculated for each training process based on the first loss function, which is determined based on fuel consumption, engine start-stop times, and battery charge. If the first loss value meets the first preset value, the first training model corresponding to the first loss value is determined as the vehicle energy management simulation model.

[0045] The first training model includes at least a Deep Q-Network (DQN) or an Actor-Critic model.

[0046] In this application embodiment, the DQN model can achieve multi-objective optimization, while the Actor-Critic model is more suitable for continuous motion spaces (such as torque control) and has a faster response.

[0047] Optionally, a first loss value is calculated for each training process based on a first loss function, wherein the first loss function is determined based on fuel consumption, engine start-stop frequency, and battery charge, and includes: Determine the first weight corresponding to fuel consumption, the second weight corresponding to the number of engine start-stop cycles, and the third weight corresponding to battery charge. Calculate the first product of fuel consumption and the first weight, the second product of engine start-stop times and the second weight, and the third product of battery charge and the third weight, respectively. The first loss function is determined by the sum of the first product, the second product, and the third product.

[0048] like Figure 3 As shown, the embodiments of this application use first sample data, which includes first vehicle data (vehicle speed, acceleration, battery SOC) and external environment (slope, traffic); The intelligent agent is the first training model: the Actor-Critic network. The Actor generates actions (target torque, clutch switch), and the Critic evaluates the reward.

[0049] The reward function, also known as the first loss function, considers factors such as fuel consumption, engine start-stop frequency, and SOC (battery charge) stability. The formula is a weighted sum of multiple objectives. First loss function (reward function) = w1 × (-fuel consumption) + w2 × (-number of start-stop cycles) + w3 × (SOC error penalty); Among them, w1 is the first weight, w2 is the second weight, and w3 is the third weight. Through testing and optimization, a minimum reward value is determined to ensure the lowest fuel consumption and the fewest start-stop cycles.

[0050] By training the first training model, the energy distribution is optimized in real time, the actor outputs the action control vehicle model, the fuel consumption and emissions are reduced, the multi-performance target optimization agent is trained, the actor-critic network is integrated with the multi-target reward function (fuel consumption, start-stop times, SOC holding), and dynamic balance is achieved.

[0051] According to some embodiments of the application, a multi-objective reward function, i.e., a first loss function, is established according to the fuel consumption, the number of engine starts and stops, and the battery power, so that the number of engine starts and stops can be controlled, the vibration and noise can be reduced, and the driving comfort can be improved.

[0052] Optionally, according to the second sample data and the second training model, a vehicle energy management model is determined, comprising: According to the second sample data and the vehicle energy management simulation model, training data is determined; The second training model includes a conservative Q-learning algorithm. According to the double Q value mean square error value, the offline training loss value, the actor loss value and the critic loss value, a second loss function is determined; In the case where the second loss value of the second loss function meets the second preset value, the second training model corresponding to the second loss value is determined as the vehicle energy management model.

[0053] As shown in Figure 4 To solve the generalization problem, the offline reinforcement learning and CQL algorithm are used in the embodiments of the application, i.e., the training data is constructed by using real vehicle data, the strategy robustness is improved by using the conservative Q-learning, and the generalization problem is solved.

[0054] • Data processing: obtaining second sample data, extracting features from the second sample data, i.e., extracting features (acceleration, speed, SOC) from real vehicle data, normalizing, constructing training data, including state s, action a, reward r, next state s', termination flag done.

[0055] The conservative Q-learning algorithm (CQL) is trained by using the training data, and the second loss function in each training process is calculated: ; Wherein: a is the conservative weight, abatch is the action of offline data, and the strategy is prevented from being overly optimistic.

[0056] In the embodiments of the present application, the second loss function is continuously calculated, the function value trap of the second loss function is judged, the Actor and Critic networks are continuously optimized through the function value of the second loss function, the gradient descent optimization is used, the target network is soft updated, the simulation deficiency is supplemented by using real vehicle data, the strategy robustness is improved, and the SOC drift is avoided.

[0057] Some embodiments of the present application comprehensively consider the second sample data and the vehicle energy management simulation model to obtain training data, solve the problems of generalization and individualization, use offline reinforcement learning and data backflow mechanism, learn driving habits through the data backflow mechanism, generate customized strategies, and improve user satisfaction.

[0058] As shown in Figure 6 The embodiments of the present application also include a training monitoring and evaluation module for recording the model training process at each step, integrating TensorBoard in Python, recording the Actor loss, Critic loss, reward value in real time, calculating the mean square error of the target Q value and the current Q value, dynamically adjusting the hyperparameters, improving the training convergence speed by 20%, and avoiding abnormal strategies.

[0059] Some embodiments of the present application provide a vehicle energy control method, comprising: obtaining actual vehicle data and actual environment data of the vehicle; using the vehicle energy management model obtained by the above-mentioned vehicle energy management model determination method to determine the running control information corresponding to the vehicle, wherein the running control information at least includes vehicle torque data, battery power data and vehicle speed. Specifically, after the terminal device completes the training of the vehicle energy management model, the terminal device loads the vehicle energy management model to the controller of the vehicle, the controller of the vehicle obtains the actual vehicle data and the actual environment data of the vehicle, inputs the actual vehicle data and the actual environment data into the vehicle energy management model, and determines the running control information corresponding to the vehicle.

[0060] The editing environment of the terminal device is Matlab / Simulink to build a VCU model, Embedded Coder to convert C code, which meets the AUTOSAR standard, uses CANoe engineering simulation signals for hardware-in-the-loop testing to verify the safety of the strategy. After compilation, it is deployed to the PDC controller to realize 50ms real-time control.

[0061] The vehicle collects the terminal recording state and action, and returns to the cloud through Ethernet and CAN bus, the terminal device combines offline data for online training, strengthens the agent, and continuously updates the agent. The Actor-Critic agent optimizes multiple objectives through the reward function, reduces fuel consumption, reduces computing power through knowledge distillation, and improves generalization through data update.

[0062] As Figure 9 shown, the vehicle controller receives user input pedals, modes, etc., and obtains driving information, including vehicle speed, engine speed, clutch state, etc., and target SOC, inputs these data into the multi-performance target optimization agent, i.e., the vehicle energy management model, to obtain engine start or stop, engine power and speed, etc., so as to improve fuel economy (fuel consumption reduced by 15-20%), improve response speed (delay reduced by 50%), and save hardware cost. Exemplarily, the hardware configuration provided by the present application is as follows: • Cloud server: store driving data.

[0063] • Industrial computer: train agent (Python / Matlab environment).

[0064] • VCU controller: TC399 chip, computing power 5TOPS, memory 50KB (document 6.4).

[0065] • Cockpit chip CSC: Snapdragon 8295, running global planning module.

[0066] • Software implementation: • Training end: SOC planning model (DP algorithm), multi-performance target agent module (Actor-Critic), offline training module (CQL algorithm).

[0067] • Car end: compile module to generate C code, data storage module (CAN protocol).

[0068] • Operation steps, as Figure 8 shown: 1. Data acquisition: record speed, acceleration, SOC, etc. when the vehicle is running.

[0069] 2. Offline training: after data preprocessing, input CQL algorithm to update strategy.

[0070] 3. Distillation compression: after Net-Student training, output small model, i.e., distilled model, vehicle energy distillation model.

[0071] 4. Deployment: compile to C code, burn to vehicle VCU after hardware-in-the-loop test.

[0072] 5. Real-time control: vehicle end agent outputs torque control according to input signal.

[0073] 6. Data backflow: update strategy to adapt to user habits, Ethernet + CAN bus backflow data, online training to strengthen personalized strategy.

[0074] 7. Compiling deployment module: Matlab / Simulink to C code, hardware-in-the-loop test to ensure secure deployment.

[0075] The test results are as follows: • Under standard operating conditions (vehicle speed 0-140km / h, SOC 10-100%), fuel consumption is reduced by 18%, and start-stop times are reduced by 30%.

[0076] • NVH optimization: noise reduction of 5dB.

[0077] Based on the above vehicle energy control method, abnormality detection can also be performed, and alarm display can also be performed.

[0078] Some embodiments of the present application install the trained vehicle energy management model on the controller of the vehicle. The current vehicle running control information can be obtained by inputting the real-time collected vehicle data and actual environment data into the vehicle energy management model, for example, multi-objective optimization to reduce start-stop times, thereby reducing fuel consumption and maintenance costs.

[0079] It should be noted that each implementable manner in the present embodiment can be implemented alone or in any combination without conflict.

[0080] Another embodiment of the present application provides a vehicle energy management model determination device for executing the vehicle energy management model determination method provided in the above embodiment.

[0081] As shown in Figure 10 FIG. 1 is a structural schematic diagram of a vehicle energy management model determination device provided in an embodiment of the present application. The vehicle energy management model determination device includes a first acquisition module 1001, a first training module 1002, a second acquisition module 1003, and a second training module 1004, wherein: The first acquisition module 1001 is configured to acquire first sample data, wherein the first sample data at least includes first vehicle data and environment data; The first training module 1002 is configured to determine a vehicle energy management simulation model according to the first sample data and a first training model; The second acquisition module 1003 is configured to acquire second sample data, wherein the second sample data includes second vehicle data; The second training module 1004 is configured to determine a vehicle energy management model according to the second sample data, the vehicle energy management simulation model, and a second training model, and the vehicle energy management model is configured to determine corresponding vehicle running control information according to different vehicle data and environment data.

[0082] With regard to the apparatus in the embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0083] Some embodiments of the present application simulate training of a training model by using vehicle data and environment data to obtain a vehicle energy management simulation model, and then train another training model again according to the vehicle energy management simulation model and the vehicle data to obtain a vehicle energy management model, which is used to determine corresponding vehicle operation control information according to different vehicle data and environment data. In this way, the internal combustion engine and the electric motor are coordinated in real time by the reinforcement learning agent, and energy waste is reduced.

[0084] Another embodiment of the present application further supplements the determination apparatus of the vehicle energy management model provided in the above embodiment.

[0085] Optionally, the second training module is configured to: Perform knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model. Optionally, the second training module is configured to: Perform distillation training on the vehicle energy management model and the target vehicle distillation model by using preset training parameters; Calculate a third loss function in each training process, wherein the third loss function is determined by using a KL loss function; In a case where a third loss value of the third loss function is less than a third preset value, determine the target vehicle distillation model as the vehicle energy distillation model.

[0086] Some embodiments of the present application perform knowledge distillation on the vehicle energy management model to obtain a vehicle energy distillation model after knowledge distillation, and the computing power requirement is reduced from 10 TOPS to 5 TOPS, and the memory occupancy is reduced from 100 KB to 50 KB, thereby reducing the hardware cost of the vehicle controller.

[0087] Optionally, the first training module is configured to: Train the first training model by using first sample data; Calculate a first loss value in each training process according to a first loss function, wherein the first loss function is determined according to fuel consumption, engine start-stop times, and battery power; In a case where the first loss value meets a first preset value, determine the first training model corresponding to the first loss value as the vehicle energy management simulation model.

[0088] According to some embodiments of the present application, a multi-objective reward function, i.e., a first loss function, is established according to the fuel consumption, the number of engine start-stop times, and the battery power, so that the number of engine start-stop times can be controlled, the vibration and noise can be reduced, and the driving comfort can be improved.

[0089] Optionally, the first training module is configured to: determine a first weight corresponding to the fuel consumption, a second weight corresponding to the number of engine start-stop times, and a third weight corresponding to the battery power; calculate a first product of the fuel consumption and the first weight, a second product of the number of engine start-stop times and the second weight, and a third product of the battery power and the third weight, respectively; determine the first loss function according to a sum of the first product, the second product, and the third product.

[0090] Optionally, the second training module is configured to: determine training data according to the second sample data and the vehicle energy management simulation model; train the second training model using the training data, the second training model comprising a conservative Q-learning algorithm; determine the second loss function according to the double Q-value mean square error value, the offline training loss value, the Actor loss value, and the Critic loss value; in a case where a second loss value of the second loss function satisfies a second preset value, determine the second training model corresponding to the second loss value as the vehicle energy management model.

[0091] According to some embodiments of the present application, the second sample data and the vehicle energy management simulation model are comprehensively considered to obtain the training data, the problems of generalization and individualization are solved, the offline reinforcement learning and the data backflow mechanism are used to learn the driving habits, the customized strategy is generated, and the user satisfaction is improved.

[0092] As to the apparatus in the present embodiment, the specific manners in which various modules perform operations have been described in detail in the embodiments relating to the method, and thus will not be described in detail here.

[0093] It should be noted that each implementable manner in the present embodiment can be implemented alone or in any combination manner without conflict.

[0094] The present embodiment further provides a computer-readable storage medium having a computer program stored thereon, and the program can implement the operations of the method corresponding to any embodiment of the determination method of the vehicle energy management model provided in the above-described embodiments when executed by a processor.

[0095] The embodiment of the present application further provides a computer program product, the computer program product comprises a computer program, and when the computer program is executed by a processor, the processor can implement the operation of the method corresponding to any embodiment of the determination method of the vehicle energy management model provided by the above embodiment.

[0096] As shown in Figure 11 some embodiments of the present application provide an electronic device 1100, which comprises a memory 1110, a processor 1120 and a computer program stored in the memory 1110 and executable on the processor 1120, wherein when the processor 1120 reads the program from the memory 1110 through the bus 1130 and executes the program, the processor 1120 can implement the method of any embodiment of the determination method of the vehicle energy management model provided by the above embodiment.

[0097] The processor 1120 can process digital signals and can include various computing structures. For example, a complex instruction set computer structure, a reduced instruction set computer structure or a structure implementing a combination of multiple instruction sets. In some examples, the processor 1120 can be a microprocessor.

[0098] The memory 1110 can be used to store instructions executed by the processor 1120 or data related to the execution process of the instructions. The instructions and / or data can include code for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 1120 of the embodiments of the present disclosure can be used to execute instructions in the memory 1110 to implement the method shown in the above. The memory 1110 includes dynamic random access memory, static random access memory, flash memory, optical memory or other memory well known to those skilled in the art.

[0099] The above is only an embodiment of the present application and does not limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0100] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be limited by the protection scope of the claims.

[0101] It is to be noted that, as used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component" can include a combination of two or more components. Additionally, the terms "comprise," "comprises," and "comprising," or any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless otherwise indicated herein, the terms "first," "second," "third," etc., are used herein merely as labels, and are not intended to impose ordinal import.

Claims

1. A method for determining a vehicle energy management model, characterized in that, The method includes: Acquire first sample data, wherein the first sample data includes at least first vehicle data and environmental data; Based on the first sample data and the first training model, a vehicle energy management simulation model is determined; Obtain second sample data, wherein the second sample data includes second vehicle data; Based on the second sample data, the vehicle energy management simulation model, and the second training model, the vehicle energy management model is determined. The vehicle energy management model is used to determine the corresponding vehicle operation control information based on different vehicle data and environmental data.

2. The method for determining the vehicle energy management model according to claim 1, characterized in that, The method further includes: The vehicle energy management model is subjected to knowledge distillation to obtain the vehicle energy distillation model.

3. The method for determining the vehicle energy management model according to claim 2, characterized in that, The knowledge distillation of the vehicle energy management model to obtain the vehicle energy distillation model includes: The vehicle energy management model and the target vehicle distillation model are trained using preset training parameters; Calculate the third loss function for each training process, wherein the third loss function is determined using the KL loss function; If the third loss value of the third loss function is less than the third preset value, the target vehicle distillation model is determined as the vehicle energy distillation model.

4. The method for determining the vehicle energy management model according to claim 1, characterized in that, The step of determining the vehicle energy management simulation model based on the first sample data and the first training model includes: The first training model is trained using the first sample data; The first loss value is calculated for each training process based on the first loss function, wherein the first loss function is determined based on fuel consumption, engine start-stop times, and battery charge. If the first loss value satisfies the first preset value, the first training model corresponding to the first loss value is determined as the vehicle energy management simulation model.

5. The method for determining the vehicle energy management model according to claim 4, characterized in that, The first loss value is calculated for each training process based on a first loss function, wherein the first loss function is determined based on fuel consumption, engine start-stop frequency, and battery charge, and includes: Determine a first weight corresponding to the fuel consumption, a second weight corresponding to the number of engine start-stop cycles, and a third weight corresponding to the battery charge. Calculate the first product of the fuel consumption and the first weight, the second product of the number of engine start-stop cycles and the second weight, and the third product of the battery charge and the third weight, respectively. The first loss function is determined based on the sum of the first product, the second product, and the third product.

6. The method for determining the vehicle energy management model according to claim 1, characterized in that, The step of determining the vehicle energy management model based on the second sample data and the second training model includes: Based on the second sample data and the vehicle energy management simulation model, training data is determined; The training data is used to train a second training model, which includes a conservative Q-learning algorithm. The second loss function is determined based on the mean squared error of the double Q-value, the offline training loss, the Actor loss, and the Critic loss. If the second loss value of the second loss function satisfies the second preset value, the second training model corresponding to the second loss value is determined as the vehicle energy management model.

7. A method for controlling vehicle energy, characterized in that, The method includes: Obtain actual vehicle data and actual environmental data; The vehicle energy management model obtained by the method for determining the vehicle energy management model as described in any one of claims 1-6 is used to determine the operation control information corresponding to the vehicle, wherein the operation control information includes at least vehicle torque data, battery charge data, and vehicle speed.

8. A device for determining a vehicle energy management model, characterized in that, The device includes: The first acquisition module is used to acquire first sample data, wherein the first sample data includes at least first vehicle data and environmental data; The first training module is used to determine the vehicle energy management simulation model based on the first sample data and the first training model. The second acquisition module is used to acquire second sample data, wherein the second sample data includes second vehicle data; The second training module is used to determine the vehicle energy management model based on the second sample data, the vehicle energy management simulation model, and the second training model. The vehicle energy management model is used to determine the corresponding vehicle operation control information based on different vehicle data and environmental data.

9. The device for determining the vehicle energy management model according to claim 8, characterized in that, The second training module is also used for: The vehicle energy management model is subjected to knowledge distillation to obtain the vehicle energy distillation model.

10. The device for determining the vehicle energy management model according to claim 9, characterized in that, The second training module is also used for: The vehicle energy management model and the target vehicle distillation model are trained using preset training parameters; Calculate the third loss function for each training process, wherein the third loss function is determined using the KL loss function; If the third loss value of the third loss function is less than the third preset value, the target vehicle distillation model is determined as the vehicle energy distillation model.

11. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method for determining a vehicle energy management model as described in any one of claims 1-6.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, characterized in that, when the program is executed by a processor, it can implement the method for determining the vehicle energy management model as described in any one of claims 1-6.

Images