Energy Management System for Electric Vehicles

Abstract

A computer of an energy management system for an electric vehicle includes a processor. The computer further includes a memory. The memory includes instructions that cause the processor to be programmed to determine a value function V based on a plurality of actions U among a plurality of states S. The processor is also programmed to select an action associated with the highest reward value in the current state S. The actions U are HVAC subsystem variables. The states S are the traction power drawn from the RESS to operate the traction subsystem, the base power input drawn from the RESS to operate the HVAC subsystem, the nominal reference cabin heat input setpoint determined by the local HVAC processor, the acceleration of the electric vehicle, the current vehicle speed, the average vehicle speed, and a calibrated average vehicle speed estimate.

Description

Technical Field

[0001] The present disclosure relates to vehicle electrical systems, and more particularly to an energy management system for an electric vehicle (EV) that coordinates multiple loads to improve battery life and increase EV range while meeting drive traction demands and cabin comfort. Background Art

[0002] Current EV operation strategies do not coordinate multiple load requesters (traction, HVAC, thermal, etc.) to manage their combined impact on battery usage and long-term battery health. Since multiple loads can draw power from the battery simultaneously, the battery usage current can peak, which can increase battery aging. Additionally, current strategies can power loads without considering the efficiency cycles of the loads. In this way, loads operating at less than average efficiency can consume more electrical energy than the same loads operating at greater than average efficiency when producing a constant output in different states.

[0003] Accordingly, while existing vehicle electrical systems achieve their intended purposes, there is a need for a new and improved energy management system that addresses these problems. Summary of the Invention

[0004] According to several aspects of the present disclosure, a computer for an energy management system for an electric vehicle is provided. The supervisory computer includes a supervisory processor and a supervisory memory. The supervisory memory includes instructions that cause the supervisory processor to be programmed to determine a value function V based on multiple actions U in multiple states S. The supervisory processor is also programmed to select the action U associated with the highest reward value in a state V corresponding to the value function V. The action U is an HVAC subsystem variable. The state S includes at least one of traction power drawn from a rechargeable energy storage system (RESS) to operate a traction subsystem, base power input drawn from the RESS to operate an HVAC subsystem, a nominal reference cabin heat input setpoint determined by a local HVAC processor, acceleration of the electric vehicle, current vehicle speed, average vehicle speed, and a calibrated average vehicle speed estimate.

[0005] In one aspect, the supervisory processor is also programmed to calculate a current reward value based on a change in battery capacity loss or a change in cabin comfort.

[0006] In another aspect, the average vehicle speed is based on V2V data or V2X data.

[0007] In yet another aspect, the calibrated average vehicle speed estimate is based on past statistical data of the electric vehicle or speed limits.

[0008] In yet another aspect, the supervisory processor is also programmed to actuate the agent based on the selected action.

[0009] In yet another aspect, the agent includes an HVAC subsystem.

[0010] In yet another aspect, the supervisory processor is also programmed to receive a state S from the traction subsystem of the electric vehicle.

[0011] According to several aspects of the present disclosure, an energy management system for an electric vehicle is provided. The system includes a rechargeable energy storage system (RESS) and an HVAC subsystem. The HVAC subsystem includes a local HVAC processor and at least one HVAC memory. The HVAC memory includes instructions executable by the local HVAC processor such that the local HVAC processor is programmed to generate a nominal signal associated with a request for a base power input from the RESS. The HVAC subsystem also includes an HVAC actuator that is capable of generating a target output within a predetermined period of time in response to the HVAC actuator receiving the base power input from the RESS. The system also includes a traction subsystem having a local traction processor. The traction subsystem also includes at least one traction memory storing instructions executable by the local traction processor such that the local traction processor is programmed to generate a traction signal associated with a request for traction power drawn from the RESS. The system also includes a supervisory computer having a supervisory processor. The supervisory computer also includes a supervisory memory that includes instructions such that the supervisory processor is programmed to determine a value function V based on a plurality of actions U among a plurality of states S. The supervisory processor is also programmed to select an action associated with the highest reward value in a state V corresponding to the value function V. The value function V is an adjusted power input setpoint of the HVAC subsystem. The action U is an HVAC subsystem variable. The state S includes at least one of traction power for operating the traction subsystem, base power input for operating the HVAC subsystem, a nominal reference cabin heat input setpoint determined by the local HVAC processor, acceleration of the electric vehicle, current vehicle speed, average vehicle speed, and a calibrated average vehicle speed estimate.

[0012] In one aspect, the HVAC actuator is configured to actuate an HVAC component. The HVAC component is configured to operate at a first efficiency in response to the electric vehicle being set in a first state and the HVAC component receiving a first power input from the RESS, thereby generating a first output. The HVAC component is further configured to operate at a second efficiency in response to the electric vehicle being set in a second state and the HVAC component receiving a second power input from the RESS, thereby generating a second output. The HVAC component is further configured to generate a target output in response to modulation between the first power input and the second power input received by the HVAC component over a predetermined period of time. The second efficiency is higher than the first efficiency such that the electrical power associated with the modulation between the first power input and the second power input over the predetermined period of time is lower than the electrical power associated with a base power input over the predetermined period of time.

[0013] In another aspect, the supervisory processor selecting an action associated with the highest reward value includes the supervisory processor generating a modulated power signal in response to the supervisory processor receiving a nominal signal from the local HVAC processor. The local HVAC processor modulates between a first power input and a second power input in response to the local HVAC processor receiving the modulated signal from the supervisory processor.

[0014] In another aspect, the supervisory processor does not control the traction subsystem.

[0015] In another aspect, the processor is further programmed to calculate a current reward value R based on a change in battery capacity loss or a change in cabin comfort.

[0016] In another aspect, the average vehicle speed is based on V2V data or V2X data.

[0017] In another aspect, the calibrated average vehicle speed estimate is based on past statistical data of the electric vehicle or a speed limit.

[0018] In another aspect, the processor is further programmed to actuate an agent based on the selected action.

[0019] In another aspect, the agent includes an HVAC subsystem.

[0020] In another aspect, the supervisory processor is further programmed to receive a state S from the traction subsystem of the electric vehicle.

[0021] According to several aspects of the present disclosure, a method for operating a computer of an energy management system for an electric vehicle is provided. The computer includes a processor and a memory. The method includes using the processor to determine a value function V based on a plurality of actions U among a plurality of states S. The method further includes using the processor to select an action associated with the highest reward value corresponding to the value function V. The action U is an HVAC subsystem variable. The state S is at least one of the power drawn from a rechargeable energy storage system (RESS) to operate a traction subsystem, the power drawn from the RESS to operate the HVAC subsystem, the current vehicle speed, the acceleration of the electric vehicle, the nominal reference cabin heat input setpoint determined by a local HVAC processor, the average vehicle speed, and a calibrated average vehicle speed estimate.

[0022] In one aspect, the method further includes using the processor to calculate a current reward value based on a change in battery capacity loss or a change in cabin comfort.

[0023] In another aspect, the value function V is calculated according to the following formula based on at least one of a change in battery capacity loss and a change in cabin comfort:

[0024]

[0025] where V(S,U) represents the current multi-dimensional table V(S,U) for the control mapping, and the current multi-dimensional table V(S,U) is a value function that provides a corresponding action U in the input state S. Additionally, arg max u V * (S,U) indicates the operation of selecting an action U associated with the highest reward value R in the state S from the current multi-dimensional table V(S,U). Further, π*(S) is a policy that places the operation in a representation such that the supervisory processor selects an action U associated with the highest reward value R in the current state S in the current multi-dimensional table V(S,U) as the final action U. The value function V is further calculated according to the following formula:

[0026]

[0027] where the current multi-dimensional table V(S,U) is updated to a new multi-dimensional table Vnew(S,U). The current multi-dimensional table V(S,U) is multiplied by [1 - α] and added to a term that is the sum of the current reward value R and the projected value of the current multi-dimensional table V(S,U) based on the next action U' in the next state S'. Additionally, α is the learning rate, and γ is the discount factor.

[0028] Other applicable fields will be apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure in any way.

[0030] Figure 1 is a schematic diagram of an example of an electric vehicle having an energy management system with a computer for coordinating multiple power users to increase battery life and maintain cabin comfort.

[0031] Figure 2 is Figure 1 a block diagram of the system.

[0032] Figure 3 is Figure 1 a flowchart of an example of a method of operating the system.

[0033] Figure 4 is a chart of typical modulations of traction power and HVAC power. DETAILED DESCRIPTION

[0034] The following description is merely exemplary in nature and is not intended to limit the disclosure, application, or uses.

[0035] The present disclosure describes an example of an energy management system (hereinafter simply referred to as the system) that coordinates multiple loads to achieve optimized load power transfer, thereby improving EV range and reducing battery load. Non-limiting examples of such loads include a traction subsystem, an HVAC subsystem, a battery heating and cooling subsystem, accessory loads, and / or other suitable subsystems. The system utilizes a multi-layer control strategy, wherein a supervisory layer EM processor (hereinafter simply referred to as the supervisory processor) adjusts or increases local power requests (e.g., via an HVAC processor) based on drive variables and preview information (e.g., integrated traction and HVAC subsystem delivery). For example, the system implements real-time feedback rewards in the form of energy consumption, long-term battery health, and cabin comfort. The system supervises individual loads (e.g., HVAC control) to minimize total energy consumption by modulating the power delivered to the load requester based on its efficiency. As described in more detail below, the system uses reward measures to implement reinforcement learning to train the system during the design phase and then adapt in real time during on-site use. In one example, the system serves as a default or selectable customer ECO load management feature (e.g., ECO cooling for HVAC) that utilizes driving information (e.g., speed and acceleration) to manage other loads and improve the long-term state of health (SOH) of the battery.

[0036] Reference Figure 1, An example of a motor vehicle 100 includes an energy management system (hereinafter simply referred to as the system) 102. The motor vehicle 100 can be an electric land vehicle, such as a car or a truck. The system 102 includes a rechargeable energy storage system (RESS) 104 and a plurality of loads 106, wherein each load 106 includes a local processor 108 and a local memory 110 for storing instructions that can be executed by the local processor 108 such that the local processor 108 is programmed to generate a nominal signal associated with a power request from the RESS 104. Each load 106 also includes an actuator 112 that is capable of actuating respective components 114 to produce a target output within a predetermined time period in response to the actuator 112 receiving power from the RESS 104. The actuator 112 is implemented via a circuit, a chip, a motor, or other electronic and / or mechanical components that can actuate respective vehicle subsystems according to known appropriate control signals.

[0037] As described in the examples detailed below, each component 114 is one or more hardware components 114 adapted to perform mechanical or electromechanical functions, such as adjusting the blower temperature or fan speed of the HVAC subsystem. The efficiency of these components 114 depends on the state S of the environment. The efficiency of a component 114 operating in one state S can be higher than its average efficiency over a period of time, and the efficiency of the same component operating in another state can be lower than the average efficiency over the same period of time. As described in detail below, the system 102 delivers more power to certain components when they operate more efficiently to reduce overall power consumption.

[0038] A typical load is the HVAC subsystem 116 that includes a local HVAC processor 118. The HVAC subsystem 116 also includes at least one HVAC memory 120 that includes instructions that can be executed by the local HVAC processor 118 such that the local HVAC processor 118 is programmed to generate a nominal signal. The nominal signal is associated with a nominal HVAC power input P HVAC.nom request from the RESS 104 to produce a nominal reference cabin heat input or to maintain the cabin at a reference cabin temperature The HVAC subsystem 116 also includes an HVAC actuator 122 that is capable of actuating respective HVAC components 124 to produce a target output within a predetermined time period in response to the HVAC actuator 122 receiving the nominal HVAC power input P HVAC.nom from the RESS 104 or Non-limiting examples of the HVAC components 124 include an A / C compressor, a radiator, a radiator fan, a condenser, and a blower.

[0039] The HVAC component 124 can be moved to multiple states in which the HVAC component 124 operates with corresponding efficiencies. The HVAC component 124 is configured to operate at a first efficiency in response to the vehicle 100 being set in a first state, thereby generating a first output. The HVAC actuator 122 is further configured to operate at a second efficiency in response to the vehicle 100 being set in a second state, thereby generating a second output. The vehicle 100 in the first state is traveling at a first vehicle speed, and the vehicle 100 in the second state is traveling at a second vehicle speed higher than the first vehicle speed, such that in the same time period, the second efficiency is higher than the first efficiency, and the second output is higher than the first output. As just one example, since the radiator transfers heat from the coolant to the air flow passing through the radiator, the efficiency of the HVAC subsystem 116 can be proportional to the vehicle speed. For example, where the vehicle 100 in the first state is the vehicle 100 traveling at a first speed, and the vehicle 100 in the second state is the vehicle 100 traveling at a second speed higher than the first speed. As described in detail below, the system 102 can modulate the power supplied to the HVAC component between a first HVAC power input and a second HVAC power input over a predetermined time period, such that when a fixed nominal HVAC power input P HVAC.nom is transmitted to the HVAC component 124, the cumulative first and second outputs can provide the same target output over the same time period or In addition, the reason the cumulative modulated power P HVAC is less than the cumulative fixed nominal HVAC power P HVAC.nom over the same time period is that the first HVAC power input is lower than the nominal HVAC power input and is transmitted to the HVAC component 124 when the HVAC component 124 operates at a lower first efficiency, and the second HVAC power input is higher than the nominal HVAC power input and is transmitted to the HVAC component 124 when the HVAC component 124 operates at a higher second efficiency.

[0040] Another exemplary load can include a thermal battery cooling and heating subsystem 126 (thermal subsystem) that includes a local thermal processor 128. The thermal subsystem 126 also includes at least one thermal memory 130 that includes instructions executable by the local thermal processor 128 such that the local thermal processor 128 is programmed to generate a nominal signal. The nominal signal is associated with a request for a nominal thermal power input from the RESS 104. The thermal subsystem 126 also includes a thermal actuator 132 that is capable of actuating various thermal components to produce a target output over a predetermined period of time in response to the thermal actuator 132 receiving the nominal thermal power input from the RESS 104. Similar to the HVAC actuator 122, the thermal actuator 132 is configured to produce an associated target output in response to the thermal actuator 132 receiving a modulation between a first power input and a second power input over a predetermined period of time. In one example, the thermal actuator 132 is configured to actuate a thermal component 134, such as a heating resistance wire or other suitable heating element and / or fan. It is contemplated that the load can include any combination of an HVAC subsystem, a thermal subsystem, or other suitable subsystems.

[0041] Yet another exemplary load is a traction subsystem 136 that is monitored by the system 102 to coordinate with other loads. The traction subsystem 136 includes a local traction processor 138. The traction subsystem 136 also includes at least one traction memory 140 that stores instructions executable by the local traction processor 138 such that the local traction processor 138 is programmed to generate a traction signal. The traction signal is associated with a request for traction power P trac drawn from the RESS 104. The traction subsystem 136 also includes a traction actuator 142 that is capable of actuating various thermal components 144, such as a motor drive unit, to produce a target output over a predetermined period of time in response to the thermal actuator 132 receiving the nominal thermal power input from the RESS 104. The system 102 does not modulate or modify the power P trac delivered to the traction subsystem 136 and does not sacrifice drive requirements.

[0042] The system 102 also includes a supervisory computer 146 that includes a supervisory processor 148 and at least one supervisory memory 150. The supervisory memory 150 includes one or more forms of computer-readable media and stores instructions executable by the supervisory computer 146 for performing various operations, including the instructions disclosed herein. Via a network 152, the vehicle communication module 154 can allow the supervisory computer 146 to communicate with a server 156.

[0043] The supervisory processor 148 can be communicatively coupled to more than one local processor 108, for example, via the vehicle communication module 154. For example, the local processor is included in an electronic control unit (ECU) or the like, and the electronic control unit (ECU) is included in the vehicle 100 for monitoring and / or controlling various vehicle components 114. In this example, the supervisory processor 148 is coupled to the local traction processor 138 to monitor traction variables. Non-limiting examples of traction variables include the current vehicle speed, the current vehicle acceleration, and the traction power. In addition, the supervisory computer 146 can communicate with a navigation system using the Global Positioning System (GPS) 158 via the vehicle communication module 154. As an example, the supervisory processor 148 can request and receive position data, speed limit data, traffic data, road conditions, etc. of the vehicle 100. The position data can be in a known form, such as geographical coordinates (latitude and longitude coordinates).

[0044] The supervisory processor 148 is generally arranged to communicate on the vehicle communication module 154 via an internal wired and / or wireless network (such as a bus in the vehicle 100 or a Controller Area Network (CAN) or the like, and / or other wired and / or wireless mechanisms).

[0045] Via the vehicle communication module 154, the supervisory processor 148 can send messages to and / or receive messages from various devices in the vehicle 100 (such as vehicle sensors 160, actuators 112, vehicle components 114, a human machine interface (HMI) 162, etc.). The HMI 162 can be configured to set the system 102 to a default or selectable ECO enhanced mode, in which the system 102 is activated to coordinate multiple loads of the vehicle 100. Alternatively or additionally, in the case where the supervisory processor includes multiple devices, the vehicle communication network can be used for communication between devices represented as the supervisory computer 146 in the present disclosure. In addition, as described below, each processor and / or vehicle sensor 160 can provide data to the supervisory computer 146.

[0046] The supervisory processor 148 is programmed to monitor the traction state, including the current vehicle speed, the current vehicle acceleration power, and the traction power. In other examples, the supervisory processor 148 is coupled to vehicle sensors 160, which can include various devices for providing data that can represent or affect the traction state. Non-limiting examples of the vehicle sensors 160 can include one or more (one or more) light detection and ranging (LiDAR) sensors 164 disposed on top of the vehicle, behind the vehicle's front windshield, around the vehicle, etc., where the sensors 164 provide the relative position, size, and shape of an object and / or the conditions around the vehicle. As another non-limiting example, one or more radar sensors 166 fixed to the vehicle bumper can provide data on the distance and speed of an object (which may include a second vehicle, etc.) relative to the vehicle's position. The vehicle sensors can also include one or more (one or more) camera sensors 168 that provide images of the field of view from inside and / or outside the vehicle 100 (e.g., front view, side view, rear view, etc.).

[0047] Additionally, the supervisory processor 148 can be configured to communicate with devices external to the vehicle 100 via the vehicle-to-vehicle communication module 154 or interface 162. For example, it can communicate with a remote server 156 (typically via a network 152) through vehicle-to-vehicle (V2V) communication 170 or vehicle-to-infrastructure (V2X) wireless communication 172. The module 154 can include one or more mechanisms through which the supervisory processor 148 can communicate, including any desired combination of wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies when using multiple communication mechanisms). Typical communications provided via the module 154 include cellular, Bluetooth, IEEE 802.11, dedicated short range communication (DSRC), and / or wide area network (WAN), including the Internet, which provides data communication services.

[0048] Network 152 includes one or more mechanisms through which supervisory processor 148 can communicate with server 156. Thus, network 152 can be one or more of a variety of wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber optic) and / or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies when multiple communication mechanisms are utilized). Typical communication networks include wireless communication networks that provide data communication services (e.g., using Bluetooth, Bluetooth Low Energy (BLE), IEEE 802.11, vehicle-to-vehicle (V2V) such as dedicated short-range communication technology (DSRC), etc.), local area networks (LANs), and / or wide area networks (WANs), including the Internet. Server 156 can be a computing device, i.e., including one or more processors and one or more memories, programmed to provide operations such as those disclosed herein. Additionally, server 156 can be accessed via network 152 (e.g., the Internet or some other wide area network).

[0049] Supervisory processor 148 can receive and analyze data from sensor 160 substantially continuously, periodically, and / or when directed by server 156, etc. Additionally, object classification or recognition techniques can be used in, for example, supervisory processor 148 based on data from lidar sensors, camera sensors, etc. to identify the type of object (e.g., vehicle, person, rock, pothole, bicycle, motorcycle, etc.) and the physical characteristics of the object, which may cause the driver to adjust driving variables such as the current vehicle speed, the current vehicle acceleration, and the traction power.

[0050] Supervisory processor 148 is coupled to local load processor 108 and programmed to determine one or more load variables to adjust or increase the local load processor to achieve and reduce the total power drawn from RESS 104. As described in detail below, supervisory processor 148 is programmed to interpret the desired cabin response associated with the variables. In one example, supervisory processor 148 can be programmed to interpret the desired cabin response, such as the nominal reference cabin temperature setpoint set by local HVAC processor 118 or the nominal reference cabin heat input setpoint Q cabin,nom . Supervisory processor 148 can determine an adjusted cabin temperature setpoint for local HVAC processor 118 The setpoint can be defined by Equation 1:

[0051]

[0052] where f RL (.) represents the correction or adjustment performed by supervisory processor 148, and Represents the adjusted cabin temperature setpoint set by the supervisory processor 148.

[0053] In another example, the supervisory processor 148 can be programmed to determine an adjusted heat input setpoint for use by the local HVAC processor 118 Adjusted heat input setpoint Can be determined by Equation 2 and Equation 3:

[0054]

[0055]

[0056] Where, Represents the flow rate from the blower to the cabin, c p Represents a specific heat constant, Represents the adjusted cabin temperature setpoint, Represents the nominal reference cabin heat input setpoint first determined by the local HVAC processor 118, f RL (.) Represents the correction or adjustment made by the supervisory processor 148, and Represents the adjusted heat input setpoint of the local HVAC processor 118. It is envisioned that the supervisory processor can implement other equations to calculate the adjusted cabin temperature setpoint Adjusted heat input setpoint Or other HVAC variables.

[0057] The supervisory processor 148 is programmed to determine the impact of the current operation on the state of health (SOH) of the battery, which can be defined by Equation 4, Equation 5, and Equation 6:

[0058]

[0059] Q loss = g(T b ,ΔSoC).(Ah) n Equation 5

[0060]

[0061] Where, Q b,nom Represents the nominal battery capacity or total charge that the new RESS 104 can hold, Q b (Ah) represents the battery capacity at the current time, Q loss (%) represents the battery capacity loss in percentage, g represents the regression or function and its input in this equation (calibrated during the design or training phase), T b Represents the battery temperature, ΔSoC represents the change in the state of charge of the battery, (Ah) nAh represents the ampere - hours defined as throughput (total usage of the battery by integrating the battery current), ΔAh represents the change in throughput, n is a calibration function, and ΔQ loss represents the incremental battery capacity loss due to the current operation. It is envisioned that the supervisory processor can implement other equations to calculate battery capacity loss or other battery SOH - affecting variables.

[0062] The supervisory processor 148 is programmed to determine the impact of the current operation on cabin comfort, which can be defined by Equation 7, Equation 8, and Equation 9:

[0063] T cabin,error = ∑(T cabin - T cabin,target ) 2 Equation 7

[0064] EHT error = ∑(EHT - EHT target ) 2 Equation 8

[0065] PMV ∈ (-0.5, 0.5 comfort range) Equation 9

[0066] where, T cabin represents the current cabin temperature, T cabin,target represents the adjusted cabin temperature determined by the supervisory processor 148, and T cabin,error represents the error in cabin temperature between the current cabin temperature and the adjusted cabin temperature; EHT represents the nominal equivalent homogeneous temperature (an HVAC - related variable used to combine additional metrics such as humidity to quantify the temperature felt by passengers) and is first determined by the local processor 108; EHT target represents the adjusted EHT setpoint determined by the supervisory processor 148; EHT error represents the error in equivalent homogeneous temperature between the measured EHT and the adjusted EHT; and PMV represents the predicted mean vote or predicted percentage of dissatisfaction, which is an index of the predicted average environment evaluated by passengers. It is envisioned that the supervisory processor can implement other equations to calculate the error in cabin temperature between the adjusted cabin temperature and the measured cabin temperature, the error in EHT between the measured EHT and the nominal EHT, PMV, or other cabin comfort variables.

[0067] The supervisory processor 148 is also programmed to increase or adjust the modulated load to produce a target output with reduced battery SOH (e.g., by drawing the least amount of electrical energy from the RESS 104). The modulated load can be the HVAC subsystem 116, the thermal subsystem 126, or other suitable vehicle subsystems. The system 102 does not modulate the traction subsystem 136 to avoid sacrificing driver demand.

[0068] In one example, the supervisory processor 148 implements a physics-based model to increase the modulated load based on a preview of the traction state (e.g., obtained from V2V communication 170, V2X communication 172, and other network communications). Continuing with the previous example, the supervisory processor 148 can be programmed to establish an adjusted hot input setpoint over a fixed time window T w to enhance the local HVAC processor 118, which can be defined by Equation 10 and Equation 11: Q

[0069] Q cabin = Q cabin,nom + g * (v speed - v speedave,pre ) Equation 10

[0070] ∑ T_w Q cabin = ∑ T_w Q cabin,nominal and ∑ T_w v speed = v speedave,pre * T w Equation 11

[0071] where represents the nominal reference cabin hot input setpoint initially determined by the local HVAC processor; g represents a calibration parameter for adjusting the amount of adjustment made by the supervisory processor 148, v speed represents the current vehicle speed; v speedave,pre represents the average vehicle speed over the future time window length T w (preview window) and can be approximated using sensor 160, V2V communication 170, V2X communication 172, or any other preview device; and represents the adjusted cabin temperature setpoint determined by the supervisory processor 148 for the local HVAC processor 118.

[0072] In another example, the supervisory processor 148 implements another physics-based model to increase the modulated load based on, for example, past statistics of the host vehicle or posted speed limits. More specifically, the supervisory processor can be programmed to establish an adjusted hot input setpoint over a variable time window T w to enhance the local HVAC processor 118, which can be defined by Equation 12 and Equation 13: Q

[0073] Q cabin = Q cabin,nom + g * (v speed - v cal ) Equation 12

[0074] Instantaneous adjustment of T w to achieve ∑ T_w g*(v cal -v speed ) = 0 Equation 13

[0075] where v cal represents the calibrated average vehicle speed estimate (using past statistical data, speed limits, etc.). Based on the law of averages and the calibrated average vehicle speed v cal , the supervisory processor 148 implements Equations 12 and 13 in real time to determine the adjusted heat input Q cabin for the same time period T cabin,nom as the original cumulative heat input Q w requested by the local HVAC processor 118.

[0076] For each of the typical modulated loads, when the vehicle speed v speed is higher than the average vehicle speed v speedave,pre or v cal , the supervisory processor 148 increases the nominal setpoint, and when the vehicle speed v speed is lower than the average vehicle speed v speedave,pre or v cal , the supervisory processor 148 decreases the nominal setpoint, such that when the component 114 operates more efficiently, more power is transferred to the component 114. It is contemplated that the supervisory processor may implement other equations to determine the adjusted cabin temperature setpoint T cabin , the adjusted heat input setpoint Q cabin or other HVAC variables.

[0077] The supervisory processor 148 is further configured to implement a data-driven model for increasing the modulated load. During the training mode in the design phase, the supervisory processor 148 implements reinforcement learning (RL) to establish a lookup table, or determines the control mapping V(S,U) through iterative learning with reward feedback. Subsequently, during on-site use, the supervisory processor 148 adopts the same structure to adapt the learning control to the reward feedback in real time in the on-vehicle scenario.

[0078] Reinforcement learning (RL) is a form of goal - oriented machine learning. For example, an agent can learn from its interactions with the environment without relying on explicit supervision and / or a complete model of the environment. RL is a framework modeled in terms of the interactions between a learning agent and its environment, in terms of states S, actions U, and rewards R. At each time step, the agent receives a state S, selects an action U based on a policy, receives a scalar reward, and transitions to the next state S'. States S and S' can be based on one or more sensor inputs indicating environmental data, such as sensor 160, V2V communication 170, VZX communication 172. The goal of the agent is to maximize the expected cumulative reward R. The agent can receive a positive scalar reward for a positive action U and a negative scalar reward for a negative action U. Thus, the agent "learns" by attempting to maximize the expected cumulative reward R. Although the agent is described herein in the context of a vehicle, it should be understood that the agent can include any suitable reinforcement learning agent. In this context, the supervisory processor 148 can be referred to as the agent. Although the present example of the supervisory processor 148 implements a multidimensional table, other examples of the supervisory computer 146 can be configured to implement a reinforcement learning process based on a deep neural network.

[0079] Reference Figure 2 , the supervisory processor 148 is also programmed to determine a control mapping V(S,U) and a learning rule by using Equation 14, Equation 15, Equation 16, and Equation 17:

[0080] U = MAP(S,R) Equation 14

[0081] R = -(ΔQ loss +α|ΔT|) Equation 15

[0082] U = π * (S) = argmax u V * (S,U) Equation 16

[0083] Vnew(S,U)←[1 - α]V(S,U)+α[R(S,U)+γmax U′ V(S′,U′)] Equation 17

[0084] where S represents states, such as traction variables (e.g., v speed , v accel , P trac etc.) and nominal reference setpoints (e.g., Q cabin,nom , P HVAC etc.); U represents actions, such as adjustment setpoints for load - increase variables (e.g., or Q cabin);R represents a reward, such as real-time feedback in the form of battery aging (e.g., capacity loss) and cabin comfort; ΔQ loss represents the incremental change in the current runtime battery capacity loss; α|ΔT| represents the difference between the target and actual cabin temperatures, used to capture cabin comfort; argmax u V * (S,U) indicates the operation of using the V(S,U) table in the control and selecting the action U with the highest V(S,U) value in the table at the current measured state S; π*(S) indicates the policy of placing the operation in a representation; V(S,U) represents the current multi-dimensional table V(S,U) with state S and action U, and outputs the action U that captures the value of a specific U at state S; α is the learning rate; γ is the discount factor; α[R(S,U)+γmax U′ V(S′,U′)] represents the projected value of the current V table at the next S and U represented by S' and U'; and Vnew(S,U) represents the updated multi-dimensional table with state S and action U.

[0085] Overall, the supervisory processor 148 uses equations 14 to 17 to update the current V(S,U) table during learning based on its performance measured by the reward value R. In other words, the reward value R is the performance metric stored in the V(S,U) table. After the training phase, the final V(S,U) table is placed in the embedded control and used to determine the action U. The same learning rule can also be further used in controls with a decreasing learning rate to slowly adjust V(S,U) for on-site changes during actual vehicle use.

[0086] More specifically, in operation, the supervisory processor 148 is also programmed to receive the state S from the traction subsystem 136 of the electric vehicle 100. The state S includes the traction power P for operating the traction subsystem 136 trac , the nominal reference cabin heat input Q determined by the local HVAC processor 118 cabin,nom , the base power input P associated with the nominal reference cabin heat input Q cabin,nom , the acceleration of the electric vehicle 100, the current vehicle speed v HVAC , the average vehicle speed v speed , and the calibrated average vehicle speed estimate v speedave,pre , and at least one of the calibrated average vehicle speed estimate v cal . One or more of these states can be based on data from at least one of the sensors 160, V2V communication 170, and V2X communication 172. The calibrated average vehicle speed estimate v cal is based on at least one of the past statistical data of the electric vehicle and the speed limit.

[0087] The supervisory processor 148 is also programmed to determine a value function V(S,U) based on multiple actions U in multiple states S. The supervisory processor 148 is also programmed to calculate a current reward value R based on at least one of a change in battery capacity loss ΔQ loss and a change in cabin comfort α|ΔT|. In this example, in response to a decrease in the change in battery capacity loss ΔQ loss and / or a decrease in the difference α|ΔT| between the target and actual cabin temperatures under the current operation, the current reward value R can gradually increase.

[0088] The supervisory processor 148 is also programmed to select an action U associated with the highest reward value U, where the highest reward value U corresponds to the current state S and the value function V(S,U). In this example, the supervisory processor 148 generates a modulation power signal associated with a request to modulate between a first power input and a second power input in response to the supervisory processor 148 receiving a nominal signal from the local HVAC processor 118, and the local HVAC processor 118 modulates between the first power input and the second power input in response to the local HVAC processor 118 receiving the modulation signal from the supervisory processor 148.

[0089] The supervisory processor 148 is also programmed to actuate an agent based on the selected action. Continuing with the previous example, the agent can be the HVAC subsystem 116. The supervisory processor 148 does not control the traction subsystem such that the traction response is not altered by the system 102, and the system 102 does not consider speed or torque shaping.

[0090] Now referring Figure 3 to, an example of method 200 is provided for operating Figure 1 the supervisory computer 146 of the energy management system 102 of the electric vehicle 100 shown. Method 200 begins at block 202, where the supervisory processor 148 learns or determines a value function V(S,U) based on multiple actions U in multiple states S, for example, during a training or design phase. In this example, the action U is an HVAC subsystem variable, and the state S includes the power P drawn from the rechargeable energy storage system (RESS) to operate the traction subsystem trac , the current vehicle speed v speed , the average vehicle speed v speedave,pre , the calibrated average vehicle speed estimate v cal , the acceleration of the electric vehicle, the nominal reference cabin heat input setpoint Q determined by the local HVAC processor 118 cabin,nom and the power P drawn from the RESS to operate the HVAC subsystem 116 HVACAt least one of them. During this learning phase, the multidimensional table V(S,U) is created with initial values (e.g., empirically determined data) and tested with multiple profiles (e.g., states S for various drive cycles and HVAC operations), and at each time step, the state S evolves in due course. The supervisory processor uses equations 14 to 17 to determine the action U in the current state S.

[0091] At block 204, the supervisory processor 148 calculates at least one reward value R according to equation 15 based on at least one of the change ΔQ in battery capacity loss loss and the change α|ΔT| in cabin comfort. During the learning phase, the supervisory processor 148 measures the next state S' and calculates the reward value R.

[0092] At block 206, the supervisory processor 148 calculates at least one updated value function Vnew(S,U) based on equations 13 to 17. During the learning phase, the supervisory processor 148 updates the multidimensional table V(S,U) based on block 202 and block 204 and the example learning rules. In this way, the multidimensional table V(S,U) is gradually updated so that the corresponding reward R from each possible action U is learned, and the supervisory processor 148 can select the best action U with the highest reward R in a given state S. Once the test profiles are used up during this learning step, the updated multidimensional table Vnew(S,U) is finalized and placed in the final embedded processor.

[0093] At block 208, the supervisory processor 148 selects the action U corresponding to the value function V(S,U) with the highest reward value R. In these examples, the action U is the adjusted HVAC setpoint sent from the supervisory processor 148 to the local processor 108 of any one or more subsystems managed by the system 102, so that the system 102 can modulate the power supplied to the corresponding subsystems to reduce the total power drawn from the RESS 104 while maintaining the same output of each subsystem.

[0094] During actual vehicle use, the pre-calibrated value function V(S,U) serves as a mapping from any state S (which is an input to the supervisory processor 148, e.g., current measurements), and the action U will be the output applied to the subsystems as the final control. In addition, the same learning rules are used to further adapt the value function V(S,U) to the actual reward value R obtained. This additional learning further adjusts the value function V(S,U) acting as a real-time learning algorithm. Overall, this mechanism and the choice of the reward R (which captures battery aging and cabin comfort effects by design) make the mapping optimal with respect to the selected reward R.

[0095] Reference Figure 4, under specific usage conditions, the action U is an addition to the nominal reference cabin temperature setpoint or the nominal reference cabin heat input setpoint Q cabin, . Considering a variable U in state S (e.g., the current traction demand P trac ), the learned action U takes effect such that when the traction power P trac is higher than a predetermined threshold, the supervisory processor 148 gradually increases the cabin temperature setpoint so that the required HVAC power is instantaneously lower than the average HVAC power demand. Similarly, if the traction power P trac suddenly drops, the supervisory processor 148 gradually decreases the cabin temperature setpoint when the traction power p trac is lower than a predetermined threshold, so that the required HVAC power P HVAC is instantaneously higher than the average HVAC power demand P HVAC . Since the adjustment of the HVAC power demand P HVAC is incremental, the average temperature in the cabin remains substantially unchanged. Additionally, since the adjustment of the HVAC power demand cancels out the change in the traction power p trac , the battery current from the RESS 104 decays or flattens out.

[0096] Generally speaking, the described computing system and / or device can employ any of a variety of computer operating systems, including but not limited to Ford versions and / or variants of application programs, AppLink / Smart Device Link middleware, Microsoft operating systems, Microsoft operating systems, Unix operating systems (e.g., the operating system distributed by Oracle Corporation of Redwood Shores, California), the AIX UNIX operating system distributed by International Business Machines of Armonk, New York, Linux operating systems, the Mac OSX and iOS operating systems distributed by Apple Inc. of Cupertino, California, the BlackBerry OS distributed by BlackBerry, Ltd. of Waterloo, Canada, and the Android operating system and Open Handset Alliance developed by Google, Inc., or the CAR Platform for Infotainment. Examples of computing devices include, but are not limited to, in-vehicle computers, computer workstations, servers, desktops, laptops, notebook computers, or handheld computers, or some other computing systems and / or devices.

[0097] Computers and computing devices typically include computer-executable instructions that can be executed by one or more computing devices such as those listed above. The computer-executable instructions can be compiled or interpreted from computer programs created using various programming languages and / or technologies (individually or in combination, including but not limited to JAVA TM , C, C++, MATLAB, SIMULINK, STATEFLOW, VISUAL BASIC, JAVA SCRIPT, PERL, HTML, TENSORFLOW, PYTORCH, KERAS, etc.). Some of these applications can be compiled and executed on virtual machines such as JAVA VIRTUAL MACHINE, DALVIK virtual machine, etc. Generally, a processor (e.g., a microprocessor) receives instructions, for example, from a memory, a computer-readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored and transmitted using various computer-readable media. Files in a computing device are typically a collection of data stored on a computer-readable medium such as a storage medium, random access memory, etc.

[0098] The memory can include a computer-readable medium (also referred to as a processor-readable medium) that includes any non-transitory (e.g., tangible) medium that participates in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of the computer). Such a medium can take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media can include, for example, optical or magnetic disks and other persistent memories. Volatile media can include, for example, dynamic random access memory (DRAM) that typically constitutes main memory. Such instructions can be transmitted through one or more transmission media, including coaxial cables, copper wires, and optical fibers, including the wires of a system bus having a processor coupled to an ECU. Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic medium, CDROM, DVD, any other optical medium, punched cards, paper tapes, any other physical medium with hole patterns, RAM, PROM, EPROM, flash EEPROM, any other memory chip or memory cartridge, or any other medium from which a computer can read.

[0099] The databases, data repositories, or other data stores described herein may include various mechanisms for storing, accessing, and retrieving various data, including hierarchical databases, sets of files in a file system, application databases in a proprietary format, relational database management systems (RDBMSs), etc. Each such data store is typically included within a computing device having a computer operating system such as one of those described above, and is accessed via a network in any one or more of a variety of ways. A file system can be accessed from a computer operating system and can include files stored in various formats. In addition to languages for creating, storing, editing, and executing stored procedures, such as the PL / SQL language mentioned above, an RDBMS typically also employs the Structured Query Language (SQL).

[0100] In some examples, system components can be implemented as computer-readable instructions (e.g., software) of one or more computing devices (e.g., servers, personal computers, etc.) stored on a computer-readable medium associated therewith (e.g., a disk, a memory, etc.). A computer program product can include instructions stored on a computer-readable medium for performing the functions described herein.

[0101] Regarding the media, processes, systems, methods, heuristics, etc. described herein, it should be understood that while the steps of such processes, etc. have been described as occurring in accordance with a certain ordered sequence, such processes can be practiced with the described steps in an order different from that described herein. It should also be understood that certain steps can occur simultaneously, other steps can be added, or certain steps described herein can be omitted. In other words, the description of the processes herein is provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claims.

[0102] Accordingly, it should be understood that the foregoing description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided will be apparent to those skilled in the art upon reading the foregoing description. The scope of the present invention should not be determined with reference to the foregoing description, but rather should be determined with reference to the appended claims and the full scope of equivalents to which those claims are entitled. It is expected that future developments will occur in the technologies discussed herein, and the disclosed systems and methods will be incorporated into such future embodiments. In summary, it should be understood that the present invention is capable of modification and variation, and its scope is limited only by the claims.

[0103] All terms used in the claims are intended to be given their simple and ordinary meaning as understood by those skilled in the art, unless an explicit contrary indication is made herein. Specifically, the use of expressions such as "the," "said," etc. should be construed as reciting one or more of the indicated elements, unless an explicit contrary limitation is recited in the claims.

[0104] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. Such variations should not be regarded as a departure from the spirit and scope of the present disclosure.

Claims

1. An energy management system for an electric vehicle, the system comprising: RESS; An HVAC subsystem, comprising: A local HVAC processor; At least one HVAC memory including instructions executable by the local HVAC processor such that the local HVAC processor is programmed to generate a nominal signal associated with a request for a base power input from the RESS; An HVAC actuator capable of generating a target output within a predetermined time period in response to the HVAC actuator receiving a base power input from the RESS; A traction subsystem, comprising: A local traction processor; At least one traction memory storing instructions executable by the local traction processor such that the local traction processor is programmed to generate a traction signal associated with a request for traction power drawn from the RESS; and A supervisory computer, comprising: A supervisory processor; and A supervisory memory including instructions such that the supervisory processor is programmed to: Determine a value function V based on a plurality of actions U among a plurality of states S; and Select an action associated with the highest reward value corresponding to the value function V; Wherein at least one of the actions U includes an HVAC subsystem variable; and Wherein at least one of the states S includes at least one of traction power for operating the traction subsystem, a nominal reference cabin heat input setpoint determined by the local HVAC processor, a base power input associated with the nominal reference cabin heat input setpoint, an acceleration of the electric vehicle, a current vehicle speed, an average vehicle speed, and a calibrated average vehicle speed estimate.

2. The energy management system according to claim 1, wherein, The HVAC actuator is configured to actuate an HVAC component, the HVAC component being configured to: Operate at a first efficiency in response to the electric vehicle being set in a first state and the HVAC component receiving a first power input from the RESS, thereby generating a first output; Operate at a second efficiency in response to the electric vehicle being set in a second state and the HVAC component receiving a second power input from the RESS, thereby generating a second output; And Generate the target output in response to modulation between the first power input and the second power input received by the HVAC actuator within the predetermined time period, and the second efficiency is higher than the first efficiency such that the electric power associated with the modulation between the first power input and the second power input within the predetermined time period is lower than the electric power associated with the base power input within the predetermined time period.

3. The energy management system according to claim 2, wherein, The supervisory processor selecting an action associated with the highest reward value includes: the supervisory processor generating a modulation power signal in response to the supervisory processor receiving the nominal signal from the local HVAC processor, and the local HVAC processor modulating between the first power input and the second power input in response to the local HVAC processor receiving the modulation signal from the supervisory processor.

4. The energy management system according to claim 1, wherein, The supervisory processor is further programmed to: Calculate a current reward value based on at least one of a change in battery capacity loss and a change in cabin comfort.

5. The energy management system according to claim 4, wherein, The average vehicle speed is based on at least one of V2V data and V2X data.

6. The energy management system according to claim 5, wherein, The calibrated average vehicle speed estimate is based on at least one of past statistical data of the electric vehicle and speed limits.

7. The energy management system according to claim 6, wherein, The supervisory processor is also programmed to: Actuate an agent based on a selected action.

8. The energy management system according to claim 7, wherein, The agent includes the HVAC subsystem.

9. The energy management system according to claim 8, wherein, The supervisory processor is also programmed to receive one of the states S from the traction subsystem of the electric vehicle.

Images