Method for operating an electrical system of a motor vehicle having at least one electrical consumer

Abstract

The invention relates to a method for operating an electrical system (1) of a motor vehicle (20), in particular an electric vehicle (21) with a battery-powered drive (22), said electrical system (1) having at least one electrical consumer (2a-2d). According to the method, a software agent (SA) is trained by means of reinforcement learning to reduce the electrical energy consumption of the electrical system (1).

Description

[0001] The invention relates to a method for operating an electrical system of a motor vehicle having at least one electrical consumer. The invention further relates to a deep learning system configured / programmed to carry out the method according to one of the preceding claims. The invention further relates to a motor vehicle having such a deep learning system or having a control / regulation device configured / programmed to carry out this method.

[0002] Electrical energy consumption in a motor vehicle's electrical system is increasing significantly due to additional comfort, infotainment, ADAS, and software functions. As a result, the energy efficiency of the electrical system is becoming increasingly important in order to achieve maximum range while keeping battery costs as low as possible. However, optimizing the electrical system with regard to its energy consumption is proving to be very complex due to the interplay of hardware and software operating strategies. Effective measures to increase energy efficiency therefore require a thorough understanding of the system.

[0003] It is therefore an object of the present invention to provide an improved method for operating a motor vehicle, which allows operation of the vehicle electrical system with improved energy efficiency.

[0004] This object is achieved by the subject matter of the independent patent claims. Preferred embodiments are the subject matter of the dependent patent claims.

[0005] The basic idea of ​​the invention is therefore to train a software agent for controlling the on-board electrical system, including the electrical consumers connected to the on-board electrical system, using reinforcement learning to reduce and preferably even minimize the electrical energy consumption of the on-board electrical system. To do this, the software agent independently learns a strategy to maximize received rewards. In the course of interacting with the on-board electrical system, in particular by controlling the electrical consumers and varying their electrical energy consumption, the software agent receives rewards that can be positive or negative. In this way, the software agent can learn to control the on-board electrical system in the most energy-efficient manner possible and, after such learning, can be used in real on-board electrical system or vehicle software to control the on-board electrical system or its electrical consumers.In this way, the actual energy consumption of the vehicle electrical system can be reduced significantly.

[0006] In accordance with the above inventive concept, the method according to the invention serves to operate an on-board electrical system of a motor vehicle having at least one electrical consumer. The motor vehicle can in particular be an electric vehicle with a battery-electric drive. In the method according to the invention, a software agent is trained by means of reinforcement learning to reduce, in particular to minimize, the electrical energy consumption of the on-board electrical system. Various algorithms can be used for reinforcement learning, such as PPO, DDPG, SAC, Q-Learning. Setting certain operating states in the electrical consumers can be accompanied by a reward assigned to the software agent in the course of reinforcement learning, which is positive if the electrical energy consumption of the on-board electrical system decreases as a result of the switching to the operating states.Conversely, switching to certain operating states of the electrical consumers may also be accompanied by a negative reward assigned to the software agent if switching to these operating states increases the electrical energy consumption of the vehicle electrical system.

[0007] In a preferred embodiment, the software agent uses reinforcement learning to learn energy-efficient operating strategies for the vehicle electrical system or its loads. For this purpose, individual, several, or all loads of the vehicle electrical system can be switched to different electrical operating states that differ from one another in terms of their electrical consumption.

[0008] According to an advantageous development, the vehicle electrical system is simulated using a vehicle electrical system model to train the software agent. For this model, at least one electrical consumer of the vehicle electrical system is represented with at least two different operating states that differ from one another in terms of their electrical energy consumption. Preferably, several, and particularly preferably even all, electrical consumers are represented in this way. Using such a vehicle electrical system simulation, it is possible to optimally train the software agent before it is actually used in a vehicle electrical system or vehicle software to control the vehicle electrical system or its electrical consumers.

[0009] In a preferred embodiment, actions performed by the software agent during reinforcement learning comprise varying at least two operating states of at least one electrical consumer. Particularly preferably, several or even all electrical consumers are varied in this manner. In this embodiment, a reward function defined for reinforcement learning decreases with increasing electrical energy consumption by the vehicle electrical system. The current energy consumption depends on the operating states in which the individual electrical consumers are currently located. Using these measures, the software agent can learn particularly efficiently how to keep electrical energy consumption in the vehicle electrical system low.

[0010] Particularly preferred is training using a trial-and-error method. This method eliminates the need for complex algorithms for the software agent to learn. Rather, the software agent can try out a variety of operating strategies and select those with high or even maximum reward.

[0011] It is particularly useful to generate and improve the on-board network model using machine learning. A variety of suitable algorithms are available for this purpose. In particular, machine learning can be performed using a neural network and / or a deep learning system.

[0012] According to an advantageous development of the method according to the invention, the simulation of the vehicle electrical system model is based on real usage scenarios. This allows learning to be as efficient as possible, since less real or non-real usage scenarios can be ignored during learning.

[0013] According to another advantageous development, the software agent is tested under real-life conditions after training in the vehicle. This ensures that the software agent functions properly not only under simulation conditions, but also under real-life conditions before real-life operation in a vehicle.

[0014] The invention further relates to a computer program product designed to execute the method, in particular by means of the deep learning system. The computer program product according to the invention contains instructions which, when the computer program product is executed by a computer system and / or by the deep learning system, cause the computer system and / or the deep learning system to execute the method. The computer program product is preferably stored / stored on a memory comprising at least one non-volatile memory.

[0015] The invention further relates to a deep learning system comprising at least one neural network, which in turn is configured / programmed to implement the above-described method according to the invention. The above-explained advantages of the method according to the invention are therefore transferred to the deep learning system according to the invention.

[0016] The invention further relates to a motor vehicle, in particular an electric vehicle with a battery-electric drive. The motor vehicle according to the invention comprises an electrical system to which electrical consumers are connected. The electrical consumers can each be switched between at least two operating states, each with a different electrical energy consumption. Furthermore, the motor vehicle according to the invention comprises a control / regulation device configured or programmed to carry out the above-presented method according to the invention and, alternatively or additionally, a deep learning system according to the invention as presented above, so that the advantages of the above-explained method according to the invention are also transferred to the motor vehicle according to the invention.

[0017] Finally, the invention relates to a data carrier containing instructions that, when executed by a computer system and / or by the deep learning system according to the invention, cause the system to execute the method according to the invention presented above. The advantages of the method according to the invention are therefore also transferred to the data carrier according to the invention.

[0018] Further important features and advantages of the invention emerge from the subclaims, from the drawings and from the associated description of the figures with reference to the drawings.

[0019] It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.

[0020] Preferred embodiments of the invention are illustrated in the drawings and are explained in more detail in the following description, wherein the same reference numerals refer to the same or similar or functionally identical components.

[0021] Showing schematically: Fig. 1 An example of a motor vehicle according to the invention in a schematic representation, Fig. 2 a flowchart illustrating the method according to the invention.

[0022] The Fig. 1 shows a schematic representation of an example of a motor vehicle 20 according to the invention. The motor vehicle 20 can also be an electric vehicle 21 with a battery-electric drive 22. The motor vehicle 20 comprises an electrical system 1, to which Fig. 1, electrical consumers 2a to 2d, which are only schematically indicated, are connected. In the example scenario, four electrical consumers 2a to 2b are shown as examples. Each of the consumers 2a to 2d can be switched between several operating states Z, which differ from one another in terms of their electrical energy consumption. The on-board electrical system 1 can, for example, be a low-voltage on-board electrical system 1a with an electrical operating voltage of 12 volts. An electrical energy source 3, typically in the form of an electrical battery 4, can also be connected to the on-board electrical system 1 or low-voltage on-board electrical system 1a, which provides the corresponding operating voltage and supplies the on-board electrical system 1, including the electrical consumers 2a to 2b, with electrical energy.

[0023] Furthermore, the motor vehicle 20 comprises a control / regulation device 5 which is configured or programmed to carry out the above-described method according to the invention and which is designed to control the electrical consumers 2a to 2d and thus to switch between their respective operating states Z. In addition, the control / regulation device 5 is configured and programmed to carry out the method according to the invention. To carry out the method according to the invention, a deep learning system (in Fig. 1 not shown). Such a deep learning system can also be part of the control / regulation device 5.

[0024] The following flow chart shows the Fig. 2 explains the method according to the invention by way of example.

[0025] According to the method, a software agent SA is trained by means of reinforcement learning to calculate an electrical energy consumption of the on-board network 1 (cf. Fig. 1) at least reduce, and preferably even minimize, the number of errors. Various algorithms can be used for reinforcement learning, such as "PPO," "DDPG," "SAC," and "Q-Learning."

[0026] With the help of reinforcement learning, the software agent SA learns energy-efficient operating strategies for the on-board network 1 and its consumers 2a-2d. To this end, the software agent SA can switch the various electrical consumers 2a to 2b of the on-board network 1 into different electrical operating states Z using appropriate control actions A, which differ from one another in terms of their electrical power or energy consumption. Certain operating states Z of the electrical consumers 2a-2d are accompanied by a respective positive reward B+ if the electrical energy consumption of the on-board network 1 decreases as a result of switching to certain operating states Z. Conversely, certain operating states of the electrical consumers 2a-2d can also be accompanied by a negative reward B- if the electrical energy consumption of the on-board network 1 increases as a result of switching to certain operating states Z.

[0027] As illustrated in schematic diagram 2, to train the software agent SA, the vehicle electrical system 1 is simulated using a vehicle electrical system model 10. For this vehicle electrical system model 10, the electrical consumers 2a-2d of the vehicle electrical system 1 are simulated with various operating states Z, which differ from one another in terms of their electrical energy consumption. The simulation of the vehicle electrical system model 10 is preferably based on real-world usage scenarios.

[0028] The on-board network model 10 can be conveniently generated using machine learning. Machine learning can be performed using a neural network and / or a deep learning system. This allows for particularly efficient learning, as few real or unreal usage scenarios can be ignored during the learning process.

[0029] During reinforcement learning, actions A performed by the software agent SA result in one of the operating states Z of the electrical loads 2a-2d. Particularly preferably, all electrical loads 2a-2d are varied in this way and switched between all possible operating states Z.

[0030] In the example scenario, reinforcement learning, and thus the training of the software agent SA, is based on a trial-and-error method. The software agent SA can test many different operating strategies and select those with high or even maximum reward. This eliminates the need for complex algorithms for training the software agent SA, allowing the software agent to learn particularly efficiently how to keep electrical energy consumption in the vehicle's electrical system 1 low.

[0031] The software agent SA can, after completion of the training in the motor vehicle 20 according to Fig.1 be tested under real conditions.

Claims

[1] Method for operating an electrical on-board network (1) of a motor vehicle (20), in particular an electric vehicle (21) with a battery-electric drive (22), having at least one electrical consumer (2a-2d), according to which method a software agent (SA) is trained by means of reinforcement learning to reduce, in particular to minimize, an electrical energy consumption of the on-board network (1). [2] Method according to claim 1, characterized by that energy-efficient operating strategies for the on-board network (1) or for its consumers (2a-2d) are learned by the software agent (SA) through reinforcement learning. [3] Method according to claim 1 or 2, characterized bythat for training purposes the vehicle electrical system (1) is simulated by a vehicle electrical system model (10) in which at least one electrical consumer (2a-2d) of the vehicle electrical system (1), preferably all consumers (2a-2d) of the vehicle electrical system (1), are represented, wherein each of the consumers (2a-2d) has at least two different operating states (Z) with different electrical energy consumption. [4] Method according to one of claims 1 to 3, characterized by , that - the actions performed by the software agent (SA) in the course of reinforcement learning comprise varying the operating states (Z) of at least one electrical consumer (2a-2d), preferably all electrical consumers (2a-2d), - a reward function (B+, B-) defined for reinforcement learning decreases with increasing electrical consumption of electrical energy by the on-board network (1), which in turn is determined by the current operating states of the consumers (2a-2d). [5] Method according to one of the preceding claims, characterized by that the training of the software agent (SA) is done using a trial-and-error method. [6] Method according to one of the preceding claims, characterized by that the on-board network model (10) is trained by means of machine learning. [7] Method according to one of the preceding claims, characterized by that the simulation of the on-board network model (10) is based on real usage scenarios. [8] Method according to one of the preceding claims, characterized by that the software agent (SA) is tested under real conditions in the motor vehicle (20) after completion of the training. [9] Deep learning system (24), comprising at least one neural network which is set up / programmed to carry out the method according to one of the preceding claims. [10] Computer program product containing instructions which, when the computer program product is executed by a computer system and / or by the deep learning system according to claim 9, cause the computer system and / or the deep learning system according to claim 9 to carry out the method according to one of claims 1 to 8. [11] Motor vehicle (20), in particular electric vehicle (21) with battery-electric drive (22), - with an electrical on-board network (1) to which electrical consumers (2a-2d) are connected, wherein the electrical consumers (2a-2d) are each switchable between at least two operating states (Z) with different electrical energy consumption, - with a control / regulation device (23) configured / programmed to carry out the method according to one of claims 1 to 8 and / or with a deep learning system (24) according to claim 9. [12] A data carrier containing instructions which, when executed by a computer system and / or by the deep learning system according to claim 9, cause the system to carry out the method according to any one of claims 1 to 8.

Images