Controlling a motor vehicle

Abstract

The invention relates to a method for controlling a motor vehicle (105), the method comprising the steps of: identifying a user (150) of a motor vehicle (105); determining prevailing boundary conditions; and determining a preset for the motor vehicle (105), which preset is associated with the boundary conditions and the user (150), on the basis of a machine learning technique (115) trained for this purpose; and controlling the determined preset.

Description

[0001] Driving a motor vehicle

[0002] The present invention relates to the control of a motor vehicle. In particular, the invention relates to the control of a presetting of the motor vehicle.

[0003] A motor vehicle includes a number of functions for controlling its movement, particularly in the longitudinal or transverse direction. Other functions relate, for example, to the safety of a person inside or outside the vehicle, on-board entertainment or information, and comfort functions such as seat adjustment or air conditioning. Functions not directly related to driving can be adjusted according to a user's preferences or requirements. For example, the user can set the ventilation to a comfortable temperature or select the volume of a hands-free system.

[0004] The number of presets that can be changed by a user can be high in a modern vehicle. It's often difficult for an average user to keep track of all the presets or remember how to change them all. Experience has shown that many users frequently change certain presets. This can easily make a user interface for changing presets confusing or difficult to use.

[0005] It has been proposed to recognize patterns according to which a user sets preferences. However, existing pattern recognition systems often cannot correctly predict a user's preferences, so the user ultimately has to enforce the preferences against the pattern recognition. One object underlying the present invention is to provide an improved technology for controlling a preset on board a motor vehicle. The invention solves this problem by means of the subject matter of the independent claims. Subclaims specify preferred embodiments.

[0006] According to a first aspect of the present invention, a first method for controlling a motor vehicle comprises steps of recognizing a user of the motor vehicle; determining prevailing boundary conditions; and determining a presetting of the motor vehicle associated with the boundary conditions and the user on the basis of a machine learning technique trained for this purpose; and controlling the determined presetting.

[0007] In contrast to existing techniques, the use of machine learning can improve the control of the preset to better approximate user expectations. Machine learning can also map a large number of input parameters to a few output parameters. Machine learning can also improve interpolation between learned data points, so that even an unlearned situation can enable meaningful control.

[0008] According to a second aspect of the present invention, a second method for training a machine learning technique comprises the steps of recognizing a user of a motor vehicle; determining a user-controlled preset of the motor vehicle; determining prevailing constraints for the preset; and training the machine learning technique to associate the preset with the constraints and the user. The second method can be used to train a machine learning technique that can then be used within the first method. The two methods can be integrated, nested, or executed sequentially by further training the machine learning technique during operation with respect to the user. In this way, even an already good initial control can be further improved over time or adapted to the user.

[0009] It should be noted that the technology described herein can also support different users of the same motor vehicle. For this purpose, one or more methods can be executed with respect to the different users. A user can be assigned a dedicated machine learning technique, which is preferably further trained only with respect to that user. The technique can, for example, comprise a model or network exclusively assigned to the user.

[0010] The user may refer to a driver or a similarly designated person on board the motor vehicle, and the preset may refer to a function related to driving the motor vehicle. For different persons on board a motor vehicle, multiple procedures may also be executed independently or concurrently. A preset defined for the driver may take precedence over a preset defined for another person on board.

[0011] Preferably, the technique is pre-trained with respect to a plurality of additional users. For this purpose, the second method can be performed with different users, and the machine learning technique can determine similarities or discrepancies between the users' preferences. Training data can be collected from any motor vehicle users while the users are driving the vehicles. Based on on-board scanning of motor vehicles in a fleet, sufficient training data for training the machine learning technique can be easily and inexpensively obtained.

[0012] For this purpose, it is recommended to observe a user over a longer period of time, for example, over a year, in order to capture climatic or weather-based influences on the presets controlled by the user. Furthermore, it is recommended to train the technology based on users who are representative of those who are expected to drive a given motor vehicle. A motor vehicle manufacturer can easily determine the types of users for which it produces motor vehicles and which user groups should be considered. In practice, it may be sufficient to pre-train the technology based on a few tens to a few hundred representative users.

[0013] The pre-trained technique can be further trained on board the motor vehicle, particularly in parallel with the execution of the first method for the assigned user. Generally, there is no upper limit on the amount of training data for the machine learning technique, so the second method can also be continuously executed alongside the first. Even slow changes such as the aging of the vehicle or the user can be tracked in this way, and the preset control can be adapted accordingly.

[0014] A boundary condition can be determined in particular on the basis of a control or measured value on board the motor vehicle. For this purpose, for example, a sensor can be queried or a message from a control system can be evaluated. Example values ​​or messages relate to an outside temperature, precipitation, a time of day, a day of the week, a route traveled, a road class traveled, a driving speed, or prevailing lighting conditions. The boundary condition can also be determined on the basis of a message received from outside the motor vehicle. The message preferably relates to the operation of the motor vehicle, in particular a driving style or route guidance. For example, the message can relate to the current or expected weather or a traffic situation.

[0015] The constraints may include a user's expression. The user may express themselves in different ways, including both an explicit expression that the user consciously makes and an implicit or covert expression that can be observed in the user even if the user does not intend to communicate.

[0016] In one embodiment, the expression includes a control intervention by the user. In particular, the user can manually change or influence a preset. To do so, the user can actuate a control element, for example, a lever, a switch, or a button.

[0017] In another embodiment, the user's expression is determined based on their facial expression or gaze direction. This can detect where the user is focusing their attention or determine the user's emotional state. Both variables can also be used independently as input parameters to determine the preset.

[0018] In yet another embodiment, the expression can be determined based on the user's posture. The posture can be determined, in particular, with respect to the user's head, an arm, and / or a leg. A movement of the user or a frequency or extent of change in their posture can also be determined. More preferably, a pattern or sequence of the user's expressions is determined. Thus, even a complex or changing expression of the user can be determined and evaluated.

[0019] The motor vehicle may comprise a system configured to control a predetermined function of the motor vehicle. The presetting may then relate to a type of control. The function itself, however, may not be the subject of the presetting. For example, lateral control of the motor vehicle may be performed by the user, but a degree of steering assistance may be controlled as a presetting depending on an applicable context. The control may relate to the driving of the motor vehicle and, for example, relate to chassis tuning, exterior lighting, or a control program for a drive motor. The control may also relate to a function of the motor vehicle other than a movement function, for example, air conditioning, seat adjustment, or an entertainment or communication system for a person on board.Furthermore, a safety, welcome, comfort, or ambience function can also be controlled, such as interior or exterior lighting or a vehicle fragrance. The preset can be controlled based on the user's preference or physical condition.

[0020] The technique preferably comprises an artificial neural network (ANN), whereby different types of artificial neural networks can be used. In a particularly preferred embodiment, a first ANN is used to determine a user utterance, and a second ANN is trained to determine a preset based on utterances. Thus, multiple machine learning techniques or a combination of different techniques can be used to control the preset. A machine learning technique used can include one of a GAN (Generative Adverserial Network), a CNN (Convolutional Neural Network), or a DNN (Deep Neural Network).

[0021] According to yet another aspect of the present invention, a control device for controlling a motor vehicle comprises a device for recognizing a user of the motor vehicle; a first interface for acquiring information indicative of prevailing boundary conditions; a processing device; and a second interface for controlling a presetting of the motor vehicle. The processing device is configured to implement a machine learning technique, wherein the technique is trained to determine a presetting of the motor vehicle associated with the boundary conditions and the user.

[0022] The control device, and in particular the processing device, can be configured to partially or completely execute a method described herein. For this purpose, the processing device can be implemented electronically and, for example, comprise an integrated circuit, a programmable logic module, or a programmable microcomputer. The method can be implemented in the form of a configuration or as a computer program product with program code means for the processing device. The configuration or the computer program product can be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the device, or vice versa.

[0023] The device for recognizing a user can, for example, comprise a camera or a biometric sensor. The user can also be recognized based on a secret, the input of which is recorded. More preferably, the user can be recognized based on a token assigned to them. The token can preferably be read wirelessly. The token can be used to control a security function of the motor vehicle, for example in the form of a digital vehicle key. The digital vehicle key (DCK) can be stored on a handheld device or a mobile device assigned to the user. To use the DCK, the user can authenticate themselves to the mobile device, for example by entering a secret or by presenting a biometric feature.

[0024] According to yet another aspect of the present invention, a motor vehicle comprises a control device as described herein. The motor vehicle preferably comprises a motorcycle or a passenger car.

[0025] The invention will now be described in more detail with reference to the accompanying drawings, in which:

[0026] Figure 1 shows a control device on board a motor vehicle;

[0027] Figure 2 shows a flow chart of a process

[0028] Figure 3 illustrates exemplary machine learning techniques for recognizing user behavior.

[0029] Figure 1 shows a control device 100 on board a motor vehicle 105. The control device 100 comprises a processing device 110 configured to execute, form, or implement a learning technique or machine learning technique. An example of such a technique is represented in the form of an artificial neural network (ANN) 115.

[0030] The processing device 110 is connected to one or more sources of information indicating boundary conditions prevailing on board or in the area of ​​the motor vehicle 105. Examples include a sensor 120, an interior camera 125, an environment camera 130, and a receiving device 135. Furthermore, an interface 140 is provided, via which a presetting of a device on board the motor vehicle 105 can be controlled.

[0031] A user 150 can control a presetting of the motor vehicle or a control system on board the motor vehicle 105. The presetting preferably influences the way in which a predetermined control function is executed by the control system. Such a presetting can also be referred to as a preference.

[0032] The control device 100 is configured to recognize the user 150 and to determine the boundary conditions prevailing on board or in the surroundings of the motor vehicle 105. The ANN 115 can be trained to establish an association between prevailing boundary conditions and presets set by the user 150. The trained ANN can then also control the presets depending on the recognized boundary conditions. Preferably, the training of the ANN 115 is nevertheless continued by determining an expression or utterance of the user 150. This makes it possible to determine whether or not the user is satisfied with the presets. The determined level of satisfaction can be incorporated into the control system as a corrective. If necessary, the preset can be improved by adjusting the control system based on the detected level of satisfaction.

[0033] For example, the user 150 can be recognized biometrically using the interior camera 125. Other possibilities include recognizing a token assigned to the user 150, for example, a preferably wirelessly readable handheld device (fob, key fob). The interior camera 125 can also be used to recognize a facial expression, gaze direction, or posture of the user 150.

[0034] The environmental camera 130 can be used to determine an environmental condition of the motor vehicle 105 that is related to the operation or driving of the motor vehicle 105, for example a prevailing weather, a current traffic density, a traffic situation or prevailing lighting conditions.

[0035] By means of the receiving device 135, information can be received from an external location indicating a boundary condition that influences the operation of the motor vehicle 105. Such a boundary condition can include, for example, weather, precipitation, temperature, or traffic information.

[0036] The sensor 120 can detect any parameter related to the operation of the motor vehicle 105, for example, an on-board noise level, an opening or closing state of a door, a flap, or a convertible top, a driving state of the motor vehicle 105, a position, a road being traveled, a date, a day of the week, a time of day, a number of passengers on board, a driving speed, a planned route, an elapsed or remaining duration of a journey, etc. The sensor 120 can also be designed as a control device or control system that provides appropriate information in the form of a message.

[0037] Figure 2 shows a schematic flow diagram of a method 200 for controlling a presetting of the motor vehicle 105 or a control system depending on a prevailing boundary condition. The control is preferably related to a predetermined user 150.

[0038] In a step 205, information is collected from sources 120-135, wherein the information indicates boundary conditions prevailing on board the motor vehicle 105. For this purpose, a measurement can be performed, a message requested, or a message received. In a step 210, the received information can be processed, grouped, and / or pre-filtered. It should be noted that this step may already involve the use of a trained machine learning technique. The information can be correlated and / or fused. Implausible information can be discarded, and plausible information can be enhanced or detailed.

[0039] Subsequently, in a step 215, one or more presets can be determined using a machine learning technique, in particular using an ANN 115, with respect to the information indicating boundary conditions of the operation of the motor vehicle 105. The presets are preferably determined as a coherent set and can be controlled on the motor vehicle 105 in a step 220. For this purpose, one or more control systems of the motor vehicle 105 can be provided with presets or configured accordingly.

[0040] The presets affect the motor vehicle 105 and thus the way the motor vehicle 105 interacts with the user 150. In particular, a function of the motor vehicle 105 can be controlled depending on the preset. Multiple presets can act in concert and influence the user's 150 experience of the motor vehicle 105.

[0041] In a step 225, an expression of user 105 can be determined that indicates how satisfied they are with the controlled presets or what changes they would like to make themselves. The expression can be recognized from user 150, or an action of user 150 regarding the presets can be recorded and evaluated. For example, it can be determined whether an automatic control of a preset, such as increasing the temperature in the interior of motor vehicle 105, is confirmed or reinforced by user 150, or whether user 150 tends to weaken, withdraw, or even counteract the control. In a step 230, this evaluation can be used to check or adapt the association of ANN 115 between boundary conditions and presets. This process can be understood as training or learning.

[0042] Figure 3 shows exemplary machine learning techniques for recognizing the behavior of a user 150 with respect to constraints and presets. Various techniques are interconnected and linked in an exemplary manner to ultimately control presets on board the motor vehicle 105. The links are graphically indicated by arrows in Figure 3 and, like the presented techniques, are to be understood as exemplary. It is not absolutely necessary to provide all of the presented techniques or to link them together in the manner shown.

[0043] In a step 305, a facial expression of user 150 can be detected. In a step 310, user 150 can be identified, for example, based on biometric characteristics. Recognition can be performed based on their face, for example. Recognition can be performed, for example, using a convolutional neural network (CNN). A known algorithm for this is known as ReLu.

[0044] In a step 315, the weather in the area of ​​the motor vehicle 105 can be predicted. A deep neural network (DNN), for example, with a sigmoid algorithm, can be used for this purpose. In a step 320, a date and / or a time can be determined. In a step 325, a music preference of the user 150 can be determined. A CNN can also be used for this purpose, for example, using the Adam algorithm. In a step 330, a mood of the user 150 can be determined, for example, using a CNN based on the ReLu algorithm. In a step 335, a travel route of the motor vehicle 105 can be determined. In this case, a position on the route or driving conditions in the area of ​​the motor vehicle based on the route, such as a class of a busy road, an incline or decline, or a nearby landmark, can also be determined.

[0045] In a step 340, a navigation destination of the motor vehicle 105 can be determined based on an observation. For this purpose, for example, a fragment of a known route that has already been traveled can be determined. A frequently traveled route, such as a route to work, shopping, or school, can be identified in this way.

[0046] In a step 345, a state of the motor vehicle 105 can be determined. The state can include, for example, a driving state and / or a traffic situation.

[0047] Various presets can then be determined with respect to the specific variables, particularly in steps 330 to 345. For example, a preset 350 can be set for the sound of an on-board audio system. Likewise, a preset for a seat adjustment 355 can be controlled. A preset for a fragrance 360 ​​can be controlled in the same way as a preset for a route 365.

[0048] Further presets that can be determined or controlled include interior lighting 370, an arrangement of control elements 375, vehicle dynamics 380, a shifting behavior 385 of a manual transmission, or a characteristic 390 of a drivetrain of the motor vehicle 105. The shifting behavior can, for example, include the selection of a shift program or a shift characteristic, such as Eco, Sport, or Winter. The characteristics of the drivetrain can include similar programs.

[0049] Reference symbol

[0050] 100 control device

[0051] 105 Motor vehicle

[0052] 110 Processing facility

[0053] 115 artificial neural network (ANN)

[0054] 120 sensors

[0055] 125 interior camera

[0056] 130 Surround camera

[0057] 135 Reception device

[0058] 140 Interface

[0059] 150 users

[0060] 200 procedures

[0061] 205 Collect information

[0062] 210 Prepare information

[0063] 215 Determine boundary conditions, derive presets

[0064] Control 220 presets

[0065] 225 Control, Adapt, Train

[0066] 305 Recognizing facial expressions

[0067] 310 Facial recognition

[0068] 315 Weather forecast

[0069] 320 Date, Time

[0070] 325 Music Preference

[0071] 330 Mood determination

[0072] 335 route model

[0073] 340 Forecast navigation destination

[0074] 345 Vehicle condition

[0075] 350 Preset Sound

[0076] 355 Preset seat adjustment

[0077] 360 Preset scenting 365 Preset route

[0078] 370 Preset interior lighting

[0079] 375 Preset arrangement of controls

[0080] 380 Preset vehicle dynamics 385 Preset gearshift behavior

[0081] 390 Drivetrain presetting

Claims

Claims 1 . Method (200) for controlling a motor vehicle (105), the method (200) comprising the following steps: Detecting (210) a user (150) of the motor vehicle (105); Determining (205) prevailing boundary conditions; and determining (215) a presetting of the motor vehicle (105) associated with the boundary conditions and the user (150) on the basis of a machine learning technique (115) trained for this purpose; and Control (220) the specific preset.

2. A method (200) for training a machine learning technique (115), the method (200) comprising the following steps: Detecting (210) a user (150) of a motor vehicle (105); Determining (205) a user-controlled presetting of the motor vehicle (105); Determining (215) prevailing boundary conditions for the presetting; and Training (225) the machine learning technique (115) to associate the preset with the constraints and the user (150).

3. The method (200) of claim 1 or 2, wherein the technique (115) is pre-trained with respect to a plurality of other users (150).

4. The method (200) according to any one of the preceding claims, wherein the boundary condition is determined on the basis of a message received from outside the motor vehicle (105).

5. The method (200) of any one of the preceding claims, wherein the constraints comprise an expression of the user (150).

6. The method (200) of claim 5, wherein the expression comprises a control intervention of the user (150).

7. The method (200) according to claim 5 or 6, wherein the expression is determined based on a facial expression of the user (150).

8. The method (200) according to any one of claims 5 to 7, wherein the expression is determined based on a posture of the user (150).

9. The method (200) of any preceding claim, wherein a pattern or sequence of expressions of the user (150) is determined.

10. The method (200) according to any one of the preceding claims, wherein the motor vehicle (105) comprises a system configured to control a predetermined function of the motor vehicle (105), and the presetting relates to a manner of control.

11. The method (200) of any preceding claim, wherein the technique (115) comprises an artificial neural network.

12. The method (200) of claim 11, wherein the technique (115) comprises one of a GAN, a CNN, or a DNN.

13. Control device (100) for controlling a motor vehicle (105), the device comprising the following elements: a device (125) for recognizing a user (150) of the motor vehicle (105); a first interface (120-135) for acquiring information indicative of prevailing boundary conditions; and a processing device (110) configured to implement a machine learning technique (115), wherein the technique (115) is trained to determine a presetting of the motor vehicle (105) associated with the boundary conditions and the user (150); and a second interface (140) for controlling a presetting of the motor vehicle (105).

14. Motor vehicle (105) comprising a control device (100) according to claim 13.

Images