Control of a motor vehicle

Abstract

The invention relates to a method (200) comprising the steps of scanning (205) surroundings of a motor vehicle (105); determining (210) a driving situation on the basis of the scan; controlling (215) the motor vehicle (105) in a driver-independent manner on the basis of the determined driving situation; determining (220) a question relating to the driving situation; capturing (225) a response of a driver (160) of the motor vehicle (105) to the question; determining (230) whether the response is correct; and providing (235) to the driver (160) a message about the correctness of the response to the question.

Description

[0001] Control of a motor vehicle

[0002] The invention relates to the control of a motor vehicle. In particular, the invention relates to the improvement of an autonomous control system for a motor vehicle.

[0003] A motor vehicle includes a driver-independent control system designed to steer the vehicle longitudinally and laterally along a roadway. SAE Standard J3016 defines six different levels that indicate the extent to which autonomous control of the vehicle can extend. Currently, a motor vehicle equipped with level 3 control can be approved for road use. Such control does not need to be constantly monitored by a driver. For example, functions such as activating a turn signal, changing lanes, or following a lane can be performed automatically. The driver can attend to other tasks but will be prompted by the system to take over control of the vehicle within a pre-warning period if necessary.

[0004] To ensure that the driver is able to take control of the vehicle within the predetermined warning time and steer it safely, they should pay a certain amount of attention to the current driving situation. This can prevent the driver from being woken from sleep by the controls, for example, and initially becoming disoriented, thus preventing them from driving the vehicle safely.

[0005] The driver's motivation to monitor the automatic control system may decrease with increasing confidence in the technology. As a result, the driver may ultimately attribute more capabilities to the autonomous control system than it can fulfill. The present invention is based on the object of motivating a driver to improve monitoring of the autonomous control system of a motor vehicle. The invention solves this problem by means of the subject matter of the independent claims. Subordinate claims specify preferred embodiments.According to a first aspect of the present invention, a method comprises steps of scanning an environment of a motor vehicle; determining a driving situation based on the scanning; driver-independently controlling the motor vehicle based on the determined driving situation; determining a question regarding the driving situation; detecting a response of a driver of the motor vehicle to the question; determining whether the response is correct; and providing the driver with an indication of the correctness of the response to the question.

[0006] The method can be used to motivate a driver to better monitor the autonomous control of a motor vehicle. By relating the question to the driving situation, the driver can better focus their attention on the vehicle's surroundings. Questions and answers can be exchanged in a game-like manner, allowing the driver to experience enjoyment in answering the question. This can playfully motivate the driver to perform a desired behavior related to autonomous control. The driver can perceive their monitoring of the control as meaningful and strive to provide a correct answer.

[0007] The method can be implemented using components already found on board a driver-independently controllable motor vehicle, in particular a sensor for scanning the environment or a device for interacting with the driver. These devices can be used for an additional purpose by providing playful guidance to the driver.

[0008] It is particularly preferred that training data is provided based on the sampling and the response. The training data can be used to train a machine capable of learning to control a motor vehicle. The sampling can be more realistic so that the training data can have a high relevance for training the machine. The response given by the driver can be used as a classification (label) of the training data. By playfully encouraging the driver to provide a correct response, the classification ultimately given by the driver can be more reliable or precise than a human assessment by a person who has not experienced the driving situation themselves. Even using just a single motor vehicle, large amounts of training data can be generated in a relatively short time.If the method is used on board several motor vehicles, for example in vehicles of a fleet, a comprehensive collection of high-quality training data can be quickly created.

[0009] In one embodiment, only training data from a driver or a trip in which more than a predetermined proportion of the questions posed were answered correctly is used. This proportion can provide an indication of the driver's attentiveness or their ability to correctly assess a driving situation. Even training data for which the driver could not provide a correct answer or for which a correct answer cannot be clearly determined can be used to train a learning machine.

[0010] In a further embodiment, training data for which a question was answered incorrectly is discarded. This allows training data to be generated for driving situations for which a correct answer to a predetermined question could be provided either rule-based or intuitively by the driver. In particular, the driver's intuitive ability to correctly assess a driving situation can be utilized. A machine capable of learning that is trained with such training data can learn to imitate the driver's intuition.

[0011] Preferably, the driver is given an incentive to answer questions correctly. The incentive can be given in the form of a game, for example in the form of points. The driver can try to collect more points than specified or more points than another person. In the spirit of a game, a leaderboard with the highest scores can be kept, and the driver can try to generate a score that entitles them to be included in the leaderboard. The driver can also compare themselves directly with another person. For example, a leaderboard can be kept within a family, a circle of friends, or a company. Participating drivers can be informed in real time of changes to the scores, allowing a type of distributed game to be played that can motivate many drivers to behave in the described way. Accumulated points can be converted into a reward.The reward could, for example, relate to a feature of the vehicle that can be used in this way without the usual payment. For example, heated seats can be used free of charge for a certain period of time in exchange for a predetermined number of points. A premium feature of the vehicle, such as a navigation system or entertainment system, can be exchanged for a game achievement in a similar way. This allows the driver to enjoy the vehicle's benefits even more by participating in improving it in a playful way.

[0012] The question may concern an interpretation of the driving situation. For example, if sensor readings from various sensors scanning the surroundings do not allow a clear determination of which object or situation is present in the environment, the driver can be presented with a corresponding question, which the driver can then answer. In this way, knowledge that goes beyond sensory capabilities can be harnessed for the benefit of the vehicle. Interpreting the driving situation can be a crucial technique for enabling correct vehicle control.

[0013] The question may relate to an upcoming event, and a correct answer can be determined after the event has occurred. The time horizon within which the event is to occur can be chosen relatively short, for example, approximately 20 seconds. After this period, it can be determined with certainty whether the driver's answer is correct or not. This can make it easier to verify the answer.

[0014] It is generally preferred that the driver be able to answer the question very succinctly. This can minimize the effort required by the driver to answer the question. Furthermore, it can avoid wasting time entering a complex answer.

[0015] In one embodiment, the question includes an assessment of whether or not the event will occur within a predetermined time period. For example, the event may involve another road user changing lanes, another road user's cooperation in merging into a lane, or another road user performing an unannounced maneuver.

[0016] In another embodiment, a measure of how challenging it is to control the motor vehicle in the driving situation is determined. The frequency or difficulty of questions asked can be controlled depending on the determined measure. In general, the more demanding the driving situation, the more or more difficult the questions that can be asked.

[0017] A distinction can be made between questions of interest related to the game and those related to the control of the vehicle. Depending on the course of the game and the reliability with which the vehicle is controlled autonomously, questions of each type can be asked more frequently or less frequently. This can also change the overall question frequency.

[0018] Autonomous vehicle control cannot typically analyze or control arbitrarily complex driving situations. As demands on autonomous control increase, adapting the questions to the driver can ensure that the driver is more readily available to take control of the vehicle if necessary.

[0019] The measure can relate to the confidence with which a driver-independent control system on board the motor vehicle can control the motor vehicle in the driving situation. For example, the measure can express a time over which the confidence that the motor vehicle can be safely controlled by the control system is above a predetermined threshold. Conversely, the measure can also include the confidence within a predetermined, upcoming time range.

[0020] As long as the driver devotes himself to answering questions, the driving function can be performed more effectively. The time required for the driver to take over control may then be shorter than usual, and the driving function can benefit from the driver's assessment or guesswork.

[0021] The method may comprise steps of determining that the driver is concentrating on answering the question posed; determining a first time period within which the driver can take over control while answering the question; determining a second time period in which the motor vehicle can still be safely controlled; and continuing an automated driving function; if the first time period is longer than the second time period, should the first time period not be longer than the second, control of the motor vehicle can be transferred to the driver.

[0022] In a simple embodiment, whether the driver is concentrating on answering questions can be determined based on the duration that elapses between asking a question and determining an answer. A variance between several such durations can indicate a degree of concentration. The greater the concentration, the smaller the variance can be. In another embodiment, the driver's facial expression, for example, can be detected to determine the driver's concentration.

[0023] According to a further aspect of the present invention, a data storage device is provided. Training data for training a learning machine to control a motor vehicle based on scanning the surroundings of the motor vehicle are stored in the data storage device. The training data are created using a method described herein.

[0024] The training data generated according to the method can be of high quality and train the learning machine to control a motor vehicle with high quality.

[0025] A driver-independent control system for a motor vehicle is further proposed, wherein the control system is trained on the basis of training data described herein. For this purpose, it is preferred that the control system implement a machine learning technique. For example, the control system may comprise an artificial neural network trained using the described data. Furthermore, a device for a motor vehicle is proposed. The motor vehicle comprises a driver-independent control system, which comprises at least one sensor for scanning an environment of the motor vehicle, a control device configured to determine a driving situation of the motor vehicle based on the scanning, and to control the motor vehicle based on the determined driving situation.The device comprises a processing device for determining a question regarding the driving situation; and an interaction device for providing the question to a driver of the motor vehicle and for detecting an input from the driver in response to the question. The processing device is configured to determine whether the question was answered correctly; and to provide the driver with an indication of the correctness of the answer to the question.

[0026] The device can be used to partially or completely carry out a method described herein. The processing device is preferably implemented electronically and can, for example, comprise a programmable microcomputer or microcontroller. The method can be in the form of a computer program product with program code means. The computer program product can be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the device, or vice versa.

[0027] According to a further aspect, a motor vehicle with driver-independent control comprises a device as described herein. The motor vehicle can, in particular, comprise a motorcycle, a passenger car, a truck, or a bus.

[0028] Training data provided by a device on board a motor vehicle can be transmitted to an external location and further processed there. It is preferred that the external location be configured to receive and store training data from a plurality of motor vehicles. Based on the stored training data, a training data set can be created to train a learning machine described herein to control a motor vehicle. The invention will now be described in more detail with reference to the accompanying drawings, in which:

[0029] Figure 1 a system; and

[0030] Figure 2 illustrates a flow chart of a process.

[0031] Figure 1 shows a system 100 comprising a motor vehicle 105 and an optional external location 110. The motor vehicle 105 includes an autonomous controller 115 and a device 120.

[0032] The autonomous control system 115 is configured to control the motor vehicle 105 depending on a driving situation. For this purpose, the autonomous control system 115 preferably comprises at least one sensor 130 for scanning an environment 135 and a control device 140. The sensor 130 preferably comprises a contactless and more preferably an imaging sensor. The sensor 130 can comprise, for example, a camera, a depth camera, a radar sensor, a LiDAR sensor, or an ultrasonic sensor. The control of the motor vehicle 105 can take place via an interface 145, to which an actuator of the motor vehicle 105 can be connected. For longitudinal control, the actuator can comprise, for example, a drive or braking system, and for lateral control, in particular a steering device.

[0033] The control device 140 is preferably designed as a learning machine and can be trained to initially determine a driving situation based on scanning the surroundings 135. In a subsequent step, the motor vehicle 105 can be controlled based on the driving situation. The driving situation can, in particular, include a driving speed, a direction of travel, or an acceleration of the motor vehicle 105. Furthermore, the driving situation can include a geographical position or a position on a traveled road. Further possible parameters include another road user or an object in the surroundings 135. The autonomous controller 115 is preferably configured to control the motor vehicle 105 at level 3 of the SAE standard J3016 for autonomous controllers for motor vehicles.The device 120 comprises a processing device 150, which is preferably connected to the control device 140 of the autonomous control system 115. An interaction device 155 can be used for the input and / or output of information between the device 120 and a driver 160 of the motor vehicle 105. The interaction device 155 can be implemented independently of one another, optically, acoustically, or haptically in both directions. For example, a question can be output visually to the driver 160 on a graphic display, and the driver 160 can respond acoustically with a spoken answer. Other combinations are also possible.

[0034] Further preferably, the device 120 comprises a data memory 165 for storing training data 170. A preferably wireless communication device 175 can be provided for communication with the external location 110.

[0035] On board the motor vehicle 105, the device 120 can obtain from the autonomous controller 115 a scan of the surroundings 135 as well as a driving situation of the motor vehicle 105 determined on the basis of the scan. Furthermore, an indication can be obtained as to how demanding the control of the motor vehicle 105 is in the current driving situation or how much processing reserve the controller 115 still has left to handle the driving task.

[0036] The device 120 can determine a question regarding the specific driving situation, which it can provide to the driver 160. The question can relate to an interpretation of a scan of the surroundings 135 or a specific driving situation. Further preferably, the driver 160 can be asked to estimate whether or not a predetermined event will occur within a predetermined future time. The event can, in particular, relate to an action by another road user in the surroundings 135 of the motor vehicle 105. Such an action can, for example, relate to a braking maneuver, a turn, or a lane change.

[0037] A response from driver 160 to a provided question can be recorded by device 120. Once the predetermined time window has passed, it can be determined with certainty whether the predetermined event has occurred or not. Knowing this fact, it can be determined whether the response provided by driver 160 was correct or not. The question, the specific driving situation, and / or a scan of the surroundings 135 that led to the determination of the driving situation can form the basis for determining training data 170. The training data 170 can be stored in data memory 165 or transmitted to external location 110 via communication device 175.It is preferred that a plurality of training data 170 be aggregated to determine a training data set with which a device such as device 120 can be trained to control a motor vehicle 105 based on a scan of the surroundings 135 of motor vehicle 105. Training data 170 of the data set can originate from one or several motor vehicles 105. It is also possible to use training data 170 provided locally on board motor vehicle 105 to train local autonomous control 115. An exchange of training data 170 with another motor vehicle 105 is recommended but not absolutely necessary.

[0038] Figure 2 shows a flowchart of a method 200, which can preferably be carried out using a system 100. In a step 205, an environment 135 can be scanned using one or more sensors 130 on board the motor vehicle 105. This can generate data, referred to herein as scanning. Based on the scanning, a driving situation of the motor vehicle 105 can be determined in a step 210. Optionally, additional data can be incorporated into the determination, for example, local weather, a geographical position, or a driving speed.

[0039] In a step 215, a confidence with which the controller 115 can control the motor vehicle 105 in the specific driving situation can be determined. A predetermined time can be used as a basis for this, over which control should be reliably maintained. In other words, it can be determined how demanding the driving situation is for the autonomous controller 115. Furthermore, it can be determined how likely it is that the driver 160 will need to be asked to take control of the motor vehicle 105. In the present case, it is assumed that the controller 115 can continue to control the motor vehicle 105 and that the part of the method 200 described up to this point is continuously executed. In a step 220, a question for the driver 160 can be determined based on the specific driving situation. The question can be posed in the manner of a quiz.A predetermined incentive, such as a number of points or other compensation, may be offered for a correct answer to the question. The question preferably concerns an interpretation of the scan or the specific driving situation or an impending event in the environment 135 of the motor vehicle 105.

[0040] In a step 225, a response from driver 160 to the question posed can be recorded. In particular, if the question was directed at a predetermined time range in the future, it can be determined after this time range has elapsed what the correct answer to the question would have been. In other cases, the correct answer can be determined based on the scan by processing device 150.

[0041] In a step 230, it may be determined whether the response provided by driver 160 correctly answers the question. If this is not the case, information underlying the provision of the question may be discarded.

[0042] Otherwise, if the question was answered correctly, feedback can be provided to the driver 160 in a step 235 regarding their success in answering the question correctly. A reward can be awarded to the driver 160. For example, game points earned for answering the question correctly can be credited to a points account of the driver 160. Accumulated points can be exchanged at a later time, for example, for a service, an item, or currency.

[0043] In a step 240, based on the answer, the underlying question, the underlying driving situation, and / or the underlying scanning of the surroundings 135 of the motor vehicle 105, training data 170 relating to the driving situation in question can be provided. The training data 170 can be collected on board the motor vehicle 105 or by the external location 110 and processed with other training data 170. In particular, the processing can comprise a selection of training data 170 so that certain types of driving situations are represented in the data set with respective associated frequencies.

[0044] In a step 245, a controller 115 can be trained with the collected training data 170. This step typically takes place on a powerful processing device, which may be available, in particular, from the external location 110. The result of the training can, for example, be a configuration of neurons of an artificial neural network (ANN) as well as weights for connections between the neurons. This information can be relatively small and can be distributed to an existing controller 115 with little effort, for example, as part of a software update.

[0045] Reference symbol

[0046] 100 systems

[0047] 105 Motor vehicle

[0048] 110 external positions

[0049] 115 autonomous control

[0050] 120 device

[0051] 130 sensors

[0052] 135 Environment

[0053] 140 Control device

[0054] 145 Interface

[0055] 150 processing facilities

[0056] 155 Interaction facility

[0057] 160 drivers

[0058] 165 data storage

[0059] 170 training data

[0060] 175 Communication device

[0061] 200 procedures

[0062] 205 Scan the environment

[0063] 210 Determine driving situation

[0064] 215 Determine confidence, control motor vehicle

[0065] 220 Determine question

[0066] 225 Enter answer

[0067] 230 Questions answered correctly?

[0068] 235 Issue feedback, determine bonus

[0069] Create 240 training data

[0070] 245 Training the controls

Claims

Claims 1. Method (200) with the following steps: Scanning (205) an environment of a motor vehicle (105); Determining (210) a driving situation based on the scanning; driver-independent control (215) of the motor vehicle (105) based on the determined driving situation; Determining (220) a question regarding the driving situation; detecting (225) a response of a driver (160) of the motor vehicle (105) to the question; Determining (230) whether the answer is correct; and providing (235) an indication of the correctness of the answer to the question to the driver (160).

2. Method (200) according to one of the preceding claims, wherein training data (170) for training a learning machine for controlling (115) a motor vehicle (105) are provided (240) on the basis of the sampling and the response.

3. The method (200) of claim 2, wherein only training data originating from a driver (160) who has correctly answered more than a predetermined proportion of the questions asked are used.

4. The method (200) according to claim 2 or 3, wherein training data (170) for which a question was not answered correctly are discarded.

5. The method (200) according to any one of the preceding claims, wherein the driver (160) is given an incentive to answer questions correctly (220).

6. Method (200) according to one of the preceding claims, wherein the question concerns an interpretation of the driving situation.

7. The method (200) of any preceding claim, wherein the question relates to an upcoming event and a correct answer is determined after the event has occurred.

8. The method (200) of claim 7, wherein the question concerns an assessment of whether or not the event will occur within a predetermined period of time.

9. The method (200) according to any one of the preceding claims, wherein a measure is determined (215) of how demanding the control of the motor vehicle (105) is in the driving situation; and wherein a frequency or degree of difficulty of questions asked is controlled as a function of the determined measure.

10. The method (200) according to claim 9, wherein the measure is related to the confidence with which a driver-independent control (115) on board the motor vehicle (105) can control the motor vehicle (105) in the driving situation.

11. The method (200) according to any one of the preceding claims, further comprising the following steps: Determining that the driver (160) is concentrating on answering the question asked; Determining a first time period within which the driver (160) can take over control while answering the question; Determining a second time period in which the vehicle can still be safely controlled; and Continuing an automated driving function if the first time period is longer than the second time period.

12. A data storage device (165) comprising training data (170) for training a learning machine to control a motor vehicle (105) based on a scan of the surroundings of the motor vehicle (105); wherein the training data (170) are created by means of a method (200) according to one of claims 2 to 10.

13. Driver-independent (160) control (115) for a motor vehicle (105), wherein the control (115) is trained on the basis of training data (170) created by means of a method (200) according to one of claims 2 to 10.

14. A device (120) for a motor vehicle (105) with a driver-independent control (115), wherein the control (115) comprises: at least one sensor (130) for scanning an environment of the motor vehicle (105); a control device configured to determine a driving situation of the motor vehicle (105) based on the scanning; and to control the motor vehicle (105) based on the determined driving situation; wherein the device (120) comprises: a processing device (150) for determining a question regarding the driving situation; and an interaction device (155) for providing the question to a driver (160) of the motor vehicle (105) and for detecting an input from the driver (160) in response to the question; wherein the processing device (150) is configured to determine whether the question was answered correctly; and to provide an indication of the correctness of the answer to the question to the driver (160).

Images