A method to adjust a control system for controlling a motion of a vehicle

Abstract

A method to adjust a control system (102) for controlling a motion of a vehicle (100), the method comprising: 1a) controlling (S700) the motion of the vehicle (100) using the control system (102); 1b) detecting (S701), while performing step a), a passenger emotion (216, 400, 600) of at least one passenger (212) in the vehicle (100); and 1c) adjusting (S702) the control system (102) depending on the detected passenger emotion (218, 402, 602).

Description

[0001] 2023PF02898

[0002] 1

[0003] A METHOD TO ADJUST A CONTROL SYSTEM FOR CONTROLLING A MOTION OF A VEHICLE

[0004] The present invention relates to a method to adjust a control system for controlling a motion of a vehicle, a computer program product, an apparatus, and a vehicle.

[0005] Most vehicles nowadays are equipped with autonomous driving systems, such as active cruise control (ACC). These systems play a crucial role in controlling the vehicle's lateral and / or longitudinal motion. They consider safety factors, derived from an environment of the vehicle, such as the maximum allowed velocity or the distance to a preceding vehicle, to manage the vehicle's motion. Depending on these factors, the ACC system determines an appropriate acceleration level to maintain a suitable velocity.

[0006] Additionally, the ACC system determines the acceleration level by taking into account the current driving mode of the vehicle. Depending on the selected driving mode, for example a sports mode, the ACC system adapts the vehicle's motion accordingly. Driving modes are predefined and set comfort aspects, such as the maximum allowed jerk. For example, the sports mode may allow a high maximum jerk relatively to other driving modes.

[0007] US 2023 / 0373506 A1 discloses a vehicle guidance system providing a driving function for automated longitudinal control of a vehicle. The vehicle guidance system is designed to detect, on the basis of environmental data, a first signaling unit which is arranged on a roadway on which the vehicle is traveling, ahead of the vehicle in the direction of travel. The vehicle guidance system is designed to determine driver data relating to the attentiveness of the driver of the vehicle when monitoring the driving function and to operate the driving function relating to the automated longitudinal control of the vehicle according to the driver data at the first signaling unit.

[0008] DE 10 2014 202 099 A1 relates to a speed warning device for a motor vehicle, which is set up to detect the presence of a vehicle speed that is higher than the speed limit based on the 2023PF02898

[0009] 2 current vehicle speed and a current speed limit. If it is determined that there is an increased vehicle speed, a warning is issued to the driver for the presence of an increased vehicle speed. According to the invention, as a warning for the presence of an increased vehicle speed, a temporary short-term reduction in the longitudinal acceleration of the motor vehicle is triggered by a certain amount, which is noticeable to the driver.

[0010] One object of the invention is to provide an improved method to adjust a control system for controlling a motion of a vehicle.

[0011] According to a first aspect, there is provided a method to adjust a control system for controlling a motion of a vehicle, the method comprising: 1a) controlling the motion of the vehicle using the control system; 1 b) detecting, while performing step a), a passenger emotion of at least one passenger in the vehicle; and 1c) adjusting the control system depending on the detected passenger emotion.

[0012] The described method enables a user-centric adaption of the control system. Additionally to taking environmental factors, such as comfort and safety factors, into account, the emotion of the passenger is considered as a passenger factor. A control system designed this way has a higher acceptance by the user.

[0013] The control system may comprise an electronic control unit (ECU).

[0014] The motion of the vehicle may be a longitudinal and / or a transversal and / or lateral motion of the vehicle. More specifically, the motion of the vehicle includes, as an example, the jerk as a time-dependent change of acceleration, the acceleration, the velocity, the lateral acceleration, and / or the yaw rate of the vehicle.

[0015] The vehicle, or ego vehicle, may be an autonomous vehicle, having the capability of being autonomously controlled by a control system. The ECU may therefore output a signal. For example, the signal may be output to a drivetrain, a braking system and / or a steering system 2023PF02898

[0016] 3 comprised of the ego vehicle. Alternatively, the ECU may give output to at least one actuator installed in the ego vehicle, acting on the brake pedal, throttle pedal, and / or steering wheel.

[0017] The ego vehicle's motion may be controlled by considering at least one comfort criterion and / or at least one safety criterion (in addition to the passenger emotion). Comfort criteria include, for example, minimizing the effects of acceleration due to the inertia of bodies, adaption to traffic flow, and / or adjusting the velocity in curves and turns. Safety criteria are, for example, to keep a safe distance from objects, such as other vehicles, and / or to comply with traffic rules.

[0018] For example, the ego vehicle's motion may be controlled by having at least one environmental sensor monitoring the environment of the ego vehicle (in addition to the passenger emotion). The environmental sensor may detect objects in the environment. The control system may control the ego vehicle's motion depending on the detected object. Additionally, the control system may control the ego vehicle's motion depending on odometry data of the ego vehicle. The odometry data may include the ego vehicle's acceleration, engaged gear, and / or velocity.

[0019] The passenger emotion may be detected using a cabin camera facing the passenger. Alternatively or additionally, the passenger emotion may be detected by using a microphone recording noises inside the cabin, a pulse or blood pressure sensor monitoring the driver, etc..

[0020] The passenger emotion may be comfort, relaxation, calm, curiosity, distrust, anxiety, fear, boredom, satisfaction, discomfort, alertness, stress, or relief. The passenger may be a driver of the ego vehicle. For example, the passenger may be any or all persons being in the ego vehicle's cabin while the ego vehicle's motion is controlled by the control system. Therefore, a multitude of sensors may be placed in the vehicle's cabin.

[0021] The adjustment by considering the passenger criteria may be to reduce an anxiety level of the passenger. For example, a face detection algorithm may be used to detect the emotion of 2023PF02898

[0022] 4 the passenger from a signal of the camera. A speech recognition algorithm and / or an environmental sound classification algorithm may be used to analyze a signal provided by the microphone.

[0023] According to an embodiment, the control system comprises a controller implementing a control function being defined by one or more parameters, wherein the one or more parameters are adjusted in step 1c).

[0024] Adjusting the parameters of the controller is performed with low computational effort. Therefore, it is performed in real time while the vehicle's motion is controlled by the control system.

[0025] The controller may be a PID controller or LQ controller, or MPC, or SMC etc, having a proportional term, an integral term, and a derivative term. Every term may have at least one parameter, which may be adjustable in step 1c). Adjusting may include applying a function to the at least one parameter, adding a term, and / or multiplying a constant.

[0026] According to an embodiment, the maximum allowed jerk and / or the maximum allowed acceleration of the vehicle are / is adjusted in step 1 c).

[0027] Longitudinal comfort is highly correlated to jerk. Therefore, adjusting the maximum allowed jerk depending on the detected passenger emotion influences an anxiety-level of the passenger, while the control system is controlling the ego vehicle longitudinally.

[0028] The jerk is defined as the change in acceleration rate over time. The maximum allowed jerk and / or the maximum allowed acceleration may be linked to a driving situation the ego vehicle is in. In particular, the adjustment of the maximum allowed jerk and / or the maximum allowed acceleration may be linked to the driving situation.

[0029] For example, a maximum allowed jerk and / or the maximum allowed acceleration may be assigned to a driving mode. The driving mode may be set by the previously mentioned driver of 2023PF02898

[0030] 5 the ego vehicle. The driving modes comprise, for example, a sports mode, a comfort, and an eco mode. These driving modes may have a different maximum allowed jerk and / or maximum allowed acceleration. In particular, in step 1c), the driving mode may be adjusted.

[0031] According to an embodiment, the method further comprises: 1 aa) detecting a first driving situation associated with the detected passenger emotion of step 1 b); 1 bb) comparing the first driving situation to a reference driving situation; 1 cc) comparing the associated passenger emotion to a reference passenger emotion; and 1 dd) in step 1c) adjusting the control system depending on the comparison of steps 1 bb) and 1cc).

[0032] Adjusting the control system depending on the comparison of the detected driving situation and the detected passenger emotion to the reference driving situation and the reference passenger emotion makes it computationally low cost to determine an adjustment for the control system. Therefore, the method may be deployed independent of the computational capabilities of the control system.

[0033] The detected first driving situation may involve a driving situation, wherein the ego vehicle may have a distance to a vehicle driving ahead of the ego vehicle, wherein the ego vehicle decelerates heavily, accelerates heavily, and / or has a high velocity. In particular, detecting the first driving situation may include objects detected by the previously mentioned environmental sensors or the previously mentioned odometry data.

[0034] Associating the detected passenger emotion with the first driving situation may include detecting the passenger emotion with the previously mentioned cabin camera and / or microphone preferably up to, for example, fifteen seconds before and / or after the first driving situation is detected, preferably up to, for example, ten seconds before and / or after the first driving situation is detected, preferably up to, for example five seconds before and / or after the first driving situation is detected. The different time scales are given for indicative, but non- limitative purpose. In general, the emotion will not be estimated at a very high rate. However, 2023PF02898

[0035] 6 we need to be sure that the previous emotion is still valid, in particular, if the current driving situation could worsen the actual emotional state.

[0036] Comparing the first driving situation to the reference driving situation includes, for example, determining a first probability for a match between the first driving situation and the reference driving situation. In particular, the first probability of the match is calculated for the detected driving situation and a multitude of reference driving situations. The reference driving situation may include preset values for braking, accelerating, and / or a distance between the ego vehicle and the vehicle driving ahead, or a combination of these. In particular, the reference driving situation may include preset objects detected by the environment sensor and / or odometry data. Presetting of objects and / or values may happen before step 1 a).

[0037] The reference passenger emotion may be comfort, relaxation, calm, curiosity, distrust, anxiety, fear, boredom, satisfaction, discomfort, alertness, stress, or relief. Comparing the associated passenger emotion to the reference passenger emotion includes, for example, determining a second probability of the match between the associated passenger emotion and the reference passenger emotion. In particular, the second probability of the match is calculated for the associated passenger emotion and a multitude of reference passenger emotions.

[0038] For example, a value for the adjustment in step 1c) is associated with the reference driving situation and / or the reference passenger emotion. The value may be an adjusted maximum allowed jerk, an adjusted maximum allowed acceleration, and / or a value to add to, subtract from, multiply with, and / or divide by a current maximum allowed jerk and / or a current maximum allowed acceleration and / or the one or more parameters of the control function of the controller.

[0039] For example, the result of the comparison of step 1 cc) and / or the adjustment performed to the control system in step 1 dd) may be stored on a storage device. The storage device may be implemented inside the vehicle, for example on the ECU. 2023PF02898

[0040] 7

[0041] According to an embodiment the method further comprises: 2aa) detecting a second driving situation; 2bb) using the adjusted control system of step 1 dd) to control the motion of the vehicle.

[0042] Using the adapted control system to control the motion of the vehicle may lead to an emotion of the passenger that differs from the emotion of the passenger when the vehicle is controlled with the unadjusted control system. The adjusted control system acts more according to the passenger's current preferences. Between two consecutive days, the same situation could lead to two different emotional states, which thus requires two different adjustments.

[0043] The detected second driving situation may involve a driving situation, wherein the ego vehicle may have a small distance (following distance) to a vehicle driving ahead of the ego vehicle, wherein the ego vehicle decelerates heavily, accelerates heavily, and / or drives fast. In particular, detecting the second driving situation may include objects detected by the previously mentioned environmental sensor or the previously mentioned odometry data. The detected driving situation in step 2aa) may correspond to the driving situation in step 1 aa).

[0044] According to an embodiment, the method further comprises: 3aa) detecting the passenger emotion associated with the second driving situation of step 2aa); 3bb) comparing the second driving situation of step 2aa) to the reference driving situation of step 1 bb) ; 3cc) comparing the detected passenger emotion of step 3aa) to the reference passenger emotion; and 3dd) further adjusting the adjusted control system of step 1 dd) depending on the comparison of steps 3bb) and 3cc).

[0045] Detecting the passenger emotion associated with the second driving situation and comparing the detected passenger emotion to the reference passenger emotion makes it possible to derive a (further) learning for further optimizing the ACC behavior and therefore better meeting the preferences of the passenger. 2023PF02898

[0046] 8

[0047] Associating the passenger emotion with the second driving situation may include detecting the passenger emotion with the previously mentioned cabin camera and / or microphone preferably up to fifteen seconds before and / or after the second driving situation is detected, preferably up to ten seconds before and / or after the second driving situation is detected, preferably up to five seconds before and / or after the second driving situation is detected.

[0048] According to an embodiment, reinforcement learning is used in steps 1 dd) and / or 3dd).

[0049] Reinforcement learning improves decision-making by continuously learning from past experiences, allowing it to refine strategies. Therefore, applying reinforcement learning strategies on the adjustment of the control system in steps 1 dd) and 3dd) enables to customize the control system depending on the detected passenger emotion while in use.

[0050] For reinforcement learning, a value-based method, a policy-based method, an actor-critic method, and / or a model-based method may be used. For example, a neural network is used for reinforcement learning. In addition, the reinforcement learning algorithm can make use of neural networks and / or tabular function where needed.

[0051] According to an embodiment, the reference passenger emotion and / or the reference driving situation are stored on a storage device of the vehicle before step 1 a).

[0052] The reference passenger emotion and / or the reference driving situation may be stored in a table. For example, multiple reference passenger emotions are associated with one reference driving situation. In particular, multiple reference driving situations are associated with one reference passenger emotion.

[0053] An adjustment of the control system, in particular an adjustment value, may be associated with every reference passenger emotion and / or reference driving situation. In particular, the maximum allowed jerk and / or the maximum allowed acceleration may be adjusted by a value 2023PF02898

[0054] 9 associated with the reference passenger emotion and / or the reference driving situation. In particular, the driving mode may be adjusted or changed.

[0055] The storage device may be a hard disk drive, a solid-state drive, an embedded multimedia card, a universal flash storage, and / or a USB flash drive.

[0056] According to an embodiment, the reference passenger emotion and / or the reference driving situation are defined depending on empirical data.

[0057] The reference passenger emotion and / or the reference driving situation are defined depending on actual observed data. Therefore, the reliability of the reference passenger emotion and / or the reference driving situation is improved.

[0058] For example, before step 1 a), a reference vehicle equipped with the control system is motion-controlled by the control system. The reference vehicle is equipped with the previously mentioned environmental sensor and the cabin camera and / or microphone. The emotions of the driver may be detected while the reference vehicle is controlled by the control system in driving situations. The driving situations may be stored on the storage device as the reference driving situations and the detected emotions of the driver may be stored on the storage device as the reference passenger emotions. Suitable adjustment values used in step 1c) to adjust the control system may also be determined from empirical data recorded before step 1 a).

[0059] According to an embodiment, the reference passenger emotion and / or the reference driving situation, the one or more parameters, the maximum allowed jerk, and / or the maximum allowed acceleration are defined using a simulation of a virtual vehicle and a virtual passenger in a virtual driving environment before step 1 a).

[0060] Comprehensive testing through virtual simulations enables thorough evaluation of the reference passenger emotion and / or the reference driving situation, the one or more parameters, 2023PF02898

[0061] 10 the maximum allowed jerk, and / or the maximum allowed acceleration across a wide range of scenarios, including extreme and rare conditions, ensuring robust and reliable performance.

[0062] The reference passenger emotion, the reference driving situation, the maximum allowed jerk, the driving mode, and / or the maximum allowed acceleration may be defined depending on empirical data and additionally tested using the simulation of the virtual vehicle and the virtual passenger in the virtual driving environment.

[0063] The one or more parameters may be defined using, for example, a stability criterion of a control theorem and further refined using the simulation of the virtual vehicle and the virtual passenger in the virtual driving environment.

[0064] The virtual vehicle may be a virtual representation of the behavior of a real vehicle, for example, the ego vehicle or the reference vehicle. The virtual passenger may be a virtual representation of a real passenger seated in the cabin of the vehicle, while a control system controls the motion of the real vehicle. The virtual passenger may, in particular, represent the emotions of the real passenger. The virtual driving environment may be a recorded environment, values representing a real environment, and / or a look-up table. The virtual vehicle and the virtual passenger in the virtual environment may be within a simulation software executed by a computer.

[0065] According to an embodiment, the control system is an ACC system, or any automated feature, which can require different emotional adjustment with respect to how much driver is expected to interact with the vehicle.

[0066] According to a further aspect, there is provided a computer program product, comprising instructions which, when carried out by a control device of the vehicle, cause the control device to perform the method of any of the preceding claims. 2023PF02898

[0067] 11

[0068] The computer program product, such as a computer program means, may be embodied as a memory card, as a flashable memory device, as a ROM, PROM or EEPROM, as a USB stick, CD-ROM, DVD, or as a file which may be downloaded from a server in a network. For example, such a file may be provided by transferring the file comprising the computer program product from a wireless communication network.

[0069] According to a further aspect, there is provided an apparatus comprising: a controlling unit for controlling the motion of the vehicle using the control system; a detecting unit for detecting, while performing step a), a passenger emotion of at least one passenger in the vehicle; and an adjusting unit for adjusting the control system depending on the detected passenger emotion.

[0070] The respective "unit", e.g., the controlling unit, the detecting unit, and / or the adjusting unit, may be implemented using hardware and / or software. In a hardware implementation, the respective unit may, for example, be designed as a computer or a microprocessor. In a software implementation, the respective unit may be designed as a computer program product, as a function, as a routine, as an algorithm, as part of a program code, or as an executable object. Furthermore, each of the above-mentioned units may also be designed as part of a higher-level control system of the vehicle, such as the ECU and / or an engine control unit.

[0071] According to a further aspect, there is provided a vehicle comprising the apparatus of claim 13.

[0072] In particular, the vehicle may be a motor vehicle such as an automobile, a bus, a lorry, a truck, and the like.

[0073] The features, embodiments, and advantages described for the method of the first aspect also apply, mutatis mutandis, to the further aspects and vice versa. 2023PF02898

[0074] 12

[0075] Where reference is made to a "first", "second", etc. element or steps a), b), etc. herein, the skilled person will understand that this merely serves the purpose of distinguishing the different elements or steps but does not imply an order, also not to the extent that the presence of a first and third element requires the presence of a second element (unless stated explicitly otherwise).

[0076] Further possible implementations or alternative solutions of the invention also encompass combinations - that are not explicitly mentioned herein - of features described above or below with regard to the embodiments. The person skilled in the art may also add individual or isolated aspects and features to the most basic form of the invention.

[0077] Further embodiments, features, and advantages of the present invention will become apparent from the subsequent description and dependent claims, taken in conjunction with the accompanying drawings, in which:

[0078] Fig. 1 shows a top view of a vehicle;

[0079] Fig. 2 shows a schematic view of an electronic control unit;

[0080] Fig. 2a shows a first table;

[0081] Fig. 2b shows a second table;

[0082] Fig. 3 shows a schematic top view of the vehicle from Fig. 1 and a second vehicle on a lane at a first time step (left) and at a second time step (right);

[0083] Fig. 4 shows a schematic side view of a passenger in the vehicle from Fig. 1 ;

[0084] Fig. 5 shows a schematic top view of the vehicle from Fig. 1 and a third vehicle on a lane at a third time step (left) and at a fourth time step (right); 2023PF02898

[0085] 13

[0086] Fig. 6 shows a schematic side view of the passenger in the first vehicle from Fig. 1 ; and

[0087] Fig. 7 shows method steps of a method in accordance with an embodiment.

[0088] In the Figures, like reference numerals designate like or functionally equivalent elements unless otherwise indicated.

[0089] Fig. 1 shows a schematic view of an ego vehicle 100 from a bird's eye view. The ego vehicle 100 is, for example, a passenger car or lorry arranged in an environment 108. The ego vehicle 100 has a control system 102, for example, an electronic control unit (ECU), configured to control a motion of the ego vehicle 100. In addition, a plurality of environmental sensor devices 104, 106 are arranged on the ego vehicle 100. These are, for example, optical sensors 104 and ultrasonic sensors 106. The optical sensors 104 comprise, for example, visual cameras, radar, and / or lidar. The optical sensors 104 may each capture an image of a respective area from the surroundings 108 of the ego vehicle 100 and output it as a sensor signal. The ultrasonic sensors 106 are set up to detect a distance to objects located in the environment 108 and provide an output as a sensor signal. The sensor signal is received and processed by the ECU 102 to control the motion of the ego vehicle 100.

[0090] For example, the ego vehicle 100 may have an automation level according to the SAE classification system. The SAE classification system was published in 2014 by SAE International, a standardization organization for motor vehicles, as J3016, ‘Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems’. It is based on six different levels of automation and takes into account the degree of system intervention and driver attention required. The SAE automation levels range from level 0, which corresponds to a fully manual system, through driver assistance systems at levels 1 to 2, to semi-autonomous (levels 3 and 4) and fully autonomous (level 5) systems, where a driver is no longer required. An autonomous vehicle (also known as a driverless car, self-driving car, and robotic 2023PF02898

[0091] 14 car) is a vehicle that is able to sense the environment 108 and navigate without human input, and it corresponds to SAE automation level 5.

[0092] The ECU 102 is set up to implement automated driving features to provide an automation level as described above, wherein the automation level can be any level from level 1 to level 5. For example, the ECU 102 is configured to provide one or more of the following functions: an adaptive cruise control (ACC), lane keeping assistance (LKA), automatic emergency braking (AEB), automatic emergency steering (AES), Traffic Jam Assist (TJA), Highway Assist (HWA) and / or Traffic Jam Pilot (TJP).

[0093] Fig. 2 shows the ECU 102 in a more detailed view. The ECU 102 comprises a controlling unit 200. The controlling unit 200 may be implemented as a PID controller, LQ, MPC, SMC or any other suitable controller. The PID controller has a proportional term, an integral term, and a derivative term, which are not shown in Fig. 2. The proportional term, the integral term, and the derivative term have respectively one parameter. The respective parameter is set before the controller is deployed on the ECU 102 depending, for example, on a stability criterion, and / or a performance criterion of at least one control theorem and may be adjusted while the ECU 102 controls the motion of the ego vehicle 100.

[0094] The controlling unit 200 receives input from a front camera 104. In another embodiment, the controlling unit 200 receives multiple inputs from multiple optical sensors 104 and multiple ultrasonic sensors 106. The front camera 104 monitors the environment 108 of the ego vehicle 100. The front camera 104 comprises a detection unit 202. The detection unit 202 detects, for example, objects in the environment 108, the distance to objects in the environment 108, motion of objects, lanes and / or traffic signs in the environment 108.

[0095] Additionally, the controlling unit 200 may receive input from an odometer 204. The odometer 204 is arranged in the ego vehicle 100 to measure odometry data of the ego vehicle 100, for example, the current velocity, a distance the ego vehicle 100 has travelled, and / or the current acceleration of the ego vehicle 100, and provides these measurements to the controlling unit 2023PF02898

[0096] 15

[0097] 200. Depending on the received input from the front camera 104 and the odometer 204, the controlling unit 200 controls the motion of the ego vehicle 100, for example implementing a following distance with respect to a vehicle in front as a function of velocity.

[0098] The input from the front camera 104 and the input from the odometer 204 are additionally provided to a determining unit 206 comprised by the ECU 102. The determining unit 206 provides a detected driving situation 208 depending on the received input. For example, the determining unit 206 analyzes the detected objects by the detecting unit 202 of the front camera 104 in the environment 108, while using the odometer 204 readings to gauge the ego vehicle's 100 velocity, thus determining the detected driving situation 208.

[0099] The ECU 102 additionally receives input from a cabin camera 210. The cabin camera 210 is placed in a cabin of the ego vehicle 100 and monitors a passenger 212, for example, the driver, of the ego vehicle 100. While the controlling unit 200 is controlling the motion of the ego vehicle 100, the cabin camera 210 monitors the passenger 212. The cabin camera 210 comprises a detecting unit 214. The detecting unit 214 detects a passenger emotion 216 of the passenger 212, while the controlling unit 200 controls the motion of the ego vehicle 100. The detecting unit 214 may categorize the passenger emotion 216 into comfort, relaxation, calm, curiosity, distrust, anxiety, fear, boredom, satisfaction, discomfort, alertness, stress, or relief, thus providing a detected passenger emotion 218.

[0100] The detected passenger emotion 218 is provided as input, in addition to the input from the determining unit 206, to an adjusting unit 220 comprised by the ECU 102. The adjusting unit 220 comprises a software application, which receives the detected passenger emotion 218 and the detected driving situation 208 as input.

[0101] The software application has access to a first look-up table 222 and a second look-up table 230 comprised by the adjusting unit 220. Fig. 2a shows an excerpt of the first look-up table 222. The first look-up table 222 is initialized before the ego vehicle 100 is manufactured in 2023PF02898

[0102] 16 the design stage of the ego vehicle 100, wherein emotions of various humans and driving situations are classified, and actions to the controlling unit 200 are associated with the classified emotions and driving situations. Within the first look-up table 222 rows and columns are arranged. The first column on the left of the first look-up table 222 is a list of reference passenger emotions 224, representing the classified emotions. In the top row, reference driving situations 226, representing the classified driving situations, are listed. Assigned to every reference passenger emotion 224 and every reference driving situation 226 are adjustments 228, representing the actions associated with the classified emotions and driving situations. The adjustments 228 are applied to the controlling unit 200 to adjust the controlling unit's 200 parameters.

[0103] Therefore, the software application compares the detected passenger emotion 218, provided by the cabin camera 210, and the detected driving situation 208, provided by the determining unit 206, to the reference passenger emotion 224 and the reference driving situation 226, for example by respectively determining a probability of match. If, for example, the match between the detected passenger emotion 218 and the reference passenger emotion 224 exceeds a threshold, the detected passenger emotion 218 is assigned to the reference passenger emotion 224. The same applies to the detected driving situation 208 and the reference driving situation 226. For example, the detected passenger emotion 218 is assigned to the reference passenger emotion 224 'fear', and the detected driving situation 226 is assigned to the reference driving situation 226 'a)'. Therefore, 'adjustments' is applied by the software application to the controlling unit 200. The adjustment, shown in the first look-up table 222 as adjustmentl - adjustment4, may for example be a value that is added to a maximum allowed jerk, to a maximum allowed acceleration, and / or a driving mode.

[0104] Fig. 2b) shows an excerpt of the second look-up table 230, which is also initialized in the design stage of the ego vehicle 100. Therefore, various test vehicles are driven in different driving situations with measurement equipment and a passenger, wherein the passenger classifies the driving of every vehicle in different driving situations into driving modes. The measurement equipment measures, for example, the jerk and / or the acceleration of every driving 2023PF02898

[0105] 17 situation classified by the passenger. In a subsequent step, the jerk and / or the acceleration are assigned to the driving mode depending on the rating of the passenger and depending on the driving situation in the second look-up table 230. The driving situations of the second look-up table 230 correspond to the reference driving situations 226 of the first look-up table 222.

[0106] For example, the driving modes in the second look-up table 230 are sport, comfort, and eco. Depending on the reference driving situation 226, each driving mode has a different maximum allowed jerk. In another embodiment, each driving mode has a different maximum allowed acceleration depending on the reference driving situation 226.

[0107] If the adjustmentsl - adjustment4 in the first look-up table 222 represent driving modes, these driving modes do not have fixed maximum allowed jerk and / or maximum allowed acceleration. Instead, the maximum allowed jerk and / or acceleration for each driving mode depends on the reference driving situation 226, as specified in the second look-up table 230.

[0108] For example, in driving situation a), the maximum allowed jerk 'jerkl ' for the sport mode according to the second look-up table 230 might be 0,7^-. However, in driving situation b), the maximum allowed jerk 'jerk2' for the sport mode might be 0,9^.

[0109] Is an adjustment 228 determined by the software application, the adjusting unit 220 applies the adjustment 228 to the controlling unit 200, adjusting the controlling unit's 200 parameters. The software application may therefore comprise a reinforcement learning method, to adjust the controlling unit 200 according to the emotions of the passenger 212. The controlling unit 200 outputs a control signal to a drive train 232 of the ego vehicle 100, wherein the control signal of the controlling unit 200 controls the drive train 232.

[0110] In Fig. 3, a first driving situation 300 is shown at a first time steps t1 and at a second time step t2. During both time steps t1 , t2, the ego vehicle 100, controlled by the ECU 102, drives on a first lane 302 and the driving mode is set to comfort with a safety distance set by the 2023PF02898

[0111] 18 passenger 212, which is not shown in Fig. 3. At the first time step t1 , a second vehicle 304 on a second lane 306 ahead of the ego vehicle 100 moves faster, with a distance D1 between the ego vehicle 100 and the second vehicle 304. The second vehicle 304 signals a merge into the ego vehicle's lane. At the second time step t2, the second vehicle 304 merges into the first lane 302, maintaining its velocity, while the ego vehicle 100 reduces its velocity, resulting in a distance D2, which is greater than the distance D1 but less than the safety distance previously set by the passenger 212.

[0112] The front camera 104 and detection unit 202 monitor and detect these changes, providing input to the controlling unit 200. The controlling unit 200 also receives input from the odometer 204 about the ego vehicle's 100 motion at both time steps t1 , t2. The controlling unit 200 adjusts the ego vehicle's 100 velocity to avoid collision and ensure comfort. Since the controlling unit 200 receives the information about the higher velocity of the second vehicle 304, the controlling unit 200 reduces the velocity of the ego vehicle 100 at the first time step t1 and lets the second vehicle 304 increase the distance D2 over time.

[0113] Detected information and inputs from the front camera 104 and the odometer 204 are processed by the determining unit 206. The determining unit 206 determines therefore a first detected driving situation 308.

[0114] Fig. 4 shows a side view of the ego vehicle 100, wherein the passenger 212 being monitored by the cabin camera 210, which detects the passenger's emotion 400 using the detecting unit 214. The detecting unit 214 categorizes the passenger emotion 400 as 'fear' and provides a detected passenger emotion 218 to the software application. The software application in the adjusting unit 220 uses the detected passenger emotion 218 and the detected first driving situation 308 provided by the determining unit 206 to adjust the controlling unit 200’s parameters, wherein the controlling unit 200 is adjusted to set the driving mode to sport next time a similar driving situation occurs, allowing a higher maximum allowed jerk. 2023PF02898

[0115] 19

[0116] In Fig. 5, a second driving situation 500 at a third time steps t3 and at a fourth time step t4 is shown. The second driving situation 500 is similar to the first driving situation 300 shown in Fig. 3. In the second driving situation 500, a third vehicle 502 indicates the intention to merge on the first lane 302, having the distance D1 to the ego vehicle 100. The controlling unit 200 sets the driving mode to sport, thus increasing the maximum allowed jerk. At the fourth time step t4, the third vehicle 502 merges, creating a distance D3 equal to the safety distance previously set by the passenger 212. The controlling unit's 200 adjustment results in safer and more comfortable driving dynamics by considering the preferences of the passenger 212.

[0117] The determining unit 206 again detects the second driving situation 500 and determines a second detected driving situation, which is provided to the adjusting unit 220.

[0118] Fig. 6 shows the passenger 212 monitored again. The passenger emotion 600 is detected by the detecting unit 214, which provides a detected passenger emotion 602. The detected passenger emotion 218 is 'calm'. The software application compares the second detected driving situation 504, provided again by the determining unit 206, and the detected passenger emotion 218, determining no further adjustment needed. The controlling unit 200 accounts for passenger anxiety and learns to adjust accordingly for future situations.

[0119] Fig. 7 shows method steps S700 - S702. The method steps S700 - S702 may be understood as instructions of a computer program, which, when carried out by, e.g. the ECU 102, causes the ECU 102 to perform the method steps S700 - S702.

[0120] In the method step S700, the motion of the ego vehicle 100 is controlled by the control system 102.

[0121] The step S701 is performed during the step S700. In the step S701 , the passenger emotion 216 of at least one passenger 212 in the ego vehicle 100 is detected. 2023PF02898

[0122] 20

[0123] In the method step S703, the control system 102 is adjusted depending on the detected passenger emotion 218.

[0124] Although the present invention has been described with reference to examples of embodi- ments, it can be modified in many ways.

[0125] 2023PF02898

[0126] 21

[0127] List of reference signs

[0128] 100 ego vehicle

[0129] 102 control system

[0130] 104 optical sensor

[0131] 106 ultrasonic sensor

[0132] 108 environment

[0133] 200 controlling unit

[0134] 202 detection unit

[0135] 204 odometer

[0136] 206 determining unit

[0137] 208 detected driving situation

[0138] 210 cabin camera

[0139] 212 passenger

[0140] 214 detecting unit

[0141] 216 passenger emotion

[0142] 218 detected passenger emotion

[0143] 220 adjusting unit

[0144] 222 first look-up table

[0145] 224 reference emotions

[0146] 226 reference driving situations

[0147] 228 adjustments

[0148] 230 second look-up table

[0149] 232 drive train

[0150] 300 first driving situation

[0151] 302 first lane

[0152] 304 second vehicle

[0153] 306 second lane

[0154] 308 first detected driving situation

[0155] 400 passenger emotion 2023PF02898

[0156] 22

[0157] 402 detected passenger emotion

[0158] 500 second driving situation

[0159] 502 third vehicle

[0160] 504 second detected driving situation 600 passenger emotion

[0161] 602 detected passenger emotion

[0162] D1 first distance

[0163] D2 second distance D3 third distance t1 first time step t2 second time step t3 third time step t4 fourth time step

Claims

2023PF0289823PATENT CLAIMS1 . A method to adjust an active cruise control (ACC) system (102) for controlling a motion of a vehicle (100), the method comprising: 1 a) controlling (S700) the motion of the vehicle (100) using the ACC control system (102) ; l b) detecting (S701 ), while performing step 1 a), a passenger emotion (216, 400, 600) of at least one passenger (212) in the vehicle (100); and lc) adjusting (S702) the ACC control system (102) depending on the detected passenger emotion (218, 402, 602); wherein the method further comprises:1 aa) detecting a first driving situation (308) associated with the detected passenger emotion (402) of step 1 b);1 bb) comparing the first driving situation (308) to a reference driving situation (226);1cc) comparing the associated passenger emotion (402) to a reference passenger emotion (224); and1 dd) in step 1c) adjusting the ACC control system (102) depending on the comparison of steps 1 bb) and 1cc), wherein the maximum allowed jerk and / or the maximum allowed acceleration of the vehicle (100) are / is adjusted in step 1 c).2023PF02898242. The method of claim 1 , further comprising:2aa) detecting a second driving situation (504);2bb) using the adjusted control system (102) of step 1 dd) to control the motion of the vehicle (100).

3. The method of claim 2, further comprising:3aa) detecting the passenger emotion (602) associated with the second driving situation (504) of step 2aa);3bb) comparing the second driving situation (504) of step 2aa) to the reference driving situation (226) of step 1 bb);3cc) comparing the detected passenger emotion (602) of step 3aa) to the reference passenger emotion (224); and3dd) further adjusting the adjusted control system (102) of step 1 dd) depending on the comparison of steps 3bb) and 3cc).

4. The method of claim 3, wherein reinforcement learning is used in steps 1 dd) and / or 3dd).

5. The method of one of the preceding claims, wherein the reference passenger emotion (224) and / or the reference driving situation (226) are stored on a storage device (222, 230) of the vehicle (100) before step 1 a).

6. The method of one of the preceding claims, wherein the reference passenger emotion (224) and / or the reference driving situation (226) are defined depending on empirical data.

7. The method of one of the preceding claims, wherein the reference passenger emotion (224) and / or the reference driving situation (226), the one or more parameters, the maximum allowed jerk, and / or the maximum allowed acceleration are defined using a simulation of a virtual vehicle and a virtual passenger in a virtual driving environment before step 1 a).2023PF02898258. The method of one of the preceding claims, wherein the control system (102) is anACC system.

9. A computer program product, comprising instructions (S700 - S702) which, when car- ried out by a control device (102) of the vehicle (100), cause the control device (102) to perform the method of any of the preceding claims.

10. An apparatus (102) comprising: a controlling unit (200) for controlling the motion of the vehicle (100) using the control system (102); a detecting unit (214) for detecting, while performing step a), a passenger emotion(216, 400, 600) of at least one passenger (212) in the vehicle (100); and an adjusting unit (220) for adjusting the control system (102) depending on the detected passenger emotion (218, 402, 602).

11. A vehicle (100) comprising the apparatus (102) of the preceding claim.

Images