Methods and apparatus for controlling autonomous driving functions
By detecting the deviation between the user's line of sight and the target's line of sight, the autonomous vehicle is controlled to slowly execute or suppress lane changes, thus solving the problem of motion sickness in autonomous driving and improving the driving experience.
Patent Information
- Application Number
- CN202610638929.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-05-11
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, autonomous vehicles are prone to causing motion sickness when changing lanes, especially at high lateral speeds, leading to nausea among vehicle users. Existing methods have failed to effectively address this issue.
By detecting the deviation between the vehicle user's line of sight and the target line of sight facing the traffic, the autonomous driving function is controlled to slowly execute or suppress lane changes when predetermined trigger conditions are met, ensuring that lane changes are only performed when the user's line of sight is consistent with the target line of sight, thus avoiding the discrepancy between vestibular perception and visual perception.
It effectively avoids motion sickness and ensures that users will not experience discomfort due to lane changes during autonomous driving, thus improving the safety and comfort of autonomous driving.
Smart Images

Figure CN123104683A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and apparatus for controlling an autonomous driving function, the autonomous driving function including triggering a lane change of the vehicle. Background Technology
[0002] In the field of autonomous driving, SAE standards distinguish different levels of automation (L: Level) from Level 0 (L0) to Level 5 (L5). Level 0 has no automation, while Level 5 describes fully autonomous driving without human intervention. In the intermediate levels, Levels 1 to 4, the driving task gradually shifts from the driver to the vehicle.
[0003] To date, solutions to motion-induced nausea within the scope of highly automated driving are unknown, a problem known as motion sickness, travel sickness, or kinesia. Motion sickness is expected to occur particularly during lane changes performed at higher lateral speeds, which are typically, but not exclusively, at high speeds.
[0004] In current development, lane-changing strategies that should be applied in higher automation levels (SAE Levels 3 through 5) are derived from systems belonging to lower automation levels (SAE Levels 0 through 2). This results in lane changes being performed as quickly and efficiently as possible, in order to accommodate normal human driving behavior. If a vehicle user is working on a laptop while driving in such a highly automated system, for example, their gaze might be directed backward towards their laptop rather than forward towards the traffic ahead. This could lead to lateral acceleration during rapid lane changes causing nausea.
[0005] Regarding prior art, reference DE 10 2023 106572 A1 describes a method for releasing the functional level of a driver assistance system. In this method, environmental data and driver eye movement data are received. The driver's attention level is determined based on the eye movement data. A release signal is output based on this attention level, which releases the corresponding functional level of the driver assistance system for operation.
[0006] Document DE 10 2021 126 254 A1 discloses a system for monitoring the attention and cognitive load of vehicle occupants by comparing visual field information with external vehicle perception. Based on this comparison, measures are taken, such as issuing warnings or activating safety systems.
[0007] A method for detecting driver attention is described in document DE 10 2015 201 588 B4. This involves detecting the driver's eye position and orientation. The attention data is stored and used to generate control parameters to enable long-term adaptation of the system.
[0008] None of the aforementioned existing technologies offer a solution designed to prevent motion sickness in the field of autonomous driving. Summary of the Invention
[0009] The objective of this invention is to describe a method and apparatus for controlling autonomous driving functions, wherein the method and apparatus prevent lane changes performed by driving functions from causing motion sickness to vehicle users.
[0010] This task is accomplished by the method and apparatus according to the invention. Advantageous improvements are described in the specification.
[0011] This invention provides a method for controlling an autonomous driving function of a vehicle, wherein the autonomous driving function is designed to perform lane changes when predetermined trigger conditions are present. In this method, the user's gaze direction is detected. The detected gaze direction is compared with a target gaze direction facing traffic. According to the invention, if the comparison reveals a deviation between the detected gaze direction and the target gaze direction facing traffic, then lane changes are performed slowly or suppressed despite the presence of the predetermined trigger conditions.
[0012] This invention enables vehicle users to utilize a driving function that also enables lane changing based on applicable traffic conditions without the risk of motion sickness associated with lane changes. Typically, this motion sickness arises from the discrepancy between the lateral and / or longitudinal acceleration perceived by the user's vestibular system and the lateral and / or longitudinal acceleration detected visually. Therefore, this invention specifies that lane changes are only performed when a match is established between the user's current line of sight and a target line of sight facing the traffic. This is determined, for example, within the scope of autonomous driving functions, particularly based on traffic-related parameters such as traffic density, speed, oncoming traffic, and line-of-sight conditions. This target line of sight indicates a predetermined direction towards which the user looks to perceive the traffic conditions ahead along the driving direction. Conversely, if such a match is determined not to exist, lane changes are suppressed or at least performed slowly, thereby avoiding motion sickness for the vehicle user.
[0013] The vehicle user whose line of sight is detected and compared with the target line of sight is typically the driver, i.e., the person in the driver's seat. However, it is also possible that lane change control according to the invention is performed with respect to one or more other vehicle passengers. For example, automatic lane changes can be performed, slowed down, or inhibited based on the line of sight of the front passenger.
[0014] In this exemplary scenario, it is subsequently assumed that the vehicle is traveling on a highway and operating at a higher level of automation. Therefore, the driver takes over guidance of the vehicle; that is, the vehicle drives itself without the driver needing to intervene in driving operations. In this case, for example, by having the driver look at his laptop on his lap, the driver can focus on tasks other than driving operations.
[0015] When a vehicle is approaching a significantly slower vehicle ahead while in autonomous driving mode, the automated driving function considers a lane change to overtake the slower vehicle. To do this, within the scope of the automated driving function, predetermined triggering conditions for the lane change are checked. These triggering conditions may be related to multiple environment- and vehicle-specific parameters that are detected and evaluated for the lane change. These include, for example, information about position, speed, and distance to adjacent vehicles, as well as identifying and evaluating lane markings and the availability of adjacent lanes. Additionally, road-related data such as road geometry, lane widths, and relevant traffic signs may be considered. Furthermore, vehicle-specific parameters such as current vehicle speed, lateral and longitudinal acceleration, vehicle position, steering angle, and hazard lights may be incorporated into the check of the lane change triggering conditions. If the triggering conditions are met, then a lane change is considered.
[0016] In this exemplary scenario, it is assumed that the driver looks down at his laptop at the point in time when all the triggering conditions for the lane change under consideration are met. Therefore, his current line of sight deviates from a predefined target line of sight, which the driver looks towards in order to observe the traffic ahead, for example, during manual driving. The method according to the invention now specifies detecting this deviation between the driver's current line of sight and the target line of sight facing the traffic, and subsequently executing or suppressing the additionally configured lane change trigger more slowly. Thus, if the driver is not looking towards the target line of sight, then at least the lane change is slowed down.
[0017] This method therefore ensures that lane changes are made in a consistent manner only when the driver is looking at the traffic area ahead. This avoids the discrepancy between vestibular and visual perception, which can cause motion sickness in drivers.
[0018] In a preferred embodiment, the autonomous driving function is designed to enable the vehicle to drive automatically without user intervention. This embodiment is designed to enable driving operations at higher levels of automation, for example, within SAE Levels 3 to 5. Therefore, lane changes controlled according to the invention can be performed within the scope of conditional automation, in which the vehicle drives automatically but allows the driver to intervene when necessary. At higher levels of automation, the vehicle drives autonomously in specific scenarios, such as on highways, without any user intervention. Finally, lane changes related to line-of-sight can be particularly used for fully autonomous driving operations that no longer require a human driver.
[0019] Preferably, once it is determined that the user's detected line of sight matches the target line of sight facing the traffic, the suppression of lane change triggering is canceled or the process is slowed down. In this embodiment, lane change is only performed in a manner that does not consider the risk of motion sickness when the user again directs their gaze toward the traffic ahead.
[0020] Here, the control according to the invention can be designed to wait directly without further action until a comparison between the user's current line of sight and a predefined target line of sight is made, so as to trigger the upcoming lane change in advance or to perform the lane change in a manner that is unaffected by possible motion sickness.
[0021] Alternatively, when a lane change is prevented or slowed down despite the presence of a predetermined triggering condition, an indication signal can be output to the vehicle user. In this embodiment, the user is indicated that the lane change triggering condition has been met under current traffic conditions, and thus a lane change is indicated. The indication signal prompts the user to direct their gaze toward the traffic direction ahead, which, subsequently, enables lane change to be performed without considering potential motion sickness, thanks to gaze direction detection. The indication signal can be optically generated, for example, through a subtle light arrangement produced by corresponding lighting devices inside the vehicle. However, an acoustic indication signal can also be output, for example.
[0022] In a specific implementation, the vehicle's speed is detected, and lane changing is prevented only if the detected speed additionally exceeds a predetermined minimum speed. In this implementation, vehicle speed is directly considered as an additional condition for preventing or slowing lane changes according to the invention, in addition to the deviation between the vehicle user's line of sight and the target line of sight facing traffic. Therefore, lane changes are only suppressed or slowed down from a certain minimum speed. This takes into account the fact that, when the speed is sufficiently high, motion sickness in the vehicle user is empirically considered first when changing lanes.
[0023] However, the lane change control according to the invention can also be used at lower speeds. Therefore, even when changing lanes at lower speeds, the lateral acceleration can be relatively high. In this case, it is advantageous, especially for users who tend to find smaller accelerations unpleasant, to control lane changes by taking into account the combination of longitudinal and lateral acceleration, thereby avoiding motion sickness.
[0024] In a preferred embodiment, the user's gaze direction is detected by means of at least one sensor disposed inside the vehicle. For example, one or more interior space cameras may be provided, each facing one or more user orientations whose gaze direction is being monitored.
[0025] The autonomous driving function can be designed to dynamically change lanes according to preset lane-change parameters when trigger conditions are met. If the detected line-of-sight direction deviates from the target line-of-sight direction facing the traffic, the lane-change dynamics are altered to execute or suppress the lane change more slowly compared to when determined by preset lane-change dynamics. Preferably, the lane-change dynamics can be individually set for the specific user. This allows us to determine under what circumstances the user accepts which lane-change dynamics.
[0026] According to another aspect of the invention, an apparatus for controlling an autonomous driving function of a vehicle is provided. The apparatus includes at least one sensor configured to detect the direction of the vehicle user's gaze. The apparatus also includes a control unit configured to control the autonomous driving function. The autonomous driving function is designed to perform a lane change of the vehicle when a predetermined trigger condition is present. The control unit is configured to compare the detected direction of the user's gaze with a target direction of gaze facing traffic. Furthermore, the control unit is configured to, if the comparison reveals a deviation between the detected direction of gaze and the target direction of gaze facing traffic, then slowly execute or suppress a lane change despite the presence of the predetermined trigger condition.
[0027] Sensors include, for example, interior space cameras integrated into the vehicle's rearview mirrors. Using such interior space cameras integrated into rearview mirrors allows for particularly reliable detection of the driver's field of vision.
[0028] The apparatus according to the invention has the same advantages as the claimed method. In particular, the apparatus can be improved using features of the method according to the invention. Attached Figure Description
[0029] The implementation methods are then described in detail with reference to the accompanying drawings.
[0030] Figure 1 A perspective schematic diagram of the vehicle's interior space is shown; and
[0031] Figure 2The image shown is taken by an interior space camera and depicts the driver. Detailed Implementation
[0032] Figure 1 A perspective schematic diagram shows the cockpit area 100 of vehicle 102. Then, using... Figure 1 The described component is understood as part of a device capable of controlling the autonomous driving function of vehicle 102.
[0033] The driver 104 sits behind the steering wheel 122 in the driver's seat of the vehicle 102, which is hidden for the sake of simplicity. A central information display device 108, implemented as a touch screen, and a display segment 110 having multiple display areas 112, 116, 118, and 120 are provided in the cockpit area 100.
[0034] Display section 110 is positioned above instrument panel 114 and projects information across the entire width of windshield 130 of vehicle 102. In display section 110, driving data is displayed to driver 104 within the driver's field of vision, such as the current gear in display area 116, the current speed in display area 118, and the remaining driving range in display area 120. Other information, such as outside temperature and weather, is displayed in display area 112, further away from driver 104. Additional information may also be displayed on central information display 108. Driver 104 can also perform inputs, such as destination input for the navigation system, and execute and change vehicle settings via central information display 108.
[0035] An interior rearview mirror 132 and an interior space camera 134 integrated in the interior rearview mirror 132 are arranged in the area above the windshield 130. In this embodiment, the interior space camera 134 is used to detect the interior space of the vehicle 102 at least in the area where the upper body of the driver 104 is located.
[0036] like Figure 1As shown in the lower part, vehicle 102 also has a control unit 138, which is connected to the aforementioned components, such as the central information display 108, display section 110, steering wheel 122, and interior space camera 134, via data input and data output terminals 140. Furthermore, control unit 138 can be coupled to other input and output units, such as microphone 124 or speaker 126. Additionally, control unit 138 communicates, for example, with functional units for detecting the surrounding environment of vehicle 102. Such functional units include, for example, a front-facing camera 144, radar sensor 146 and / or laser rangefinder 148, as well as other sensors, such as rain sensor and temperature sensor. Coupling with control unit 138 can be wired or wireless, or can be achieved via a bus system and / or network.
[0037] The control unit 138 also includes a communication module 142 configured to establish a connection with a telecommunications network, particularly a mobile network. Data can be transmitted from and to the vehicle 102 via the communication module 142, particularly from and to the central database and / or processing unit of the vehicle manufacturer and / or service provider.
[0038] Control unit 138 is configured to control vehicle 102 according to an autonomous driving function. Here, the driving function is designed in principle to trigger lane changes in vehicle 102 when predetermined triggering conditions are met. To determine whether the triggering conditions are met, control unit 138 further utilizes data provided to it by a front-facing camera 144, radar sensor 146, and / or laser rangefinder 148. This data represents parameters such as position, speed, distance to adjacent vehicles, current road geometry, lane width, traffic signs, etc.
[0039] The control unit 138 is also configured to consider the driver 102's line of sight in addition to the aforementioned triggering conditions when controlling lane changes. To this end, the control unit 138 evaluates image data represented by the image of the driver 104 captured by the interior space camera 134. An example of such an image of the driver 104 is shown in... Figure 2 As shown in the image.
[0040] An interior space camera 134 oriented towards the driver 104 has a sufficiently large field of view to detect the driver 102's eyes. A control unit 138 processes the image data provided by the interior space camera 134 to determine the driver 104's gaze direction. Figure 2 In the example shown, the driver 104 is not looking at the traffic in front of him through the windshield 130, but is looking down at the laptop 250 placed on his lap.
[0041] The control unit 138 compares the detected line-of-sight direction with the target line-of-sight direction facing the traffic. This target line-of-sight direction is predefined as the direction the driver 104 views when observing traffic ahead through the windshield 130. If the comparison indicates a deviation between the driver 104's current line-of-sight direction and the target line-of-sight direction facing the traffic, then a lane change is either slowly executed or completely suppressed under the control of the control unit 138. Therefore, even if the triggering conditions for a lane change are met, the considered lane change may be slowly executed or not executed at all at a given time.
[0042] By slowing down or inhibiting lane changes when the driver 104’s current detected line of sight is inconsistent with the predefined target line of sight, motion sickness that the driver 104 may suffer due to additional differences between the driver’s vestibular perception and the driver’s visual perception can be reliably avoided.
[0043] This is particularly applicable when lane changes should be performed at high speeds. Accordingly, the control unit 138 can also be configured such that, in order to control lane changes, the control unit additionally considers the speed of the vehicle 102 in addition to the predetermined triggering conditions and the driver 104's line of sight. In this embodiment, the control unit 138 detects the speed of the vehicle 102 and compares that speed with a predetermined minimum speed. If the control unit 138 determines that the speed of the vehicle 102 is less than the minimum speed, then the considered lane change will not be slowed down or inhibited even if the driver 104's current line of sight deviates from the predefined target line of sight. In other words, in this case, the slowing down or inhibiting of lane changes according to the invention is only used when the vehicle speed is high enough that there is a risk of motion sickness in the event of an unexpected lane change for the driver.
[0044] Lane change control according to the invention will then be illustrated by example. Consider the following traffic situation where driver 104 uses his vehicle 102 to drive onto a highway in manual driving operation at the lowest automation level, SAE Level 0. There, driver 104 activates a higher automation level, such as SAE Level 3 or 4, which temporarily takes over the driver's guidance of vehicle 102. This automation level allows driver 104 to open his laptop 250 and work while vehicle 102 is driving autonomously on the highway. Here, control unit 138 further continuously monitors the autonomous driving operation based on data provided by front-facing camera 144, radar sensor 146, and / or laser rangefinder sensor 148. In particular, control unit 138 continuously checks whether a lane change trigger condition exists. If vehicle 102, for example, approaches a significantly slower truck ahead, then control unit 138 determines that, from a certain point in time, the aforementioned trigger condition has been met and thus indicates a lane change. However, according to the present invention, in this case, the control unit 138 also checks, using image data provided by the interior space camera 134, whether the user 104's current line of sight is consistent with a predefined target line of sight. If the control unit 138 determines such consistency, it triggers a lane change, in which case the vehicle 102 overtakes a slow-moving truck ahead. If the control unit 138, conversely, determines that the driver 104's current line of sight deviates from the line of sight facing traffic, it prevents the lane change from being triggered, or at least ensures that the lane change is performed more slowly than it would be if possible motion sickness were not considered. The control unit 138 releases the lane change only when continuous monitoring of the driver 104's line of sight results in consistency with the target line of sight, for example, without considering possible motion sickness. Of course, release is only made if the predetermined triggering conditions for the lane change are always met or have been met again at that point in time.
[0045] It should be noted that the above-described embodiments are merely exemplary. Therefore, in these embodiments, the driver's gaze direction is determined using image data provided by a single interior space camera 134 integrated in the rearview mirror 132. Multiple interior space cameras may also be configured to detect gaze direction and arranged in other locations within the vehicle's interior space. Furthermore, determining the gaze direction is not limited to the use of cameras. Other types of sensors may also be used, as long as these sensors provide data capable of detecting the driver's gaze direction.
[0046] Lane change control according to the invention is not limited to detecting the gaze direction of a vehicle user in the driver's seat. Especially in applications with the highest level of automation where the vehicle operates fully autonomously without a driver, lane changes can also be controlled based on the gaze direction of a vehicle user located in the front passenger seat or in the rear area of the vehicle's interior. In the latter case, the gaze directions of multiple vehicle passengers can also be considered when controlling lane changes. In these embodiments, an appropriate number of gaze direction sensors with corresponding orientations must be provided to ensure reliable detection of the gaze direction of the respective vehicle user.
[0047] Of course, the above scenario of user 104 working on their laptop 250 is merely an example. Other situations where the user's gaze is no longer focused on traffic and therefore motion sickness may occur when changing lanes include situations where the user is reading, napping, or playing, chatting while making eye contact with one of the passengers in another vehicle, conducting video conferences, watching movies, etc. List of reference numerals 100 cockpit area Vehicle 102 104 drivers 108 Central Information Display Device 110 display segment 112 display area 114 Dashboard 116 display area 118 display area 120 display area 122 steering wheel 124 microphones 126 speakers 130 front windshield 132 interior rearview mirror 134 interior space cameras 138 control unit 140 Data Input Terminal, Data Output Terminal 142 communication module 200 images 250 laptop
Claims
1. A method for controlling an autonomous driving function of a vehicle (102), wherein, The autonomous driving function is designed to perform lane changing of the vehicle (102) when predetermined triggering conditions are present, the method comprising: Detect the direction of the user's (104's) gaze in the vehicle (102); The detected gaze direction of the user (104) is compared with the target gaze direction facing the traffic; and If the comparison reveals a deviation between the detected line of sight and the target line of sight facing traffic, then the lane change is suppressed or performed slowly despite the presence of the predetermined triggering condition.
2. The method according to claim 1, wherein, The autonomous driving function is designed to enable the vehicle (102) to drive autonomously without the intervention of the user (104).
3. The method according to claim 1 or 2, wherein, Once the comparison determines that the detected line of sight of the user (104) is consistent with the target line of sight facing the traffic, the suppression of the lane change is canceled or the lane change is slowed down.
4. The method according to any one of the preceding claims, wherein, When the lane change is suppressed or slowed down despite the existence of the predetermined triggering condition, an indication signal is output to the user (104).
5. The method according to any one of the preceding claims, wherein, The speed of the vehicle (102) is detected, and the lane change is suppressed or slowed down only when the detected speed of the vehicle (102) is additionally greater than a predetermined minimum speed.
6. The method according to any one of the preceding claims, wherein, The autonomous driving function is designed to dynamically execute the lane change according to a preset lane change when the triggering condition is met; and The method includes changing the lane change dynamics so that when the detected line of sight deviates from the target line of sight facing the traffic, the lane change is executed or suppressed more slowly than by determining the lane change dynamics through a preset method.
7. The method according to any one of the preceding claims, wherein, The user's (104) line of sight is detected by means of at least one sensor (134) arranged inside the vehicle (102).
8. An apparatus for controlling an autonomous driving function of a vehicle (102), comprising: At least one sensor (134) configured to detect the direction of the gaze of a user (104) of the vehicle (102); and Control unit (138), the control unit is designed to control the autonomous driving function, wherein the autonomous driving function is designed to perform lane changing of the vehicle (102) when a predetermined triggering condition is present; The control unit (138) is also designed to compare the detected gaze direction of the user (104) with the target gaze direction facing the traffic, and The control unit (138) is designed to suppress or slowly perform the lane change even though the predetermined triggering condition exists, if the comparison shows a deviation between the detected line of sight and the target line of sight facing the traffic.
9. The apparatus according to claim 8, wherein, The sensor includes an interior space camera (134) integrated into the rearview mirror (132) of the vehicle (102).
Citation Information
Patent Citations
Method and system for carrying out automatic control of the movement of a vehicle
DE102015201588B4
Monitoring the attention and cognitive load of occupants for autonomous and semi-autonomous driving applications
DE102021126254A1
METHOD FOR RELEASING A FUNCTIONAL STAGE OF A DRIVING ASSISTANCE SYSTEM, COMPUTING DEVICE, COMPUTER-READY STORAGE MEDIUM, VEHICLE
DE102023106572A1
