Method and system for issuing a warning signal when the optical detection of a vehicle occupant is impaired

Abstract

The invention relates to a method for operating a driver assistance system (11) having an optical detection device (12) by means of which an occupant (18) is recorded and in which at least one warning signal is generated upon a detected impairment of the recording by at least one object (21) in the recording field (13) of the optical detection device (12). Provision is made here for the warning signal to trigger a display of the recording in a display device (24) arranged in a field of vision (19) of the occupant (18). The invention further relates to such a driver assistance system (11).

Description

[0001] The invention relates to a method for operating a driver assistance system according to the preamble of patent claim 1. Furthermore, the invention relates to such a driver assistance system.

[0002] In modern vehicles without an instrument cluster in the line of sight, the driver often doesn't need to adjust their seat and steering wheel position. However, this can result in the driver attention camera installed in the field of vision not having an optimal view of essential driver information such as head position and eyes. If the steering wheel or seat positions are incorrectly adjusted, certain driver assistance systems, such as automatic restart via ACC, the emergency stop assistant, eye recognition, the hands-off detection (HOO) system, or the drowsiness assistant, may be limited in their functionality or unavailable.

[0003] DE 10 2018 004 277 A1 discloses a method for operating a driver-side camera in which objects that restrict the driver's view are automatically detected and classified. The driver is informed of the detected object so that they can take measures to restore the camera's functionality for assistance systems, such as autonomous driving.

[0004] The object of the invention is to detect an impairment of an optical detection device of a driver assistance system as quickly and reliably as possible in order to make the functioning of the driver assistance system more reliable.

[0005] This object is achieved according to the invention by a method having the features of patent claim 1 and by means of a driver assistance system according to the invention. Advantageous embodiments of the invention are the subject of the dependent patent claims and the description.

[0006] One aspect of the invention relates to a method for operating a driver assistance system with an optical detection device which is designed to detect an occupant and to generate a warning signal upon detection of impairments of the recording by at least one object in the recording field of the detection device.

[0007] The optical detection device is, for example, a vehicle camera positioned or installed and thus arranged in the interior of the vehicle. This vehicle camera can also be, for example, a mirror camera, an infrared camera, or a high-resolution RGB camera designed to detect or record the occupant, in particular the driver, and other occupants, even in poor visibility conditions. It can be arranged at various positions in the interior of the motor vehicle, for example, in the area of ​​the interior mirror, in the instrument panel, or on the A-pillar, to enable the most unobstructed view of the occupant possible.

[0008] This positioning allows the vehicle camera to ensure unobstructed detection of the head and eye movements of the occupant / driver as well as potential objects in the field of view.

[0009] The driver assistance system itself can, for example, be such a system or be coupled with one that uses facial recognition to monitor and assist the occupant / driver. One example of this is a Driver Monitoring System (DMS), which monitors the driver's attention by detecting eye and head movements. This system can be used, in particular, for lane departure warning systems and / or emergency braking systems to determine whether the driver is paying attention at all times. Systems such as Adaptive Cruise Control (ACC) or autonomous driving functions can also either require facial recognition themselves or be coupled with another system that can be used to transfer control of the vehicle to the driver in critical situations.In addition, facial recognition can be used in fatigue detection systems connected to the driver assistance system, in particular to identify early signs of fatigue and exhaustion and to warn the driver.

[0010] To achieve the object of the invention, the invention provides that the warning signal triggers a display of the current image on a display device located in the occupant's field of vision. This display device shows the captured real image of the interior and simultaneously indicates the detected impairment, such as eye or head obstruction, with a text message. In addition, the occupant can be advised of an optimal seat and steering wheel positioning to improve the functionality of the driver assistance system.

[0011] The positioning of the display for the occupants, especially the driver, requires particularly quick and clear information delivery. Existing display devices in the instrument panel, such as digital displays or displays in the center console area, can be used to show the images directly in or near the driver's field of vision. A central infotainment screen, for example, offers another option for displaying the images. Alternatively, a head-up display (HUD) can be used, which projects information directly onto the windshield in the driver's field of vision, so the driver doesn't have to take their eyes off the road.

[0012] The image can be displayed as sharply as possible on the display to better illustrate the recognition. For example, the background of the image can be blurred, while the driver / passenger's face is shown particularly clearly and sharply. A high-contrast representation of the face can also be used to emphasize important details and provide better visual feedback for the driver / passenger.

[0013] The optical detection device is designed to reliably detect and analyze common objects such as hats, headgear, glasses, and other items that could interfere with facial recognition. Depending on the detection of the object, automatic categorizations can be performed, providing appropriate prompts, such as prompts to remove hats or glasses or to adjust the seating position to improve the function of the optical detection device. Individual driver / passenger settings can also be stored to enable faster detection and adaptation of suggestions or prompts. This could include, for example, the preferred seating position or the wearing of glasses.

[0014] In summary, the driver assistance system comprises an optical detection device and an electronic computing device that work together to enable automated analysis of the driver's environment. The optical detection device continuously provides data on the position of the driver / occupant and any objects that could interfere with facial recognition. This information is processed by the electronic computing device, which uses pattern recognition and object classification algorithms to identify relevant objects such as hats or glasses. Based on this analysis, the driver assistance system generates specific instructions or adjustment suggestions for the driver / occupant, such as requesting them to remove headgear or change their seating position.By storing individual preferences of the driver / occupant, the driver assistance system can also use previous data to suggest faster adjustments and thus improve interaction between the driver / occupant and the driver assistance system.

[0015] In an advantageous embodiment of the invention, it is provided that, in addition to the inclusion in the display device, at least one information text is added and / or a further visual and / or auditory and / or haptic indication is triggered. Thus, it is provided to additionally trigger further visual, acoustic or haptic signals in order to draw the driver's / passenger's attention to the required measures shown in the display device. Visual indications can be provided, for example, by bright, flashing symbols on the display, while acoustic warnings are issued via a loudspeaker system, e.g. of the infotainment system. Haptic feedback could be provided, for example, by vibration elements integrated into the steering wheel or driver's seat, which alert the driver / passenger to the need for a reaction through noticeable vibrations.

[0016] In another advantageous embodiment of the invention, the warning text indicates the object. The warning text thus indicates the detected object and provides the driver / passenger with specific information, such as a request to change their seating position or posture and / or to remove the object. The warning can, for example, indicate that the head or eyes are being obstructed and that repositioning the seat or adjusting the steering wheel position is necessary to avoid impairing the function of the optical detection device or vehicle camera and the associated assistance systems.

[0017] In a further advantageous embodiment of the invention, the information text indicates an alternative seating position for recording by the optical detection device without interference. The driver / passenger is advised to adjust their seating position so that their head and eye movements can be recorded by the camera without interference, in order to ensure the full functionality of the driver assistance systems. For example, visual boundaries or areas can be shown on the display device which mark the optimal positions for the driver / passenger. These areas show the positions in which the driver's head and eyes are ideally recorded in order to enable uninterrupted function of the optical detection device. This supports the driver / passenger in positioning themselves within the optimal optical detection range.recording field to enable the best possible recordings for the driver assistance system(s).

[0018] In another advantageous embodiment of the invention, the information text for the alternative seating position is only added when the at least one object is detected for a defined time period. In particular, if an obscuration of the driver's face or head is detected at speeds above a certain threshold for a specified period of time, feedback is provided to the driver via the driver's interior camera. This feedback is intended to help the driver adjust the seat and steering wheel position so that the camera can again detect objects without restrictions and the assistance systems function properly. Furthermore, this time period can be designed to be adaptable in order to be able to react flexibly to different driving situations.The driver assistance system is designed to detect whether the object impairing the view of the optical detection device remains permanently in the field of view or optical detection range or is only present temporarily. Based on this analysis, a decision is made as to whether and when the warning text to adjust the seat or steering wheel position is issued in order to avoid unnecessary warnings and to only notify the driver when detection is permanently impaired. In particular, objects can be detected, recognized, and automatically categorized in order to adjust the corresponding settings of the driver assistance system. Depending on the classification of the object, such as temporary or permanent obstruction by the steering wheel, headgear, or other objects, the driver assistance system adapts the type and duration of the warnings as well as the suggested actions.This enables the optical detection device to react flexibly to different objects and to provide appropriate feedback to the driver / occupant, in particular to enable undisturbed and clean detection or recording.

[0019] In a further advantageous embodiment of the invention, it is provided that the warning signal is triggered if the driver assistance system does not recognize the occupant after a defined number of failed recognition attempts. This applies in particular to driver assistance systems such as Adaptive Cruise Control (ACC) or other coupled driver assistance systems that rely on reliable recognition of the driver's face in order to function correctly. If, for example, ACC is unable to automatically restart after several unsuccessful attempts to detect or identify the driver / occupant, the warning signal is triggered and the display device is applied. Furthermore, the other previously mentioned acoustic and / or visual and / or haptic signals / feedback can also be triggered. The driver assistance system could also, for example,Provide a variety of specific indications (text or other indications through contrast or color), such as that the driver's face is obscured or outside the field of view. This allows the driver to react proactively, for example, by adjusting their seating position or removing an object such as glasses or headgear that interferes with facial recognition. This functionality ensures that the driver assistance systems that rely on driver / passenger detection continue to operate optimally / efficiently / at a high level, and that safety while driving is essentially maintained.Such feedback not only increases the responsiveness of the driver / occupant, but also reduces the risk that critical functions such as emergency braking assistants and / or fatigue warning systems and / or automatic restart in ACC are deactivated due to non-detection.

[0020] In another advantageous embodiment of the invention, the warning signal is only generated in relevant and / or predefined driving scenarios. For example, in driver assistance systems such as hands-on detection (HOO), the warning signal is only activated when a lack of attention on the part of the driver / passenger is detected or recorded. In cases where an obscuration of the face or eyes impairs the field of view of the optical detection device, the image is displayed, thus providing a real / live image. This allows the driver to visually understand the limitations of the optical detection device or camera and make necessary corrections, such as adjusting the seat or steering wheel position.This targeted activation of the warning signal prevents unnecessary interruptions during normal driving conditions and ensures that the driver is only notified in safety-critical driving situations. In particular, it is also intended and noted that the mere activation of the indicator device can / should serve as a sufficient warning signal to alert the driver / occupant that there is a compromise to the optical detection system. This visual feedback alone can prompt or indicate to the driver / occupant the need to take the aforementioned actions, without the need for additional acoustic or haptic warnings.

[0021] In another advantageous embodiment of the invention, it is provided that the occupant's head pose is additionally digitally framed for recording without interference and displayed on the display device. In particular, upper and lower maximum edges of the field of view are specified to ensure that the driver / occupant's head position remains within the recording field of the optical detection device. Furthermore, a nod of the head can be interpreted as a loss of attention on the part of the driver / occupant, and in this case, the driver assistance system automatically triggers a warning signal or alert. A tolerance for head movement is also provided so that slight movements do not immediately trigger a warning signal. However, this tolerance can be adapted to enable more precise or more flexible detection of the head position depending on the driving situation or the personal preferences of the driver / occupant.

[0022] In a further advantageous embodiment of the invention, a continuous analysis of the images is carried out in the background, for example at predetermined time intervals. This analysis records the head and / or eye positions of the driver / passenger without the images being directly visible on the display device. This ensures that the driver always remains in the field of view of the optical detection device. The driver assistance system carries out this monitoring unnoticed in the background and only intervenes when a deviation is detected that could impair or already impairs the proper detection of the driver / passenger. In this way, the driver / passenger is only informed of a necessary correction in relevant cases, without the ongoing analysis distracting from their attention.

[0023] It is also possible for the warning signal to be issued not only to the driver but also to another occupant to increase safety in the vehicle. This can be particularly advantageous in critical situations when the driver fails to respond to a warning or is distracted. Another occupant could be informed of the safety-critical condition through acoustic, visual, or haptic signals, enabling them to intervene if necessary and thus increase driving safety.

[0024] A further aspect of the invention relates to a driver assistance system with an electronic computing device and an optical detection device designed to detect an occupant. This driver assistance system generates a warning signal as soon as an impairment of the image is detected by an object in the image field of the optical detection device. A display device of the driver assistance system is arranged in the occupant's field of vision and displays the captured image depending on the warning signal. The driver assistance system comprises all necessary components, modules, and software to fully execute all process steps, from acquisition and analysis to the output of the warning signal and display.Furthermore, components already installed in the vehicle can be used, which are coupled with the driver assistance system or are already integrated to carry out the method steps. This enables, in particular, the integration of existing systems, such as the infotainment screen, the rearview camera, or the on-board computer, thereby increasing the functionality of the driver assistance system according to the invention without requiring additional hardware. This combination of existing and new systems contributes to reducing costs and complexity during implementation.

[0025] Further features of the invention emerge from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combination, but also in other combinations or on their own.

[0026] The invention will now be explained in more detail using a preferred embodiment and with reference to the drawing. Fig. 1 a schematic side view of a motor vehicle to illustrate a driver assistance system implemented therein for carrying out a method according to the invention. Fig. Figure 1 shows a schematic side view of a motor vehicle 10 implementing a driver assistance system 11 for carrying out a method according to the invention. The motor vehicle 10 comprises all the necessary components that carry out all the method steps.

[0027] In the interior of the motor vehicle 10 there is an optical detection device, which is designed, for example, as a driver camera 12 and is thus already installed in the motor vehicle 10 and which is mounted, for example, on the upper area of ​​the A-pillar 14 or alternatively near the rearview mirror 16. This driver camera 12 could be aligned to detect the occupant, in particular the driver 18. The driver camera 12 is designed to record and monitor the face and eye position of the driver 18 in order to check their attention and potential impairments by objects 21 in a detection area or in the recording field 13 of the optical detection device or driver camera 12.

[0028] The driver assistance system 11 may, for example, comprise an electronic computing device 20, which is installed or arranged centrally in the motor vehicle, preferably near the dashboard or in the area of ​​the center console. This electronic computing device 20 is directly connected or electronically coupled to the driver camera 12, and the images or image data captured by the camera 12 can therefore be processed by this electronic computing device 20. The computing device 20 could, for example, be equipped with algorithms, software, machine learning, or AI to detect and classify objects 21 in the field of view or recording field 13 of the driver camera 12. This could include both static objects X, such as the steering wheel 22, if, for example, the driver camera 12 is arranged behind it, and temporary objects X, such as glasses or headgear worn by the driver 18.

[0029] Accordingly, a method for operating the driver assistance system 11 with the optical detection device 12 or driver camera 12 is provided. An occupant, in particular the driver 18, is recorded by means of the driver camera 12. Subsequently, at least one warning signal is generated upon detection of an impairment of the recording by at least one object 21 in the recording field 13 or detection range of the optical detection device 12 or driver camera 12. The warning signal triggers a display of the recording on a display device 24 arranged in a field of vision 19 of the occupant or driver 18. This means that upon detection of the object 21, this warning signal is then forwarded to the display device 24, which, for example, could be integrated into the dashboard and thus be located in the field of vision 19 of the driver 18.The display device 24 could display a real-time image of the driver 18 captured by the camera 12, as well as a prompt text that advises the driver 18 to change the seating position or steering wheel position to improve the view of the driver camera 12.

[0030] The driver camera 12 is thus also intended to capture the face and eye position of the driver 18 within a defined recording field 13. The recording field 13 is determined by the orientation of the driver camera 12 and comprises the area in which the driver 18 can be optimally captured by the driver camera 12. If an obstruction is detected in the recording field 13, the driver assistance system 11 can display an information text on the display device 24 to provide assistance to the driver 18. This information text can, for example, contain a request to adjust the seat position or steering wheel position to enable better positioning of the driver 18 relative to the driver camera 12 and to ensure complete recording. These instructions help the driver 18 adopt the best head and body positions within the recording field 13 so that the driver assistance system 11 can operate correctly.

[0031] Additionally, an acoustic warning device 26 could be installed in the motor vehicle 10, which can be activated or controlled by the electronic computing device 20 if the driver 18 does not respond to the visual indications of the display device 24. This acoustic warning device 26 could be installed, for example, in the area of ​​the dashboard or the door panel and emit a warning tone to attract the attention of the driver 18. A haptic feedback system 30 could also be integrated into the center console of the motor vehicle 10, which would inform the driver 18 through vibrations that there is an impairment of the camera's view or the detection area 13. This haptic feedback could be transmitted, for example, via the seats 32 or the steering wheel 22 to provide the driver 18 with an additional warning.

[0032] The electronic computing device 20 could also be connected to a head-up display (HUD) 38, which would be displayed on the windshield in the driver's 18 field of vision. In addition to critical information, the HUD 38 could also display the image captured by the driver assistance system 11. This visual display would assist the driver 18 without requiring them to take their eyes off the road.

[0033] The entire driver assistance system 11 in the motor vehicle 10 thus comprises all the necessary components to carry out the described method steps. The optical detection device 12 in the form of the driver camera 12, the electronic computing device 20, the display device 24, the acoustic warning device 26, the haptic feedback system 30, and the HUD 38 could be coupled and work together to enable reliable monitoring of the driver 18 and thus support the driver assistance system 11.

[0034] In Fig. 1 also schematically illustrates the potential connection or electronic coupling of the individual components to the central electronic computing device 20, particularly in dashed lines. The driver camera 12 could be coupled to the electronic computing device 20 via a data connection, which would continuously analyze image data. If an impairment is detected, the warning signal could be forwarded to the display device 24. At the same time, the acoustic warning device 26 could be activated if the driver 18 does not react. The haptic feedback system 30 could also be connected to the electronic computing device 20, so that the entire driver assistance system 11 could coordinate and execute the procedure accordingly.

[0035] In summary, the invention proposes camera feedback in the event of visual obscuration for driver assistance systems 11. This feedback is provided when the view of the driver camera 12 is obstructed by objects 21. By detecting and analyzing the obscuration, the driver 18 can be visually, acoustically, or haptically informed that an adjustment of the seat position or steering wheel position is necessary to ensure the proper functioning of the driver assistance systems 11. List of reference symbols 10 motor vehicles 11 Driver assistance system 12 optical detection device / driver camera 13 Recording field 14 A-pillar 16 rearview mirrors 18 passengers / drivers 19 Field of view 20 electronic computing devices 21 objects 22 Steering wheel 24 Display device 26 Warning device 30 Feedback system 32 seats 38 HUD

Claims

[1] Method for operating a driver assistance system (11) with an optical detection device (12) by means of which an occupant (18) is recorded and in which at least one warning signal is generated in the event of a detected impairment of the recording by at least one object (21) in the recording field (13) of the optical detection device (12), wherein the warning signal triggers a display of the recording in a display device (24) arranged in a field of vision (19) of the occupant (18), and wherein the warning signal is triggered if the driver assistance system (11) does not recognize the occupant (18) after a defined number of failed recognition attempts. [2] The method of claim 1, wherein: in addition to the recording in the display device (24), at least one information text is added, and / or another visual and / or auditory and / or haptic cue is triggered. [3] Method according to claim 2, wherein the reference text refers to the object (21). [4] Method according to claim 2 or 3, wherein the information text indicates an alternative seating position for recording by the optical detection device (12) without interference. [5] Method according to claim 4, wherein the information text for the alternative seating position is only added when the at least one object (21) is detected beyond a defined time horizon. [6] Method according to one of the preceding claims, wherein the warning signal is only generated in relevant and / or predetermined driving scenarios. [7] Method according to one of the preceding claims, wherein the occupant (18) is additionally digitally framed with a head pose for a recording without interference and displayed in the display device. [8] Method according to one of the preceding claims, wherein a continuous analysis of the recordings is carried out. [9] Driver assistance system (11) with an optical detection device (12) by means of which an occupant (18) can be recorded and in which at least one warning signal can be generated in the event of a detected impairment of the recording by at least one object (21) in the recording field (13) of the optical detection device (12), wherein a display device (24) is arranged in a field of vision (19) of the occupant (18), which display device is designed to display the recording in dependence on the warning signal, and wherein the warning signal is triggered if the driver assistance system (11) does not recognize the occupant (18) after a defined number of failed recognition attempts.

Images