Hidden camera subsystem for monitoring a vehicle driver

Abstract

A hidden camera subsystem for monitoring a vehicle driver includes an instrument panel, a camera sensor, a reflector, and a first electronic control unit. The instrument panel is located within the vehicle. The instrument panel defines the driver's lower line of sight. The camera sensor has a field of view and is positioned below the lower line of sight such that the camera sensor is hidden from the driver by the instrument panel. The camera sensor operates to generate a sensed video signal of the scene within its field of view. The reflector is positioned above the lower line of sight and is oriented to guide the camera sensor's field of view to include the driver's head in the scene. The first electronic control unit operates to receive the sensed video signal and monitor the driver in response to the driver's head appearing within the sensed video signal.

Description

Cross-reference to related applications

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 594,550, filed October 31, 2023, which is hereby incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure generally relates to systems and methods for using a hidden camera subsystem for monitoring drivers of vehicles. Background Technology

[0003] Monitoring the driver in a car environment typically involves mounting a camera above the dashboard to provide a clear view of the driver. Such a mounting arrangement can cause glare to the driver due to stray light reflected from the camera lens. Summary of the Invention

[0004] This paper provides a hidden camera subsystem for monitoring a vehicle driver. The hidden camera system includes an instrument panel, a camera sensor, a reflector, and a first electronic control unit. The instrument panel is located in the vehicle. The instrument panel defines the driver's downward line of sight. The camera sensor has a field of view and is positioned below the downward line of sight such that the camera sensor is hidden from the driver by the instrument panel. The camera sensor operates to generate a sensed video signal of the scene within its field of view. The reflector is positioned above the downward line of sight and is oriented to guide the camera sensor's field of view to include the driver's head in the scene. The first electronic control unit operates to receive the sensed video signal and monitor the driver in response to the driver's head within the sensed video signal.

[0005] In one or more embodiments of the hidden camera subsystem, the reflector reflects light at multiple polarization axes.

[0006] In one or more embodiments of the hidden camera subsystem, a filter in front of the camera sensor has a specific polarization axis, and a reflector reflects light to match that specific polarization axis.

[0007] In one or more embodiments of the hidden camera subsystem, the reflector is part of the windshield of the vehicle.

[0008] In one or more embodiments of the hidden camera subsystem, the reflector is part of the windshield of the vehicle.

[0009] In one or more embodiments of the hidden camera subsystem, the reflector has a curved shape.

[0010] In one or more embodiments of the hidden camera subsystem, the scene in the sensed video signal is distorted by the curved shape of the reflector, and the first electronic control unit further operates to dedistort the scene in the sensed video signal.

[0011] In one or more embodiments of the hidden camera subsystem, the reflector has a transmittance that allows ambient light entering the vehicle to reach the driver through the windshield at a perceptible magnitude.

[0012] In one or more embodiments, the hidden camera subsystem includes an opaque material disposed between the reflector and the windshield of the vehicle.

[0013] In one or more embodiments of the hidden camera subsystem, the reflector and opaque material are laterally offset from the driver, so that the driver can see the road where the vehicle is when looking straight ahead.

[0014] In one or more embodiments of the hidden camera subsystem, the light rays intersecting the driver's eye movement range and the vehicle's windshield are mirror angles of the camera sensor's axis.

[0015] In one or more embodiments of the hidden camera subsystem, the dashboard has a cavity and the camera sensor is disposed in the cavity.

[0016] In one or more embodiments of the hidden camera subsystem, a first electronic control unit operates to generate output data in one or more control signals based on sensed video signals.

[0017] In one or more embodiments, the hidden camera subsystem includes a display subsystem that communicates with and operates in response to output data received in one or more control signals to generate a plurality of visible images.

[0018] In one or more embodiments, the hidden camera subsystem includes one or more illuminators that communicate with and operate to generate illumination light, which is then applied to the scene.

[0019] In one or more embodiments, the hidden camera subsystem includes an ambient light sensor that communicates with and operates in conjunction with a first electronic control unit to measure the ambient light level of the scene.

[0020] This document provides a vehicle. The vehicle includes a seat for a driver, an instrument panel, a camera, a reflector, and a first electronic control unit. The instrument panel is positioned in front of the seat and establishes the driver's lower line of sight within the seat. A camera sensor has a field of view and is positioned below the lower line of sight such that the camera sensor is hidden from the driver by the instrument panel. The camera sensor operates to generate a sensed video signal of a scene within the field of view. The reflector is positioned above the lower line of sight and is oriented to guide the camera sensor's field of view to include the driver's head in the scene. The first electronic control unit operates to receive the sensed video signal and monitor the driver in response to the driver's head within the sensed video signal.

[0021] In one or more embodiments of the vehicle, a first electronic control unit operates to generate output data in one or more control signals based on sensed video signals.

[0022] In one or more embodiments, the vehicle includes a display subsystem that communicates with and operates in response to output data received in one or more control signals to generate a plurality of visible images.

[0023] In one or more embodiments, the vehicle includes a second electronic control unit that operates to determine one or more driver conditions based on sensed video signals.

[0024] The above-mentioned features and advantages, as well as other features and advantages, of this doctrine will be readily understood from the following detailed description of the best mode for carrying out the doctrine in conjunction with the accompanying drawings. Attached Figure Description

[0025] Figure 1 and Figure 2 A hidden camera subsystem in a vehicle according to one or more exemplary embodiments is shown.

[0026] Figure 3 A flowchart illustrating an operation method according to one or more exemplary embodiments is shown. This disclosure may have various modifications and alternatives, and some representative embodiments are illustrated by way of example in the accompanying drawings and will be described in detail herein. The novel aspects of this disclosure are not limited to the specific forms illustrated in the foregoing drawings. Rather, this disclosure is intended to cover modifications, equivalents, and combinations that fall within the scope of this disclosure as defined by the appended claims. Detailed Implementation

[0027] Embodiments of this disclosure generally provide a system and / or method for monitoring a driver of a vehicle using a hidden camera subsystem. The system / method can detect one or more aspects of the driver (or related driver information) via the camera subsystem, wherein the camera subsystem is invisible to the driver and / or other occupants. The camera subsystem is embedded in the vehicle and cannot be directly viewed by the driver. The camera subsystem includes an instrument panel, a camera sensor, a reflector, and a first electronic control unit.

[0028] A camera sensor receives driver reflections from a fully or partially reflective surface. The camera sensor generates and presents a sensed (or raw or initial) video signal of the driver reflections. The driver reflections typically include at least the driver's head, including the driver's eyes and eyelids.

[0029] In various embodiments, the reflective surface is integrated with the vehicle's windshield (e.g., a glass pane). In other embodiments, the reflective surface is separate from the windshield. The reflective surface may reflect all light polarizations, or it may reflect only on a specific polarization axis aligned (or matched) with a filter in front of the camera sensor, thereby reducing further detectability of the camera sensor to the driver and other occupants.

[0030] The electronic control unit operates to perform image processing capabilities to interpret driver behavior and / or other relevant characteristics.

[0031] The camera subsystem may include a display subsystem embedded in the vehicle. The display subsystem operates to generate a virtual and / or reflective image plane that can be observed only by the driver or by the driver and other occupants.

[0032] In various embodiments, the camera subsystem is independent. Sensing video signals generated by the camera sensor are transmitted to an electronic control unit for processing. One or more control signals are then transmitted by the first electronic control unit in response to the sensing video signals, generating output data that is transmitted to the display subsystem. The display subsystem presents the output data to the driver and / or other occupants of the vehicle.

[0033] In some embodiments, the sensed video signal generated by the camera is also presented to a computer outside the camera subsystem. The computer operates to perform image processing on the sensed video signal to generate additional output data. This additional output data can be transmitted to other electronic devices within the vehicle to take action based on perceived driver behavior and / or other relevant characteristics.

[0034] Figure 1 and Figure 2A hidden camera subsystem in a vehicle according to one or more exemplary embodiments is illustrated. The vehicle 90 typically includes at least a windshield 92, an instrument panel 94, a seat 96 for a driver 80, a camera subsystem 100, a computer 102, and a (second) electronic control unit (ECU B) 104. The camera subsystem 100 includes the instrument panel 94, a camera sensor 110, a reflector 112, one or more illuminators 114, an optional ambient light sensor 116, and a (first) electronic control unit (ECU A) 118. In some embodiments, the camera subsystem 100 includes a display subsystem 120.

[0035] Vehicle 90 operates to carry driver 80 and, possibly, one or more passengers. Vehicle 90 may include mobile vehicles such as cars, trucks, motorcycles, boats, trains, and / or aircraft. Other types of vehicles may be implemented to meet design guidelines for specific applications.

[0036] The driver 80 is shown seated in seat 96 behind the display subsystem 120 of the vehicle 90. In various embodiments, a passenger seated in another seat may be positioned to view the display subsystem 120. The driver 80 has a head 82 (or face) with eyes, eyelids, and a mouth. Ambient light 84 inside the vehicle 90 and illuminating the driver 80's head 82 forms a driver optical image (DOI) 122 that moves toward the windshield 92 and reflector 112. Infrared light 124 generated by illuminator 114 and / or ambient light 86 entering the vehicle 90 through the windshield 92 may also illuminate the driver 80's head 82 to form or supplement the driver optical image 122.

[0037] Camera subsystem 100 operates to monitor driver 80. Camera subsystem 100 operates to receive at least a reflected optical image 126 of the driver 80's head 82 (or face). The reflected optical image 126 is converted into a sensed video signal 128 within camera subsystem 100. Camera subsystem 100 then processes the reflected optical image 126 in the sensed video signal 128 to generate one or more control signals 130. Output data from the control signals 130 can be presented to display subsystem 120 for presentation to driver 80 as one or more visible images 132. In various embodiments, the sensed video signal 128 and / or one or more processed signals 134 can be presented to a circuitry external to camera subsystem 100. In some embodiments, the external circuitry may include one or more computers 102 and / or a second electronic control unit 104.

[0038] The instrument panel 94 can implement a configuration in the vehicle 90 positioned between the camera sensor 110 and the driver 80 (e.g., in front of the seat 96). The instrument panel 94 typically defines a downward line of sight 136 (or line of sight) when the driver 80 looks forward. In various embodiments, the instrument panel 94 may include a cavity 138. The camera sensor 110 is positioned within or behind the instrument panel 94 and below the downward line of sight 136, such that the camera sensor 110 is hidden from the driver 80.

[0039] The camera sensor 110 has a field of view 140 spanning scene 142. The driver 80 typically sits within the field of view 140. The camera sensor 110 also has a camera axis centered within the field of view 140. The camera axis intersects with reflector 112.

[0040] Camera sensor 110 is positioned relative to seat 96 within vehicle 90, with a portion or majority of dashboard 94 between them. The vertical position of camera sensor 110 is below the driver's downward line of sight 136, such that the highest-seeing driver 80 cannot see camera sensor 110 due to dashboard 94. Camera sensor 110 operates to convert a reflected optical image 126 (e.g., a reflected version of driver optical image 122) into a sensed video signal 128. The sensed video signal 128 is presented to a first electronic control unit 118. In various embodiments, the sensed video signal 128 is also presented to an external circuitry system (e.g., computer 102 and / or second electronic control unit 104).

[0041] Camera sensor 110 is oriented such that light rays intersecting the driver's eye movement range 144 and windshield 92 are at a mirror angle (α) of the camera axis. Camera sensor 110 can be mounted in a cavity 138 of instrument panel 94 below the driver's downward line of sight 136. Cavity 138, together with optional dark reflector 112, can be used to form a shadow frame around camera sensor 110, further concealing it from view. In various embodiments, reflector 112 can be configured to reflect only the polarization axis of interest, and one or more filters 111 (e.g., polarizers, analyzers, infrared pass-throughs, and / or visible blockers) can be included in front of (or integrated into) camera sensor 110. Thus, stray light incident on filter 111 is extinguished by reflector 112 and not reflected back to driver 80, further concealing camera sensor 110 from detection. Stray light can be generated by direct exposure to sunlight, sunlight reflection, other lights around vehicle 90 (e.g., streetlights), lights inside vehicle 90 (e.g., roof lights), headlights of other vehicles, etc.

[0042] Reflector 112 serves as a reflective surface or object. Reflector 112 may have a transmittance that allows ambient light 86 entering the vehicle 90 to pass through the windshield 92 and reach the driver 80 at a perceptible amplitude. In some embodiments, reflector 112 may reflect visible light at all polarization axes. In other embodiments, reflector 112 may reflect light only on polarization axes aligned with the polarization axis of filter 111. In other embodiments, the reflective surface redirects light based on wavelength. For example, certain primary colors may be highly reflective, while other colors may not be reflected to the same extent as the primary colors.

[0043] Reflector 112 may be part of windshield 92, attached to a surface of windshield 92 (e.g., an inner surface, intermediate surface, or outer surface), or it may be a separate item from windshield 92. In the implementation where reflector 112 is separate from windshield 92, the reflective surface of reflector 112 may be flat or geometrically shaped (e.g., curved). The curved shape of the reflective surface typically distorts scene 142 in the reflected optical image 126.

[0044] In various embodiments, the reflective surface may have low reflectivity, allowing mirroring techniques (whether films, deposited coatings, or others) to be applied uniformly to the windshield 92 without affecting the transmittance of light through the windshield 92. When a low-reflectivity, high-transmittance coating is applied to the windshield 92, an opaque material 146 may also be applied to the surface of the windshield 92 behind the reflective surface to create a "black mirror" effect. In other words, the reflective surface redirects light, but the background appears black to an observer (e.g., driver 80) instead of the usual silver or white. The opaque material 146 may be positioned so that it is visible to driver 80 but not so high as to obstruct the driver's view of the road 70 and other visible objects outside the vehicle 90. In some embodiments, the reflector 112 and the opaque material 146 may be laterally offset from driver 80 (e.g., offset to the driver's left or right) so that driver 80 can see the road 70 when looking straight ahead without being obstructed by the reflector 112 or the opaque material 146. Additional details of embodiments of reflector 112 and / or opaque material 146 can be found in co-pending U.S. Provisional Application No. 63 / 682,782, filed August 13, 2024, which is hereby incorporated herein by reference in its entirety.

[0045] The first electronic control unit 118 implements one or more image processing circuits 148 and one or more display driver circuits. The first electronic control unit 118 typically operates to generate control signals 130 that drive the display subsystem 120. In various embodiments, the control signal 130 may be configured to provide instrumentation (e.g., speed, tachometer, fuel, temperature, etc.) to at least one of the display subsystems 120. In some embodiments, the control signal 130 may also be configured to provide video (e.g., rearview camera video, frontview camera video, in-vehicle DVD player, etc.) to the display subsystem 120. In other embodiments, the control signal 130 may be further configured to provide alphanumeric information displayed on the display subsystem 120. The first electronic control unit 118 may also implement image de-distortion techniques that correct the reflective optical image 126 to address the curvature of the windshield 92 and / or reflector 112 in cases where the reflective surface is uneven. Other image processing techniques may be implemented to meet the design guidelines for specific applications.

[0046] The second electronic control unit 104 implements the driver monitoring system 150. The driver monitoring system 150 typically operates to monitor one or more aspects of the driver 80 based on the posture of the driver 80's head 82, the direction of the driver 80's gaze, the opening / closing state of the driver 80's eyelids, the movement of the driver 80's mouth, etc. The driver monitoring system 150 may generate warning signals (e.g., physical, optical, acoustic, and / or hepatic) when it determines that the driver 80 is inattentive and not driving cautiously.

[0047] In various embodiments, the first electronic control unit 118 and the second electronic control unit 104 typically include at least one microcontroller. The at least one microcontroller may include one or more processors, each of which may be embodied as a separate processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a dedicated electronic circuit system. The at least one microcontroller may be any kind of electronic processor (implemented in hardware, software running on hardware, or a combination of both). The at least one microcontroller may also include tangible, non-transitory memory (e.g., read-only memory in the form of optical, magnetic, and / or flash memory). For example, the at least one microcontroller may include accompanying hardware in the form of a suitable amount of random access memory, read-only memory, flash memory, and other types of electrically erasable programmable read-only memory, as well as high-speed clocks or timers, analog-to-digital and digital-to-analog circuit systems and input / output circuit systems and devices, and appropriate signal conditioning and buffering circuit systems.

[0048] The computer-readable and executable instructions embodying this method may be stored in memory and executed as described herein. The executable instructions may be a series of instructions for running an application on at least one microcontroller (in the foreground or background). At least one microcontroller may receive commands and information in the form of one or more input signals from various controls or components in vehicle 90 and transmit instructions to display subsystem 120 via control signal 130.

[0049] Display subsystem 120 is typically mounted to instrument panel 94. Display subsystem 120 operates to generate a visible image 132 in response to a control signal 130 received from first electronic control unit 118. Visible image 132 is directed at driver 80. In some embodiments, display subsystem 120 may be a cluster display positioned for use by driver 80. Display subsystem 120 may be or include a console display positioned for use by driver 80 and passengers and / or a passenger display positioned for use by passengers.

[0050] Illuminator 114 operates to generate illumination light (e.g., infrared light 124) that is directed at and illuminates scene 142 / driver 80. Control of illuminator 114 is provided by a first electronic control unit 118 via control signal 130. In various embodiments, illuminator 114 may be disposed within cavity 138 of instrument panel 94 and / or on instrument panel 94 adjacent to display subsystem 120 (as shown). In various embodiments, illuminator 114 may be located adjacent to camera sensor 110 and oriented such that the illuminating infrared light 124 is pointed upward parallel to the camera axis. Infrared light 124 is then redirected by reflector 112 to illuminate scene 142 / driver 80. Other locations of illuminator 114 may be implemented to meet design guidelines for specific applications.

[0051] An ambient light sensor 116 may be located below the lower line of sight 136 and communicate with the first electronic control unit 118. In some cases, the ambient light sensor 116 may be disposed in a cavity 138 of the instrument panel 94 and adjacent to the camera sensor 110. The ambient light sensor 116 operates to measure ambient light 84 reflected by reflector 112 from the interior of the vehicle 90 (e.g., scene 142 / driver 80) toward the camera sensor 110. The ambient light level may be transmitted to the first electronic control unit 118 in a light sensor signal 117. As scene 142 changes from bright ambient conditions to near-complete darkness, the first electronic control unit 118 may use the ambient light level detected by the ambient light sensor 116 to adjust the driver monitoring system. In some cases, the first electronic control unit 118 may use the detected ambient light level to adjust the exposure settings of the camera sensor 110, where the camera sensor 110 is not built into the system.

[0052] Figure 3 A flowchart illustrating a method of operation according to one or more exemplary embodiments is shown. Method 160 (or process) typically includes steps 162 to 168, as illustrated. Other sequence of steps may be implemented to meet design guidelines for a particular application.

[0053] In step 162, reflector 112 is positioned above the lower line of sight 136, guiding the field of view 140 of camera sensor 110 to include the head 82 of driver 80 and / or scene 142. In step 164, camera sensor 110 operates to generate a sensed video signal 128 of scene 142 within the field of view 140 of camera sensor 110. Camera sensor 110 is positioned below the lower line of sight 136 of driver 80 such that camera sensor 110 is hidden from driver 80 by instrument panel 94. Instrument panel 94 defines the lower line of sight 136 of driver 80.

[0054] In step 166, the sensed video signal 128 is processed (e.g., dedistorted, decompressed, color corrected, filtered, etc.). In step 168, the first electronic control unit 118 monitors the driver 80 in response to the head 82 of the driver 80 in the sensed video signal 128.

[0055] Those skilled in the art will recognize that terms such as “above,” “below,” “front,” “back,” “upward,” “downward,” “top,” “bottom,” etc., are used descriptively herein without implying any limitation on the scope of this disclosure. Furthermore, this teaching may describe functional and / or logical block components and / or various processing steps. Such block components may consist of various hardware components, software components executing on hardware, and / or firmware components executing on hardware.

[0056] The foregoing detailed description and accompanying drawings support and illustrate this disclosure, but the scope of this disclosure is defined only by the claims. As will be understood by those skilled in the art, various alternative designs and embodiments may exist to practice the disclosure as defined in the appended claims.

Claims

1. A hidden camera subsystem for monitoring vehicle drivers, comprising: An instrument panel, wherein the instrument panel is disposed in the vehicle, and wherein the instrument panel establishes the driver's downward line of sight; A camera sensor having a field of view and positioned below the downward line of sight such that the camera sensor is hidden from the driver by the dashboard, wherein the camera sensor operates to generate a sensed video signal of the scene within the field of view; A reflector, the reflector being positioned above the downward line of sight and oriented to guide the field of view of the camera sensor to include the head of the driver in the scene; as well as A first electronic control unit operates to receive the sensed video signal and monitor the driver in response to the driver's head movement within the sensed video signal.

2. The hidden camera subsystem according to claim 1, wherein: The reflector reflects light at multiple polarization axes.

3. The hidden camera subsystem according to claim 1, wherein: The filter in front of the camera sensor has a specific polarization axis; and The reflector reflects light to match the specific polarization axis.

4. The hidden camera subsystem according to claim 1, wherein: The reflector is part of the windshield of the vehicle.

5. The hidden camera subsystem according to claim 1, wherein: The reflector is separate from the windshield of the vehicle.

6. The hidden camera subsystem according to claim 1, wherein: The reflector has a curved shape.

7. The hidden camera subsystem according to claim 6, wherein: The scene in the sensed video signal is distorted by the curved shape of the reflector; and The first electronic control unit further operates to dedistort the scene in the sensed video signal.

8. The hidden camera subsystem according to claim 1, wherein: The reflector has a transmittance that allows ambient light entering the vehicle to pass through the windshield and reach the driver at a perceptible amplitude.

9. The hidden camera subsystem according to claim 1, further comprising: An opaque material is disposed between the reflector and the windshield of the vehicle.

10. The hidden camera subsystem according to claim 9, wherein: The reflector and the opaque material are laterally offset from the driver, so that the driver can see the road where the vehicle is located when looking straight ahead.

11. The hidden camera subsystem according to claim 1, wherein: The light rays intersecting the driver's eye movement range and the vehicle's windshield are a mirror angle of the camera sensor's axis.

12. The hidden camera subsystem according to claim 1, wherein: The instrument panel has a cavity, and the camera sensor is disposed in the cavity.

13. The hidden camera subsystem according to claim 1, wherein: The first electronic control unit operates to generate output data in one or more control signals based on the sensed video signal.

14. The hidden camera subsystem according to claim 13, further comprising: A display subsystem communicates with and operates in response to the output data received in one or more control signals to generate a plurality of visible images.

15. The hidden camera subsystem according to claim 1, further comprising: One or more illuminators that communicate with and operate the first electronic control unit to generate illumination light, wherein the illumination light is directed onto the scene.

16. The hidden camera subsystem according to claim 1, further comprising: An ambient light sensor communicates with and operates in conjunction with the first electronic control unit to measure the ambient light level of the scene.

17. A means of transportation, comprising: Seat for the driver; An instrument panel is positioned in front of the seat, wherein the instrument panel establishes the driver's downward line of sight in the seat; A camera sensor having a field of view and positioned below the downward line of sight such that the camera sensor is hidden from the driver by the dashboard, wherein the camera sensor operates to generate a sensed video signal of the scene within the field of view; A reflector, the reflector being positioned above the downward line of sight and oriented to guide the field of view of the camera sensor to include the head of the driver in the scene; as well as A first electronic control unit operates to receive the sensed video signal and monitor the driver in response to the driver's head movement within the sensed video signal.

18. The means of transport according to claim 17, wherein: The first electronic control unit operates to generate output data in one or more control signals based on the sensed video signal.

19. The means of transport according to claim 18, further comprising: A display subsystem communicates with and operates in response to the output data received in one or more control signals to generate a plurality of visible images.

20. The means of transport according to claim 17, further comprising: A second electronic control unit operates to determine one or more conditions of the driver based on the sensed video signals.

Images