Windshield-reflected infrared imaging of vehicle occupants

Abstract

An occupant monitoring system for a vehicle comprises a first light source for generating a first infrared (IR) light that illuminates at least a portion of a body of a vehicle occupant, where the first light source is time modulated with a duty cycle; an IR reflective surface for reflecting second IR light reflected from the body of the occupant; and a first IR camera for receiving at least a portion of the second IR light reflected by the IR reflective surface, where the first IR camera is time modulated with the duty cycle of the first light source.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. patent application Ser. No. 17 / 645,422, filed Dec. 21, 2021, entitled "WINDSHIELD-REFLECTED INFRARED IMAGING OF VEHICLE OCCUPANT," the disclosure of which is incorporated by reference herein in its entirety.

[0002] FIELD OF THE DISCLOSURE This disclosure relates to vehicle occupant monitoring. [Background technology]

[0003] It is becoming increasingly important to be able to reliably monitor the driver or other occupants in vehicles (e.g., cars, trucks, etc.), typically for safety purposes, such as monitoring the driver for signs of drowsiness and distraction while driving. Summary of the Invention

[0004] In one aspect, an occupant monitoring system for a vehicle comprises: a first light source for generating a first infrared (IR) light that illuminates at least a portion of a body of a vehicle occupant, where the first light source is time modulated with a duty cycle; an IR reflective surface for reflecting second IR light reflected from the body of the occupant; and a first IR camera for receiving at least a portion of the second IR light reflected by the IR reflective surface, where the first IR camera is time modulated with the duty cycle of the first light source.

[0005] Implementations may include any or all of the following features: the occupant monitoring system further comprises an occupant monitoring component that processes an output of the first IR camera; the occupant monitoring component performs distortion correction on the output; the output of the first IR camera encompasses all front row occupants of the vehicle; the first IR camera has a fisheye lens; the occupant monitoring system further comprises a second light source for generating a third IR light that illuminates at least a portion of the body of the occupant, where each of the first and third IR lights includes a dot pattern; and and a second IR camera for receiving a fourth IR light reflected by the IR reflective surface, where the occupant monitoring component processes a three-dimensional image of the occupant based on the second and fourth IR lights. At least the first IR camera is at a centered location in a transverse direction of an instrument panel of the vehicle. At least the first IR camera is at an off-center location in a transverse direction of an instrument panel of the vehicle. The off-center location is aligned with the occupant in the transverse direction of the instrument panel. The occupant is a driver of the vehicle, and the off-center location is aligned with an occupant in the transverse direction of the instrument panel. The IR reflective surface is positioned at a location on a windshield of the vehicle. The IR reflective surface includes a film or coating applied to the windshield. The film or coating is a temperature control film or coating applied to the windshield. The vehicle includes a head-up display (HUD) system, and the HUD system presents a HUD at the location on the windshield. The first light source is an IR light source. The first light source is a vertical cavity surface emitting laser (VCSEL), the VCSEL producing non-IR light along with the first IR light, the occupant monitoring system further comprising a filter for the first IR camera to remove reflections of the non-IR light. The occupant monitoring system further comprises a curved cover lens for the IR camera. At least the first IR camera is mounted on an instrument panel of the vehicle. The first IR camera is mounted in a recess in the instrument panel, the recess facing the windshield of the vehicle. The occupant monitoring system further comprises flocking or a glare trap in the recess. The vehicle has a display device mounted on the instrument panel so as to float above the instrument panel, the display device obscuring the first IR camera from view of the occupant. The duty cycle is comprised of pulses extending for less than about 5% of the duration of the duty cycle. The first IR light is polarized. The first IR light includes a pseudo-random dot pattern.The first IR camera is in or on an instrument panel of the vehicle and the first light source is located elsewhere inside the vehicle other than the instrument panel. The occupant monitoring system is configured to adjust a field of view of the first IR camera based on a height of the occupant. The IR reflective surface includes a surface of the vehicle other than a windshield. The occupant monitoring system is configured to modify an aspect of the first IR light or the IR reflective surface. The occupant monitoring system modifies an intensity of the first IR light or a color shade of the IR reflective surface. The occupant monitoring system modifies the aspect based on at least one of weather information, map information, a navigation signal, or an output of an external camera of the vehicle. [Brief description of the drawings]

[0006] [Figure 1] 1 illustrates a schematic diagram of an example of windshield reflected infrared (IR) imaging of a vehicle occupant.

[0007] [Diagram 2] 1 illustrates a schematic of an example occupant monitoring system recessed into a vehicle instrument panel of a vehicle.

[0008] [Diagram 3] 1 shows a schematic diagram of an example of performing 3D imaging using an IR camera.

[0009] [Figure 4] 1 illustrates a schematic diagram of an example of a head-up display (HUD) system that presents a HUD at a location on the windshield where IR light is also reflected.

[0010] [Diagram 5] 1 illustrates an exemplary exterior view of an occupant monitoring system.

[0011] [Figure 6] 1 illustrates an example of an occupant monitoring system having a curved lens.

[0012] [Figure 7] 1 shows an example of a passenger monitoring system.

[0013] [Figure 8] 1 shows an example of a recess with an occupant monitoring system in a vehicle instrument panel.

[0014] [Figure 9] 1 illustrates an example of a vehicle passenger compartment having an instrument panel.

[0015] [Figure 10] 1 illustrates an example of a vehicle instrument panel having a display device suspended above the instrument panel.

[0016] [Figure 11] 1 shows an example of the duty cycle of the light source and the IR camera.

[0017] [Figure 12] 1 illustrates an example architecture of a computer system.

[0018] Like reference characters or numbers in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Systems and methods (collectively, occupant monitoring systems) for monitoring vehicle occupants for one or more purposes are described herein. The present disclosure illustrates an unobtrusive occupant monitoring camera system capable of capturing clear, unobstructed images of occupants at nearly all times. An infrared (IR) camera and a reflective coating on the windscreen are combined to utilize principles of folded optics to allow images of occupants reflected off the windshield to be captured by a camera system placed in or behind the instrument panel and aimed at the windscreen. Such an approach can allow the camera system to be out of the occupant's field of view while still allowing efficient occupant monitoring, and can allow the use of existing occupant monitoring systems designed to be placed in direct (e.g., not reflected) view of the occupant. One or more aspects of the system can be designed to eliminate or reduce capture of ambient light (e.g., direct sunlight) by the system. In some implementations, high pulse intensity, short duration, gated exposure to the IR camera can ensure effective occupant monitoring over a variety of ambient conditions. The field of view gain from aiming the camera at the windshield can enable monitoring of all front row occupants of the vehicle and can eliminate or reduce image obstruction from driver and steering wheel movements.

[0020] Examples herein refer to vehicles. A vehicle is a machine that transports passengers or cargo, or both. A vehicle may have one or more motors that use at least one type of fuel, or other energy source (e.g., electricity). Examples of vehicles include, but are not limited to, cars, trucks, and buses. The number of wheels may vary between vehicle types, and one or more (e.g., all) of the wheels may be used to propel the vehicle. A vehicle may include a passenger compartment that accommodates one or more people. At least one vehicle occupant may be considered to be the driver; in this case, various tools, implements, or other devices may be provided to the driver. In examples herein, any person carried by a vehicle may be referred to as the "driver" or "passenger" of the vehicle, regardless of whether or to what extent the person drives the vehicle, or whether the person has access to all or only some of the controls to drive the vehicle, or whether the person has no controls to drive the vehicle. The terms windscreen and windshield are used interchangeably in this disclosure and refer to a transparent (e.g., flat or curved) window pane positioned at least in front of the front row passengers. For example, the windscreen / windshield may extend continuously over the front row of seats and may be part of the roof of the vehicle.

[0021] Examples herein refer to vertical cavity surface emitting lasers (VCSELs). As used herein, a VCSEL includes any semiconductor laser diode that emits a laser beam perpendicular to its top surface. A VCSEL can include two mirrors (e.g., distributed Bragg reflector mirrors) parallel to the wafer surface. For example, the mirrors can be doped as p-type and n-type materials. The wafer can form one or more quantum wells between the mirrors to generate laser light. In some implementations, a gallium arsenide wafer (e.g., including indium gallium arsenide) can be used.

[0022] Examples herein refer to IR cameras. As used herein, an IR camera includes any image sensor that detects electromagnetic radiation at least in the IR range. In some implementations, the IR camera can register images in a wavelength range of at least about 700 to 14,000 nanometers (nm). For example, the IR camera can register images in one or more of the following ranges: about 700 nm to about 800 nm; about 800 nm to about 900 nm; about 900 nm to about 1000 nm; about 1000 nm to about 1500 nm; about 1500 nm to about 2000 nm; about 2000 nm to about 3000 nm; about 3000 nm to about 10,000 nm; or about 10,000 nm to about 14,000 nm; or combinations thereof. The output of the IR camera can be an image in which pixel elements are coded according to the wavelength of radiation received.

[0023] Examples herein refer to IR reflective surfaces. As used herein, an IR reflective surface includes any surface that reflects at least a threshold amount of incident IR light. In some implementations, the threshold amount can be about 20%. In some implementations, an IR film or IR coating can be applied to the windshield to reflect about 75-80% of the IR light. The IR film / coating can be applied to the exterior surface, and / or the interior surface, and / or between layers of the windshield. The IR film / coating can include one or more layers of a metallic material. To name just one example, multiple layers including silver can be used. The IR reflective surface may be provided solely for the purpose of occupant monitoring, or may additionally function for one or more other purposes (e.g., temperature control of the passenger compartment).

[0024] 1 illustrates a schematic of an example of windshield-reflected IR imaging of an occupant of a vehicle 100. The vehicle 100 or components thereof may be used with one or more other examples described elsewhere herein. Only a portion of the vehicle 100 is shown here in partial cross-section. An occupant 102 is positioned in a passenger compartment of the vehicle 100. In some implementations, the occupant 102 is in a front row seat. For example, the occupant 102 may be the driver of the vehicle 100.

[0025] The vehicle 100 includes an instrument panel 104 and a windshield 106. An IR reflective surface 108 is positioned at a location on the windshield 106. For example, the location is normal to the line of sight for the occupant 102. The IR reflective surface 108 may comprise a film applied to the windshield 106. For example, the film may be a temperature control film applied to the entire windshield 106.

[0026] The vehicle 100 includes an occupant monitoring system 110 that can operate by detecting reflected IR light from an occupant 102. The IR light can be generated by a light source (not shown). For example, the light source can generate only IR light or can generate both IR and non-IR light. The occupant monitoring system 110 includes an IR camera 112 that receives at least a portion of the IR light reflected by the IR reflecting surface 108. For example, the IR camera 112 can be provided with a filter 114 that is transparent to IR light. The occupant monitoring system 110 can be configured in one or more ways, including but not limited to those exemplified below with reference to other descriptions, to reduce or eliminate the effects of ambient light on the occupant monitoring performed by the IR camera 112.

[0027] 2 illustrates a schematic of an example occupant monitoring system 200 installed in a recess 202 of a vehicle instrument panel 204 of a vehicle 206. The occupant monitoring system 200 or components thereof may be used with one or more other examples described elsewhere herein. The vehicle 206 includes a windshield 208. The recess 202 faces toward the windshield 208. The IR reflective surface 210 is positioned at a location on the windshield 208. For example, the location is normal to the line of sight for an occupant of the vehicle 206. The IR reflective surface 210 may comprise a film applied to the windshield 208. For example, the film may be a temperature control film applied to the entire windshield 208.

[0028] Passenger monitoring system 200 can operate by detecting reflected IR light from a passenger. The IR light can be generated by a light source in passenger monitoring system 200. For example, the light source can generate only IR light or can generate both IR and non-IR light. Passenger monitoring system 200 includes an IR camera 212 that receives at least a portion of the IR light reflected by IR reflecting surface 210. For example, IR camera 212 can be provided with a filter 214 that is transparent to IR light.

[0029] In operation, the light source may generate IR light 216, shown here diagrammatically using dotted lines. For example, the IR light 216 may reflect off the IR reflective surface 210 to illuminate at least a portion of an occupant's body. IR light 218 arriving from the occupant (e.g., from the person's face or other part of the body) may be incident on the IR reflective surface 210. IR light 220 reflected off the IR reflective surface 210 may be received by the IR camera 212.

[0030] Passenger monitoring system 200 can be configured in one or more ways to reduce or eliminate the effects of ambient light on the passenger monitoring performed by IR camera 212. Such configurations can include, but are not limited to, a glare trap inside recess 202. For example, flocking may be provided on the inside of recess 202.

[0031] 3 shows a schematic diagram of an example of performing three-dimensional (3D) imaging using an IR camera. The example is described with reference to a vehicle 300, which is partially shown in a top view. The vehicle 300 or components thereof can be used with one or more other examples described elsewhere herein. The vehicle 300 has an IR-reflective surface 302. For example, the IR-reflective surface 302 can be a windshield. An occupant 304 is positioned in the vehicle 300.

[0032] Multiple IR cameras can be used in the vehicle 300. Here, two IR cameras 306 are shown. The IR cameras 306 can be positioned symmetrically with respect to the occupant 304. In some implementations, either or both of the IR cameras 306 can also include an (IR) light source. The light source can illuminate the occupant 304 with uniform or structured IR light. Here, a light cone 308 is shown that generally represents the light from the IR camera 306 that is incident on the occupant 304 as well as the IR light reflected by the occupant 304 and received by the IR camera 306. In some implementations, a dot pattern 310 of IR light can be generated on the occupant 304. For example, a pseudo-random dot pattern can be used. This can enable the occupant monitoring system to process a 3D image of the occupant 304.

[0033] 4 illustrates generally one example of a head-up display (HUD) system 400 that presents a HUD 402 at a location on a windshield where IR light 404 is also reflected. The HUD system 400 or components thereof can be used with one or more other examples described elsewhere herein. The vehicle also includes an occupant monitoring system 406 operable using IR light 404. Thus, the same location on the windshield where the HUD 402 is presented can be used to reflect the IR light 404. For example, the IR light 404 may be reflected as part of its propagation from the occupant monitoring system 406 to the occupant, or from the occupant to the occupant monitoring system 406, or both.

[0034] FIG. 5 illustrates an exemplary exterior of passenger monitoring system 500. Passenger monitoring system 500 can be used with one or more other examples described elsewhere herein. Passenger monitoring system 500 includes lens 502. For example, lens 502 represents one or more optical components (e.g., for performing focusing and / or dispersion of a light beam) that are at least partially transparent to IR light. In some implementations, a sensor can be positioned inside lens 502. For example, the sensor can include an IR sensor. Passenger monitoring system 500 includes one or more light sources 504. For example, light source 504 can generate only IR light or can generate both IR light and non-IR light.

[0035] FIG. 6 illustrates an example of a passenger monitoring system 600 having a lens 602. The passenger monitoring system 600 and / or the lens 602 can be used with one or more other examples described elsewhere herein. The lens 602 can provide optical conditioning of the IR light 604 propagating from and / or into the passenger monitoring system 600. For example, the lens 602 can be a curved lens. As another example, the lens 602 can eliminate or resist reflections generated by inlet optics. For example, this can be done to make the passenger monitoring system 600 less noticeable to the passengers of the vehicle. In some implementations, the lens 602 is a fisheye lens. This can be done to provide a wider (e.g., distorted) field of view to capture more than one passenger in the vehicle. For example, all passengers in the front row can be covered by the passenger monitoring system 600. As another example, one or more passengers in the second row can be covered.

[0036] 7 illustrates an example of a passenger monitoring system 700. Passenger monitoring system 700 or components thereof may be used with one or more other examples described elsewhere herein. Passenger monitoring system 700 may be implemented using one or more aspects described below with reference to FIG. 12. For example, the components may be implemented by one or more processors executing instructions stored on a computer-readable medium.

[0037] The passenger monitoring system 700 includes one or more light sources 702. The light sources 702 may generate only IR light or may generate both IR and non-IR light. For example, the light sources 702 may generate polarized light (e.g., an IR light source generates polarized IR light). One or more of the light sources 702 can be oriented to point directly toward the passenger (e.g., without an intermediate light reflection). One or more of the light sources 702 can be oriented to point indirectly toward the passenger (e.g., with at least one intermediate light reflection).

[0038] The passenger monitoring system 700 includes one or more IR cameras 704. The IR cameras 704 can be sensitive to IR light in one or more wavelength ranges. At least one of the IR cameras 704 can include a laser and / or a light emitting diode. In some implementations, a VCSEL can be used. For example, an overcurrent can be provided so that the VCSEL operates at a much higher efficiency than its continuous power rating, but only if the high efficiency operation occurs during a minimum amount of duty cycle. If the average power is about 0.5 W, a peak power of about 50 W can be provided if the pulses occur only about 1% of the total time. In some implementations, the IR camera 704 can be provided with a filter that is transparent to IR light.

[0039] The occupant monitoring system 700 includes an IR reflective surface 706 that reflects IR light from an occupant toward at least one of the IR cameras 704. In some implementations, the IR reflective surface 706 can include a surface that is already present in the vehicle (e.g., functions as a structural or decorative component). In some implementations, the IR reflective surface 706 can include a coating (e.g., a film or other surface treatment) that is applied for the purpose of providing IR reflection. For example, the IR reflective surface 706 can include a vehicle windshield, side window, sunroof, moonroof, or any other surface (e.g., an opaque surface) that reflects IR light.

[0040] The passenger monitoring system 700 includes a passenger monitoring component 708 that processes the output of the one or more IR cameras 704. For example, the passenger monitoring component 708 can generate an output (e.g., a reminder or an alert) and / or take corrective action in response to the results of the passenger monitoring. In some implementations, the passenger monitoring component 708 includes a pulse control component 710 that can control the output of light by the light source 702. For example, the duty cycle can include a high pulse intensity for a short duration to mitigate the effects of ambient light. In some implementations, the passenger monitoring component 708 includes an integration control component 712 that can control the integration of radiation received by the one or more IR cameras 704. For example, the integration control component 712 can provide a duty cycle of gated exposure that mitigates the effects of ambient light.

[0041] In some implementations, the occupant monitoring component 708 includes a distortion correction component 714 that can correct for non-planar IR-reflective surfaces. For example, when the IR camera 704 receives IR light reflected off a curved windshield, the distortion correction component 714 can perform image processing on the received image to remove some or all of the distortion.

[0042] In some implementations, the occupant monitoring component 708 includes a pattern control component 716 that can facilitate illuminating the occupant with non-uniform (e.g., structured) IR light. For example, the pattern control component 716 can generate a dot pattern (e.g., a pseudo-random dot pattern) with the IR light source 702.

[0043] In some implementations, the occupant monitoring component 708 includes a position control component 718 that can adjust the IR light incident on the occupant. The position control component 718 can adjust the field of view of the IR camera 704 based on the height of the occupant. For example, an image received by the IR camera 704 can indicate whether the occupant is properly positioned within the field of view, and if not, the position control component 718 can make an adjustment accordingly.

[0044] In some implementations, the occupant monitoring component 708 includes a distraction monitoring component 720 that can analyze the image output of the IR camera 704 to detect whether an occupant (e.g., the driver) appears distracted or inattentive (e.g., by directing their attention elsewhere than to the controls of the vehicle). The distraction monitoring component 720 can take into account the output of an eye tracking component 722 (e.g., analyzing the IR image to determine where the occupant's gaze is directed).

[0045] The occupant monitoring component 708 can include an authentication component 724 that can perform one or more authentications on the occupant. The authentication component 724 can monitor one or more actions performed by the occupant. In some implementations, the authentication can be performed before driving the vehicle. For example, the authentication can ensure that the occupant is authorized to drive / occupy the vehicle and / or can be used to activate or deactivate occupant-specific vehicle settings. As another example, the authentication can be performed for purchases via the vehicle's computer system. As another example, the authentication can reflect whether a particular vehicle seat is occupied (e.g., for purposes of activating or deactivating an airbag).

[0046] The passenger monitoring component 708 can change one or more aspects of the passenger monitoring. In some implementations, this aspect relates to the light source 702 and / or the IR reflective surface 706. For example, the intensity of the (IR) light or the color shade of the IR reflective surface 706 (e.g., liquid crystals in a film on a window can be controlled to increase or decrease transparency). The sun tracking component 726 can be used to track when conditions exist where IR illumination may be changed. For example, the sun tracking component 726 can access an online resource that provides local weather conditions (e.g., whether the sunlight is bright). As another example, the sun tracking component 726 can correlate the vehicle's location with a map to determine whether sunlight is likely to be incident on the IR camera 704. As another example, the sun tracking component 726 can use navigation signals (e.g., from a global positioning system and / or a global navigation satellite system) to determine whether sunlight is likely to be incident on the IR camera 704. As another example, the sun tracking component 726 can use the output of the vehicle's external camera 728 to determine whether sunlight is likely to be incident on the IR camera 704.

[0047] The occupant monitoring system 700 can be integrated with a HUD system 730, where the occupant monitoring system 700 and the HUD system 730 are shown as part of a common system. The HUD system 730 can include a vehicle controller interface 732 (e.g., for receiving information from a vehicle controller to be presented on the HUD). The HUD system 730 can include a video generation component 734 (e.g., for dynamically generating image content for the HUD). The HUD system 730 can include a projector 736 (e.g., for projecting and updating the HUD on the windshield). Such integration can provide for the HUD to be presented at a location on the windshield that also reflects IR light for the occupant monitoring system 700.

[0048] FIG. 8 illustrates an example of a recess 800 having an occupant monitoring system 802 in an instrument panel 804 of a vehicle. The recess 800 or components thereof can be used with one or more other examples described elsewhere herein. The recess 800 can face the windshield of the vehicle. The occupant monitoring system 802 can include a light source 806 and an IR camera 808. The recess 800 can be protected by a cover 810 (e.g., transparent to IR light). The recess 800 can include a glare trap 812 inside the recess. For example, flocking 814 can be provided on the recess 800. Arrow 816 indicates generally that the occupant monitoring system 802 is adjustable. For example, the height of the occupant monitoring system 802 and / or the field of view of the IR camera 808 can be adjusted.

[0049] FIG. 9 illustrates an example of a passenger compartment 900 of a vehicle having an instrument panel 902. The passenger compartment 900 or components thereof may be used with one or more other examples described elsewhere herein. The passenger compartment 900 facilitates seating for at least two passengers in the front row. The passenger monitoring system may be configured such that the IR camera covers one or more (e.g., all) of the passengers in the front row of the vehicle. Arrow 904 generally indicates that the passenger monitoring system (e.g., its IR camera) may be centrally located across the instrument panel 902 of the vehicle. Arrows 906A-906B generally indicate that the passenger monitoring system (e.g., its IR camera) may be off-centered across the instrument panel 902 of the vehicle. For example, arrow 906A may correspond to an IR camera being centered on a passenger in the left front seat. As another example, arrow 906B may correspond to the IR camera being aligned with a front right seat occupant. The off-center position of either arrow 906A or 906B may be aligned with an occupant of the vehicle other than the driver in the transverse direction of the instrument panel. For example, arrow 906B may be aligned with a front row occupant in a left hand drive vehicle, and arrow 906A may be aligned with a front row occupant in a right hand drive vehicle.

[0050] The location of the occupant monitoring system (e.g., its IR camera) can be selected to mitigate the effects of ambient light (e.g., sunlight). For example, a windshield-mounted rearview mirror 908 may provide a shaded area over a portion of the instrument panel 902, which can guide the placement of the occupant monitoring system.

[0051] The light sources of the occupant monitoring system may be positioned according to any of the arrows 904, 906A, or 906B, or at other locations in the vehicle other than the instrument panel 902 (eg, in an A-pillar of the passenger compartment 900).

[0052] The passenger compartment 900 can have one or more IR reflective surfaces. For example, the windshield can be used. As another example, the side windows, moonroof / sunroof, and / or any other opaque surface of the passenger compartment 900 can be used.

[0053] 10 illustrates an example of a vehicle instrument panel 1000 having a display device 1002 floating therefrom. The vehicle instrument panel 1000 or components thereof can be used with one or more other examples described elsewhere herein. The display device 1002 can be attached to the vehicle instrument panel 1000 such that the display device 1002 floats therefrom. In some implementations, the display device 1002 is self-contained and not embedded within the vehicle instrument panel 1000 (e.g., not positioned flush with or submerged therein). For example, the display device 1002 is positioned (e.g., freestanding) in front of and / or on the vehicle instrument panel 1000.

[0054] The occupant monitoring system 1004 may be recessed within the vehicle instrument panel 1000 (such as in a recess therein, as shown in dashed outline) or may be mounted on an exterior surface of the vehicle instrument panel 1000 (e.g., as shown in solid outline). Arrow 1006 indicates that the display device 1002 prevents the occupant monitoring system 1004 from being viewed by an occupant of the vehicle.

[0055] FIG. 11 illustrates an example 1100 of a duty cycle of a light source and an IR camera. The example 1100 can be used with one or more other examples described elsewhere herein. A graph 1102 illustrates light generation (e.g., by an IR light source) on the vertical axis as a function of time on the horizontal axis. The time can be divided into duty cycles 1104. For example, each of the duty cycles 1104 can be a fraction of a second (including, but not limited to, about 30 milliseconds (ms)). A pulse 1106 of light generation occurs only during a fraction of the duty cycle 1104 (including, but not limited to, about 1 ms thereof). In some implementations, the light source (e.g., a laser such as a VCSEL) can be rated at a continuous average power of about 0.5 W, and the pulse 1106 can have a magnitude of about 50 W.

[0056] Graph 1108 shows camera image integration on the vertical axis as a function of time on the horizontal axis. Time can be divided into duty cycles 1110. For example, each of the duty cycles 1110 can be a fraction of a second (including but not limited to about 30 ms). The duty cycles 1110 of graph 1108 can correspond to the duty cycles 1104 of graph 1102. The pulses 1112 of camera image integration occur only for a fraction of the duty cycle 1110 (including but not limited to about 1 ms thereof). In some implementations, during a 1 second period, the camera integrates for only about 30 ms, which is about 3% of the time. For example, the pulses 1106 and / or 1112 can extend for less than about 5% of the duration of the corresponding duty cycle. This can correspond to capturing about 100% of the light generated by the pulses 1106 while only integrating about 3% of the ambient light.

[0057] The examples described herein illustrate examples showing that an occupant monitoring system for a vehicle may include a first light source (e.g., light source 702 in FIG. 7) for generating a first IR light (e.g., IR light 216 in FIG. 2) that illuminates at least a portion of a body of an occupant of the vehicle. The first light source is time modulated with a duty cycle (e.g., shown in graph 1102 in FIG. 11). The occupant monitoring system may include an IR reflective surface (e.g., IR reflective surface 706 in FIG. 7) for reflecting the second IR light (e.g., IR light 218 in FIG. 2) reflected from the body of the occupant. The occupant monitoring system may include a first IR camera (e.g., IR camera 704 in FIG. 7) for receiving at least a portion of the second IR light (e.g., IR light 220 in FIG. 2) reflected by the IR reflective surface. The first IR camera is time modulated with a duty cycle of the first light source (e.g., shown in graph 1108 in FIG. 11).

[0058] FIG. 12 illustrates an example architecture of a computing device 1200 that may be used to implement aspects of the disclosure, including any of the systems, devices, and / or techniques described herein, or any other systems, devices, and / or techniques that may be utilized in various possible embodiments.

[0059] The computing device illustrated in FIG. 12 may be used to execute the operating system, application programs, and / or software modules (including software engines) described herein.

[0060] In some embodiments, the computing device 1200 includes at least one processing device 1202 (e.g., processor), such as a central processing unit (CPU). A variety of processing devices are available from a variety of manufacturers, such as Intel or Advanced Micro Devices. In this example, the computing device 1200 also includes a system memory 1204 and a system bus 1206 that couples various system components including the system memory 1204 to the processing device 1202. The system bus 1206 is one of a number of types of bus structures that may be used, including, but not limited to, a memory bus or memory controller; a peripheral bus; and a local bus using any of a variety of bus architectures.

[0061] Examples of computing devices that may be implemented using computing device 1200 include a desktop computer, a laptop computer, a tablet computer, a mobile computing device (such as a smartphone, touchpad mobile digital device or other mobile device), or other device configured to process digital instructions.

[0062] The system memory 1204 includes a read-only memory 1208 and a random access memory 1210. A basic input / output system 1212, containing the basic routines that function to transfer information within the computing device 1200, such as during start-up, may be stored in the read-only memory 1208.

[0063] In some embodiments, computing device 1200 also includes a secondary storage device 1214, such as a hard disk drive for storing digital data. The secondary storage device 1214 is connected to the system bus 1206 by a secondary storage interface 1216. The secondary storage device 1214 and its associated computer-readable media provide non-volatile, non-transitory storage of computer-readable instructions (including application programs and program modules), data structures, and other data for computing device 1200.

[0064] Although the exemplary environment described herein uses a hard disk drive as a secondary storage device, other embodiments use other types of computer-readable storage media. Examples of these other types of computer-readable storage media include magnetic cassettes, flash memory cards, solid-state drives (SSDs), digital video disks, Bernoulli cartridges, compact disk read-only memories, digital versatile disk read-only memories, random access memories, or read-only memories. Some embodiments include non-transitory media. For example, a computer program product may be tangibly embodied in a non-transitory storage medium. Additionally, such computer-readable storage media may include local storage or cloud-based storage.

[0065] A number of program modules may be stored on the secondary storage device 1214 and / or in the system memory 1204, including an operating system 1218, one or more application programs 1220, other program modules 1222 (such as the software engines described herein) and program data 1224. The computing device 1200 may utilize any suitable operating system.

[0066] In some embodiments, a user provides input to the computing device 1200 through one or more input devices 1226. Examples of input devices 1226 include a keyboard 1228, a mouse 1230, a microphone 1232 (e.g., for voice and / or other audio input), a touch sensor 1234 (such as a touchpad or touch-sensitive display), and a gesture sensor 1235 (e.g., for gesture input). In some implementations, the input devices 1226 provide presence, proximity, and / or motion-based detection. Other embodiments include other input devices 1226. The input devices may be connected to the processing device 1202 through an input / output interface 1236 coupled to the system bus 1206. These input devices 1226 may be connected by any number of input / output interfaces (such as a parallel port, a serial port, a game port, or a universal serial bus). In some possible embodiments, wireless communication between the input device(s) 1226 and the input / output interface 1236 is also possible, including infrared, BLUETOOTH® wireless technology, 802.11a / b / g / n, cellular, ultra-wideband (UWB), ZigBee®, or other radio frequency communication systems, to name just a few.

[0067] In this exemplary embodiment, a display device 1238, such as a monitor, liquid crystal display device, light emitting diode display device, projector, or touch-sensitive display device, is also connected to the system bus 1206 via an interface, such as a video adapter 1240. In addition to the display device 1238, computing device 1200 may also include various other peripheral devices (not shown), such as speakers or printers.

[0068] Computing device 1200 may be connected to one or more networks through network interface 1242. Network interface 1242 may provide wired and / or wireless communication. In some implementations, network interface 1242 may include one or more antennas for transmitting and / or receiving wireless signals. When used in a local area networking environment or a wide area networking environment (such as the Internet), network interface 1242 may include an Ethernet interface. Other possible embodiments use other communication devices. For example, some embodiments of computing device 1200 include a modem for communicating over a network.

[0069] Computing device 1200 may include at least some form of computer-readable media. Computer-readable media includes any available media that can be accessed by computing device 1200. By way of example, computer-readable media include computer-readable storage media and computer-readable communication media.

[0070] Computer-readable storage media include volatile and nonvolatile, removable and non-removable media implemented in any device configured to store information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory, or other memory technology, compact disc read-only memory, digital versatile disk or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computing device 1200.

[0071] Computer-readable communication media typically embodied computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, computer-readable communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency, infrared and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.

[0072] The computing device illustrated in FIG. 12 is also an example of a programmable electronic device that may include one or more such computing devices, where multiple computing devices are coupled together with an appropriate data communications network to collectively perform various functions, methods or operations disclosed herein.

[0073] In some implementations, the computing device 1200 may be characterized as an ADAS computer. For example, the computing device 1200 may include one or more components that may be used to process tasks arising in the field of artificial intelligence (AI). The computing device 1200 then generally includes sufficient forward power and necessary support architecture for the demands of ADAS or AI. For example, the processing device 1202 may include a multi-core architecture. As another example, the computing device 1200 may include one or more co-processors in addition to or as part of the processing device 1202. In some implementations, at least one hardware accelerator may be coupled to the system bus 1206. For example, a graphics processing unit may be used. In some implementations, the computing device 1200 may implement neural network specific hardware to process one or more ADAS tasks.

[0074] The terms "substantially" and "about" used throughout this specification are used to describe and report small variations, such as those due to processing variations. For example, they can refer to less than or equal to ±5%, such as less than or equal to ±2%, such as less than or equal to ±1%, such as less than or equal to ±0.5%, such as less than or equal to ±0.2%, such as less than or equal to ±0.1%, such as less than or equal to ±0.05%. Also, as used herein, indefinite articles such as "a" or "an" mean "at least one".

[0075] It will be understood that all combinations of the above concepts, and additional concepts discussed in more detail below (provided that such concepts are not mutually inconsistent), are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

[0076] Although several implementations have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the specification.

[0077] Furthermore, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Additionally, other operations may be provided or removed from the described flows, and other components may be added to or removed from the described systems. Accordingly, other implementations are within the scope of the following claims. While certain features of the described implementations are shown as described herein, those skilled in the art will now recognize many modifications, substitutions, changes, and equivalents. Accordingly, it should be understood that the appended claims are intended to cover all such modifications and changes that fall within the scope of these implementations. They are presented merely as examples, not as limitations, and it should be understood that various changes in form and details may be made. Any portion of the apparatus and / or methods described herein may be combined in any combination, except mutually exclusive combinations. The implementations described herein may include various combinations and / or subcombinations of the functions, components, and / or features of the different implementations described.

Claims

1. 1. An occupant monitoring system for a vehicle, comprising: a first light source for generating a first infrared (IR) light for illuminating at least a portion of a body of a vehicle occupant, wherein the first light source is time modulated with a duty cycle; an IR reflective surface for reflecting second IR light reflected from the body of the occupant; and a first IR camera for receiving at least a portion of the second IR light reflected by the IR reflective surface, wherein the first IR camera is time modulated with the duty cycle of the first light source; A passenger monitoring system.

2. The occupant monitoring system of claim 1 , further comprising an occupant monitoring component that processes an output of the first IR camera.

3. The occupant monitoring system of claim 2 , wherein the occupant monitoring component performs distortion correction on the output.

4. 3. The occupant monitoring system of claim 2, wherein the output of the first IR camera covers all front row occupants of the vehicle.

5. The occupant monitoring system of claim 4 , wherein the first IR camera has a fisheye lens.

6. a second light source for generating a third IR light for illuminating at least a portion of the body of the occupant, where each of the first IR light and the third IR light includes a dot pattern; and a second IR camera for receiving a fourth IR light reflected by the IR reflecting surface, wherein the occupant monitoring component processes a three-dimensional image of the occupant based on the second IR light and the fourth IR light. The occupant monitoring system of claim 2 further comprising:

7. The occupant monitoring system of claim 1 , wherein at least the first IR camera is located at a transversely central position of an instrument panel of the vehicle.

8. The occupant monitoring system of claim 1 , wherein at least the first IR camera is off-center in a transverse direction of an instrument panel of the vehicle.

9. 9. The occupant monitoring system of claim 8, wherein the off-center location is aligned with the occupant in the transverse direction of the instrument panel.

10. 9. The occupant monitoring system of claim 8, wherein the occupant is a driver of the vehicle and the off-center location is aligned with an occupant in the transverse direction of the instrument panel.

11. The occupant monitoring system of claim 1 , wherein the IR reflective surface is positioned at a location on a windshield of the vehicle.

12. 12. The occupant monitoring system of claim 11, wherein the IR reflective surface comprises a film or coating applied to the windshield.

13. 13. The occupant monitoring system of claim 12, wherein the film or coating is a temperature control film or coating applied to the windshield.

14. 13. The occupant monitoring system of claim 12, wherein the vehicle includes a head-up display (HUD) system, the HUD system presenting a HUD at the location on the windshield.

15. The occupant monitoring system of claim 1 , wherein the first light source is an IR light source.

16. 2. The occupant monitoring system of claim 1, wherein the first light source is a vertical cavity surface emitting laser (VCSEL), the VCSEL generating non-IR light along with the first IR light, and the occupant monitoring system further comprising a filter for the first IR camera to remove reflections of the non-IR light.

17. The occupant monitoring system of claim 1 further comprising a curved cover lens for the first IR camera.

18. The occupant monitoring system of claim 1 , wherein at least the first IR camera is mounted on an instrument panel of the vehicle.

19. 20. The occupant monitoring system of claim 18, wherein the first IR camera is recessed in the instrument panel, the recess facing toward a windshield of the vehicle.

20. 20. An occupant monitoring system as recited in claim 19, further comprising flocking or a glare trap in said recess.

21. 20. The occupant monitoring system of claim 18, wherein the vehicle includes a display device mounted to the instrument panel so as to float therefrom, the display device obscuring the first IR camera from view of the occupant.

22. 2. The occupant monitoring system of claim 1, wherein the duty cycle comprises a pulse that extends for less than about 5% of the duration of the duty cycle.

23. The occupant monitoring system of claim 1 , wherein the first IR light is polarized.

24. The occupant monitoring system of claim 1 , wherein the first IR light includes a pseudo-random dot pattern.

25. 2. The occupant monitoring system of claim 1, wherein the first IR camera is inside or on an instrument panel of the vehicle, and the first light source is positioned elsewhere inside the vehicle other than the instrument panel.

26. The occupant monitoring system of claim 1 , wherein the occupant monitoring system is configured to adjust the field of view of the first IR camera based on a height of the occupant.

27. The occupant monitoring system of claim 1 , wherein the IR reflective surface comprises a surface of the vehicle other than a windshield.

28. 28. An occupant monitoring system as claimed in any preceding claim, wherein the occupant monitoring system is configured to change aspects with respect to the first IR light or the IR reflective surface.

29. 30. The occupant monitoring system of claim 28, wherein the occupant monitoring system varies an intensity of the first IR light or a tint of the IR reflective surface.

30. 30. The occupant monitoring system of claim 28, wherein the occupant monitoring system varies the aspect based on at least one of weather information, map information, navigation signals, or an output of an external camera of the vehicle.

Images