System and method for estimating natural daylight using an image sensor

Abstract

A rearview assembly for a vehicle is provided including an electro-optic mirror element having variable reflectivity; an image sensor to capture images of a rearward scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels; and a controller configured to determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR, Green+IR, and Blue+IR channels; determine the relative ratios of the amounts of Red, Green, Blue, and IR light; determine an amount of natural daylight detected by the image sensor based on the ratios; and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

Description

SYSTEM AND METHOD FOR ESTIMATING NATURAL DAYLIGHT USING AN IMAGE SENSORCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application No. 63 / 613,154, filed on December 21, 2023, entitled "SYSTEM AND METHOD FOR ESTIMATING NATURAL DAYLIGHT USING AN IMAGE SENSOR," by Keith W. Bigoness, the entire disclosure of which is incorporated herein by reference.TECHNOLOGICAL FIELD

[0002] The present invention relates to a system and method for estimating ambient light exterior to a vehicle and, more particularly, relates to a system and method for estimating ambient light using an image sensor of a vehicle.SUMMARY OF THE INVENTION

[0003] It is one aspect of the present disclosure to provide a rearview mirror assembly for a vehicle, the rearview mirror assembly comprising: an electro-optic mirror element disposed to provide a driver of the vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity; an image sensor associated with the vehicle to capture images of a rearward scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein, upon capturing the rearward scene, the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel; and a controller communicatively connected to the electro-optic mirror element and the image sensor. The controller configured to determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel; determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor; determine an amount of natural daylight detected by the image sensor based on the ratios, and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

[0004] According to another aspect of the invention, a method is provided for determining an amount of natural daylight using an image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel. The method comprising the steps of determining the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel, determining the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor, and determining an amount of natural daylight detected by the image sensor based on the ratios.

[0005] According to another aspect of the invention, a rearview mirror assembly is provided for a vehicle, the vehicle including an image sensor associated with the vehicle to capture images of an exterior scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein, upon capturing the exterior scene, the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel, the rearview mirror assembly comprising: an electro-optic mirror element disposed to provide a driver of the vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity; and a controller communicatively connected to the electro-optic mirror element and the image sensor, the controller configured to: determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel, determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor, determine an amount of natural daylight detected by the image sensor based on the ratios, and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

[0006] These and other features, advantages, and objects of the present device will be further understood and appreciated by those skilled in the art upon studying the following specification, claims, and appended drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The embodiments will now be described with reference to the following drawings, in which:

[0008] FIG. 1 is a projected view of an interior rearview mirror assembly of a vehicle;

[0009] FIG. 2 is an electrical circuit diagram in block form illustrating a driver identification / monitoring system implemented in the rearview mirror assembly shown in FIG. 1; and

[0010] FIG. 3 is a flow chart of a method that may be performed using the circuit shown in FIG. 2.

[0011] The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles described herein.DETAILED DESCRIPTION OF EMBODIMENTS

[0012] For purposes of description herein, the terms "upper," "lower," "right," "left," "rear," "front," "vertical," "horizontal," and derivatives thereof shall relate to the rearview assembly as oriented in FIG. 1. Unless stated otherwise, the term "front" shall refer to the surface of the element closer to an intended viewer of the rearview mirror assembly, and the term "rear" shall refer to the surface of the element further from the intended viewer of the rearview mirror assembly. However, it is to be understood that the invention may assume various alternative orientations, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings and described in the following specification are simply exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific dimensions and other physical characteristics relating to the embodiments disclosed herein are not to be considered as limiting, unless the claims expressly state otherwise.

[0013] The terms "including," "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "comprises a . . ." does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0014] As defined herein, "approximately" and "about," when used in reference to angles, proportions, and the like, may, in some embodiments, mean within plus or minus ten percent of the stated value. In other embodiments, "approximately" and "about," when used in reference to angles, proportions, and the like, may mean within plus or minus five percent of the stated value. In further embodiments, "approximately" and "about," when used in reference to angles, proportions, and the like, may mean within plus or minus three percent of the stated value. In yet other embodiments, "approximately" and "about," when used with reference to angles, proportions, and the like, may mean within plus or minus one percent of the stated value.

[0015] It is known to provide dedicated discrete light sensors for sensing ambient light levels exterior to a vehicle. Such sensors are often directed generally forward to capture light levels of the forward scene as would be seen by a driver of the vehicle. The sensed ambient light level is typically used for controlling vehicle headlamps to automatically switch from a daytime mode to a nighttime mode and / or to control an electro-optic element of a rearview mirror assembly or multiple electro-optic elements if used in both interior and exterior rearview mirror assemblies. In general, an ambient light level is used to control the electro-optic element of a rearview mirror assembly by comparing a time- averaged sensed ambient light level forward of the vehicle to the glare light level sensed from the rear of the vehicle using a rearward facing glare light sensor. In this way, the reflectivity of the electro-optic mirror may be decreased so that the glare reflected to the driver due to light from trailing vehicle headlamps is reduced so as to not significantly exceed the detected ambient light. Examples of rearview mirror assemblies having such dedicated light sensors is disclosed in commonly-assigned U.S. Patent No. 6,831,268, the entire disclosure of which is incorporated herein by reference.

[0016] It is also known to provide driver identification and driver monitoring systems in vehicles. One way to implement such systems is to provide an image sensor in the rearview mirror assembly. The image sensor may be located in the bezel adjacent the electro-optic mirror element or behind a transflective (partially reflective, partially transmissive) coating of the electro-optic mirror element. Some image sensors sense both visible light and infrared light.

[0017] Referring to FIGS. 1 and 2, the disclosure may provide for a rearview mirror assembly 12 including a mirror element 14 that may be an electro-optic mirror element 34disposed to provide a driver of the vehicle with a view rearward relative to the vehicle. The electro-optic mirror element 34 has variable reflectivity. The electro-optic element 34 may comprise an electrochromic (EC) mirror element or a liquid crystal mirror element. In this configuration, the electro-optic element 34 may vary in reflectivity in response to a control signal from a controller 50. The control signal may change an electrical potential supplied to the electro-optic element 34 to control the reflectivity. The controller 50 selects the electrical potential to supply to the electro-optic element 34 based at least in part on a determined amount of natural daylight. The controller 50 may select the electrical potential to supply to the electro-optic element 34 also based on a glare light level received from a rearward facing glare sensor 52, which may also be disposed in the rearview mirror assembly 12. Alternatively, the glare light from the rear of the vehicle may be sensed using a camera as disclosed in commonly-assigned U.S. Patent No. 11,027,657, the entire disclosure of which is incorporated herein by reference. Thus, the image sensor 16 may be used to sense the glare light or a separate image sensor may be used. The manner in which the controller 50 determines the amount of natural daylight is described further below.

[0018] The controller 50 may use the detected amount of natural daylight to control the reflectivity of the electro-optic element 34 by first determining if the amount of natural daylight falls below a threshold level and then adjusting the reflectivity based on the glare light from the rear of the vehicle. If the amount of natural daylight does not exceed the threshold level, then the controller 50 may be configured to not adjust the reflectivity of the electro-optic element 34. If the image sensor 16 is used to detect both the natural daylight and any glare light, the controller 50 may distinguish between light from headlights of vehicles and the natural daylight by either determining that the light from headlights of vehicles are point light sources and thereby excluding those light levels from the computation of natural daylight, or by determining that the relative ratios of the amounts of Red, Green, Blue, and IR light corresponding to the headlights are not natural daylight and thereby excluding those light levels from the computation of natural daylight.

[0019] There may be circumstances where the controller 50 may be configured to adjust the reflectivity of the electro-optic element 34 when the amount of natural daylight exceeds the threshold level, such as when the sun is rising or setting behind the vehicle. Because such direct sunlight reflecting from the mirror(s) may cause annoying glare to the driver, the electro-optic element 34 may be dimmed to reduce the glare. In this case, thecontroller 50 may detect the brightness and / or color temperature of the natural daylight so as to distinguish between a bright sunny day with the sun high overhead versus a sunrise or sunset behind the vehicle.

[0020] An image sensor 16 is provided that is associated with the vehicle to capture images of a rearward scene. The image sensor 16 may be an RGB-IR image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light. Upon capturing the rearward scene, the image sensor 16 detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel. The image sensor 16 may be disposed in the rearview mirror assembly 12 or elsewhere in the vehicle and may be located in the bezel adjacent the electro-optic mirror element 34 or proximate a rear surface of the electrooptic assembly. The image sensor 16 may correspond to, for example, a digital charge- coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) active pixel sensor, although not be limited to these exemplary devices.

[0021] The controller 50 is communicatively connected to the electro-optic element 34 and the image sensor 16. When the image sensor 16 is located inside the vehicle, during daytime, the ambient lighting shines through the windows of the vehicle into the field of view of image sensor 16. Because the sun has a specific emission waveform, it can be distinguished from artificial lighting (such as headlamps, tail lamps and streetlights) by comparing the ratio of IR response to the color response. Adaptive algorithms and assumptions regarding the color content of the vehicle may be input to the controller 50 or be calculated. The controller 50 may thus be configured to: determine the amount of each of Red, Green, and Blue light detected by the image sensor 16 by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel; determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor 16; determine an amount of natural daylight detected by the image sensor 16 based on the ratios; and determine a reflectivity of the electro-optic mirror element 34 based, at least in part, on the determined amount of natural daylight.

[0022] By using the image sensor 16 to detect ambient light levels (the amount of natural daylight), the need for a dedicated ambient light sensor can be avoided along with its added cost and complexity. Further, as described below, the image sensor 16 may be usedfor other functions such as a driver identification and / or monitoring system 10. In addition, the present embodiments allow characterization and prediction of the lighting present in the vehicle.

[0023] To avoid momentary changes in ambient lighting from impacting the determined amount of natural daylight, the values of the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are time-averaged values. Further, the values of the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel may be taken from only those pixels that are with a sub-region of the array of pixels. For example, those pixels that image an upper portion of a rearward scene, such as that portion above the horizon.

[0024] A method 100 of determining an amount of natural daylight is illustrated in the flowchart of FIG. 3. This method 100 may be performed by the controller 50 or other structures. The method 100 begins with sensing values of the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel (step 102). Then, the value of the IR only channel is subtracted from the Red+IR channel, the Green+IR channel, and the Blue+IR channel to yield Red (R), Green (G), and Blue (B) values (step 104). Next, ratios of the R, G, B to IR values are determined (step 106). Last, the amount of natural daylight is detected based on the ratios (step 108).

[0025] Although the image sensor 16 is shown and described as being located in the rearview mirror assembly 12, it is also possible that it could be located remote from the rearview mirror assembly such as at the rear of the vehicle and also function as a back-up assist camera or a full-time display mirror (FDM) camera.

[0026] Although the control of the electro-optic mirror element 34 is described above, the controller 50 may additionally, or alternatively, use the determined amount of natural daylight to generate signals that control the brightness of at least one of the following: the vehicle's headlamps (via a headlamp controller 62), a display 38, and the instrument panel displays (via an instrument panel display brightness controller 64). In addition, the controller 50 may generate signals that control the reflectivity of at least one exterior electro-optic mirror elements 66.

[0027] As noted above, the image sensor 16 may be part of a driver identification and / or monitoring system 10, which is described further below. The system 10 may be operable to perform an identification function. In an exemplary embodiment, the driveridentification and / or monitoring system 10 may be incorporated in the rearview mirror assembly 12 as shown in FIG. 1.

[0028] The driver identification and / or monitoring system 10 may be configured to process and / or control an identification function. The identification function may comprise an eye-scan or retinal identification function orfacial recognition. In this configuration, the driver identification and / or monitoring system 10 may provide for the interior rearview mirror assembly 12 to be configured to identify an operator or passenger of a vehicle based on the eye-scan or facial recognition identification functions. The identification function may be processed by the controller and / or communicated from the controller to one or more vehicle systems to provide for an identification of the operator or passenger of the vehicle.

[0029] The eye-scan-identification function may utilize an infrared illumination of an iris of an eye for the identification. The illumination of the eye(s) may be optimized in conditions allowing for a high optical transmittance in the near infrared (NIR) range. Accordingly, the disclosure provides for a mirror element that may have a high light transmittance in wavelengths ranging from about 800 nm to 1000 nm in the optical spectrum. Additionally, in some implementations, the electro-optic assembly may comprise a plurality of light sources 18 configured to illuminate at least one iris of the operator of the vehicle.

[0030] The image sensor 16 may be configured to be able to detect light projected from at least one light source 18 that reflects back from the illuminated scene. The at least one light source 18 may correspond to one or more infrared emitters configured to output an emission 20 of light in the NIR range. In this configuration, the controller 50 may be configured to selectively activate the one or more infrared emitters corresponding to the at least one light source 18 to illuminate the iris, such that an identity of an operator (driver) 22 of the vehicle may be determined.

[0031] The driver identification and / or monitoring system 10 may further be used for driver and / or cabin monitoring purposes as a driver monitoring system (DMS). Such a DMS may detect whether the driver 22 appears drowsy or falls asleep while driving the vehicle. The DMS may also monitor for inattentiveness and other driver states. In addition, the DMS may monitor the presence / absence of a driver.

[0032] The infrared emitters or the light sources 18 may correspond to a plurality of infrared emitter banks. Each of the infrared emitter banks may comprise a plurality of light emitting diodes, which may be grouped in a matrix or otherwise grouped and disposed behind a rear surface of the electro-optic device. In an exemplary embodiment, the plurality of light sources 18 may correspond to a first emitter bank 24 and a second emitter bank 26. The first emitter bank 24 may be configured to output the emission in the NIR range from a first side portion 28 of a front surface 30 of the mirror element 14. The second emitter bank 26 may be configured to output the emission in the NIR range from a second side portion 32 of the front surface 30 of the mirror element 14, which may comprise an electro-optic mirror element 34. In this configuration, the monitoring apparatus 10 may be configured to illuminate the eyes of the operator 22, such that the image sensor 16 may capture an image of the irises of the eyes.

[0033] The controller 50 may be configured to either not determine the amount of natural daylight when IR emitters 18 are activated or else may subtract a baseline value corresponding to the contribution of the IR emitters to the value of the IR only channel.

[0034] The image sensor 16 may be disposed on a circuit board 36, for example, a printed circuit board in communication with the controller 50. The controller 50 may further be in communication with various devices that may be incorporated in the vehicle via the communication bus 60 or any other suitable communication interface 55. The controller 50 may correspond to one or more processors or circuits, which may be configured to process image data received from the image sensor 16. In this configuration, the image data may be communicated from the image sensor 16 to the controller 50. The controller 50 may process the image data with one or more algorithms configured to determine the identity of the operator of the vehicle.

[0035] The controller 50 may further be in communication with the display 38. The display 38 may be disposed in the mirror assembly 12 behind the rear surface. The controller 50 may be operable to display the image data received from the image sensor 16, such that the operator 22 may view the image data. In this configuration, the operator 22 may adjust a position of the eyes shown on the display 38 to position the eyes such that the image data may include the necessary features required to identify the operator. In an exemplary embodiment, the features required to identify the operator of the vehicle may correspond to features of the eyes of the operator 22 (e.g., the irises).

[0036] The display 38 may correspond to a partial or a full display mirror configured to display an image data through at least a portion of the mirror assembly 12. The display 38 may be constructed utilizing various technologies, for example LCD, LED, OLED, plasma, DLP or other display technology. Examples of display assemblies that may be utilized with the present disclosure may include U.S. Patent No. 6,572,233 "Rearview Mirror With Display," U.S. Patent No. 8,237,909 entitled "Vehicular Rearview Mirror Assembly Including Integrated Backlighting for a Liquid Crystal Display (LCD)," U.S. Patent No. 8,411,245 "Multi-display Mirror System and Method for Expanded View Around a Vehicle," and U.S. Patent No. 8,339,526 "Vehicle Rearview Mirror Assembly Including a High Intensity Display," which are incorporated herein by reference in their entirety.

[0037] The driver identification and / or monitoring system 10 may further comprise an indicator 40 in the mirror assembly 12. The indicator40 may be in communication with the controller and configured to output a signal to identify a state of the scanning apparatus 10 and / or a rearview camera. The indicator may correspond to a light source that may be operable to flash and / or change colors to communicate a state of the monitoring apparatus 10. The indicator 40 may correspond to a light emitting diode (LED), and in an exemplary embodiment, the indicator 40 may correspond to a red, green, and blue (RGB) LED operable to identify the state of the scanning apparatus 10 by outputting one of more colored emissions of light.

[0038] As noted above, the mirror element 14 may be an electro-optic element 34. As also noted above, one non-limiting example of an electro-optic mirror element 34 is an electrochromic element, which includes an electrochromic medium having at least one solvent, at least one anodic material, and at least one cathodic material. Both of the anodic and cathodic materials are electroactive and at least one of them is electrochromic. It will be understood that regardless of its ordinary meaning, the term "electroactive" will be defined herein as a material that undergoes a modification in its oxidation state upon exposure to a particular electrical potential difference. Additionally, it will be understood that the term "electrochromic" will be defined herein, regardless of its ordinary meaning, as a material that exhibits a change in its extinction coefficient at one or more wavelengths upon exposure to a particular electrical potential difference. Electrochromic components, as described herein, include materials whose color or opacity are affected by electric current, such that when an electrical current is applied to the material, the color or opacitychanges from a first phase to a second phase. The electrochromic component may be a single-layer, single-phase component, multi-layer component, or multi-phase component, as described in U.S. Patent No. 5,928,572 entitled "Electrochromic Layer And Devices Comprising Same," U.S. Patent No. 5,998,617 entitled "Electrochromic Compounds," U.S. Patent No. 6,020,987 entitled "Electrochromic Medium Capable Of Producing A Preselected Color," U.S. Patent No. 6,037,471 entitled "Electrochromic Compounds," U.S. Patent No. 6,141,137 entitled "Electrochromic Media For Producing A Pre-selected Color," U.S. Patent No. 6,241,916 entitled "Electrochromic System," U.S. Patent No. 6,193,912 entitled "Near Infrared-Absorbing Electrochromic Compounds And Devices Comprising Same," U.S. Patent No. 6,249,369 entitled "Coupled Electrochromic Compounds With Photostable Dication Oxidation States," U.S. Patent No. 6,137,620 entitled "Electrochromic Media With Concentration Enhanced Stability, Process For The Preparation Thereof and Use In Electrochromic Devices," U.S. Patent No. 6,519,072, entitled "Electrochromic Device," and International Patent Application Serial Nos. PCT / US98 / 05570 entitled "Electrochromic Polymeric Solid Films, Manufacturing Electrochromic Devices Using Such Solid Films, And Processes For Making Such Solid Films And Devices," PCT / EP98 / 03862 entitled "Electrochromic Polymer System," and PCT / US98 / 05570 entitled "Electrochromic Polymeric Solid Films, Manufacturing Electrochromic Devices Using Such Solid Films, And Processes For Making Such Solid Films And Devices," which are herein incorporated by reference in their entirety.

[0039] According to one aspect of the invention, a rearview mirror assembly for a vehicle, comprises an electro-optic mirror element disposed to provide a driver of the vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity; an image sensor associated with the vehicle to capture images of a rearward scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein, upon capturing the rearward scene, the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel; and a controller communicatively connected to the electro-optic mirror element and the image sensor. The controller configured to determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel,and the Blue+IR channel; determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor; determine an amount of natural daylight detected by the image sensor based on the ratios; and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

[0040] According to various aspects, the disclosure may implement one or more of the following features or configurations in various combinations:- wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are time-averaged values of at least some of the pixels of the image sensor;- wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are received from only those pixels that are within a sub-region of the array of pixels;- wherein the sub-region is the upper portion of the rearward scene;- wherein the controller determines a reflectivity of the electro-optic mirror element based on glare light from the rear of the vehicle in addition to the determined amount of natural daylight;- wherein the controller determines the reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight by determining if the amount of natural daylight is below a threshold level and then adjusting the reflectivity based on the glare light from the rear of the vehicle;- wherein the rearview mirror assembly further comprises a glare light sensor for sensing the glare light from the rear of the vehicle;- wherein the glare light from the rear of the vehicle is sensed by the image sensor;- wherein the controller is further configured to generate signals to control the brightness of at least one display based on the determined amount of natural daylight;- wherein the controller is further configured to generate signals to control the brightness of headlamps of the vehicle based on the determined amount of natural daylight;- wherein the controller is further configured to generate signals to control the reflectivity of at least one exterior electro-optic mirror element based on the determined amount of natural daylight; and- wherein a driver monitoring system is provided comprising the rearview mirror assembly above including the image sensor and the controller, wherein the controller is further configured to monitor and / or identify the driver based on images received from the image sensor.

[0041] According to another aspect of the invention, a rearview mirror assembly for a vehicle is provided, the vehicle including an image sensor associated with the vehicle to capture images of an exterior scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein, upon capturing the exterior scene, the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel, the rearview mirror assembly comprising: an electro-optic mirror element disposed to provide a driver of the vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity; and a controller communicatively connected to the electro-optic mirror element and the image sensor, the controller configured to: determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel, determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor, determine an amount of natural daylight detected by the image sensor based on the ratios, and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

[0042] According to various aspects, the disclosure may implement one or more of the following features or configurations in various combinations:- wherein the exterior scene captured by the image sensor is a rearward scene relative to the vehicle;- wherein the image sensor is located in the rearview mirror assembly;- wherein the image sensor is located remote from the rearview mirror assembly;- wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are time-averaged values of at least some of the pixels of the image sensor;- wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are received from only those pixels that are within a sub-region of the array of pixels;- wherein the controller determines a reflectivity of the electro-optic mirror element based on glare light from the rear of the vehicle in addition to the determined amount of natural daylight;- wherein the controller determines the reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight by determining if the amount of natural daylight is below a threshold level and then adjusting the reflectivity based on the glare light from the rear of the vehicle;- wherein the rearview mirror assembly further comprises a glare light sensor for sensing the glare light from the rear of the vehicle;- wherein the glare light from the rear of the vehicle is sensed by the image sensor;- wherein the controller is further configured to generate signals to control the brightness of at least one display based on the determined amount of natural daylight;- wherein the controller is further configured to generate signals to control the brightness of headlamps of the vehicle based on the determined amount of natural daylight;- wherein the controller is further configured to generate signals to control the reflectivity of at least one exterior electro-optic mirror element based on the determined amount of natural daylight; and wherein a driver monitoring system is provided comprising the rearview mirror assembly above including the image sensor and the controller, wherein the controller is further configured to monitor and / or identify the driver based on images received from the image sensor.

[0043] According to another aspect of the invention, a method is provided for determining an amount of natural daylight using an image sensor having pixels sensitive to Red, Green,Blue (RGB) and infrared (IR) light, wherein the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel. The method comprises the steps of: determining the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel; determining the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor; and determining an amount of natural daylight detected by the image sensor based on the ratios.

[0044] According to various aspects, the disclosure may implement one or more of the following features or configurations in various combinations:- wherein the method is executed by a controller communicatively connected to the image sensor and to an electro-optic mirror element disposed to provide a driver of a vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity, and wherein the controller is further configured to determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

[0045] It will be understood by one having ordinary skill in the art that construction of the described invention and other components is not limited to any specific material. Other exemplary embodiments of the invention disclosed herein may be formed from a wide variety of materials, unless described otherwise herein.

[0046] For purposes of this disclosure, the term "coupled" (in all of its forms, couple, coupling, coupled, etc.) generally means the joining of two components (electrical or mechanical) directly or indirectly to one another. Such joining may be stationary in nature or movable in nature. Such joining may be achieved with the two components (electrical or mechanical) and any additional intermediate members being integrally formed as a single unitary body with one another or with the two components. Such joining may be permanent in nature or may be removable or releasable in nature unless otherwise stated.

[0047] It is also important to note that the construction and arrangement of the elements of the invention as shown in the exemplary embodiments is illustrative only. Although only a few embodiments of the present innovations have been described in detail in this disclosure, those skilled in the art who review this disclosure will readily appreciate thatmany modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.) without materially departing from the novel teachings and advantages of the subject matter recited. For example, elements shown as integrally formed may be constructed of multiple parts or elements shown as multiple parts may be integrally formed, the operation of the interfaces may be reversed or otherwise varied, the length or width of the structures and / or members or connector or other elements of the system may be varied, the nature or number of adjustment positions provided between the elements may be varied. It should be noted that the elements and / or assemblies of the system may be constructed from any of a wide variety of materials that provide sufficient strength or durability, in any of a wide variety of colors, textures, and combinations. Accordingly, all such modifications are intended to be included within the scope of the present innovations. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the desired and other exemplary embodiments without departing from the spirit of the present innovations.

[0048] It will be understood that any described processes or steps within described processes may be combined with other disclosed processes or steps to form structures within the scope of the present device. The exemplary structures and processes disclosed herein are for illustrative purposes and are not to be construed as limiting.

[0049] It is also to be understood that variations and modifications can be made on the aforementioned structures and methods without departing from the concepts of the present device, and further it is to be understood that such concepts are intended to be covered by the following claims unless these claims by their language expressly state otherwise.

[0050] The above description is considered that of the illustrated embodiments only. Modifications of the device will occur to those skilled in the art and to those who make or use the device. Therefore, it is understood that the embodiments shown in the drawings and described above are merely for illustrative purposes and not intended to limit the scope of the device, which is defined by the following claims as interpreted according to the principles of patent law, including the Doctrine of Equivalents.

Claims

What is claimed is:

1. A rearview mirror assembly for a vehicle, the rearview mirror assembly comprising: an electro-optic mirror element disposed to provide a driver of the vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity; an image sensor associated with the vehicle to capture images of a rearward scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein, upon capturing the rearward scene, the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel; and a controller communicatively connected to the electro-optic mirror element and the image sensor, the controller configured to: determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel, determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor, determine an amount of natural daylight detected by the image sensor based on the ratios, and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

2. The rearview mirror assembly of claim 1, wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are time- averaged values of at least some of the pixels of the image sensor.

3. The rearview mirror assembly of any one of claims 1 and 2, wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IRchannel are received from only those pixels that are within a sub-region of the array of pixels.

4. The rearview mirror assembly of any one of claims 1-3, wherein the sub-region is the upper portion of the rearward scene.

5. The rearview mirror assembly of any one of claims 1-4, wherein the controller determines a reflectivity of the electro-optic mirror element based on glare light from the rear of the vehicle in addition to the determined amount of natural daylight.

6. The rearview mirror assembly of claim 5, wherein the controller determines the reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight by determining if the amount of natural daylight is below a threshold level and then adjusting the reflectivity based on the glare light from the rear of the vehicle.

7. The rearview mirror assembly of any one of claims 5 and 6, and further comprising a glare light sensor for sensing the glare light from the rear of the vehicle.

8. The rearview mirror assembly of any one of claims 5 and 6, wherein the glare light from the rear of the vehicle is sensed by the image sensor.

9. The rearview mirror assembly of any one of claims 1-8, wherein the controller is further configured to generate signals to control the brightness of at least one of the following based on the determined amount of natural daylight: headlamps of the vehicle and at least one display.

10. The rearview mirror assembly of any one of claims 1-9, wherein the controller is further configured to generate signals to control the reflectivity of at least one exterior electro-optic mirror element based on the determined amount of natural daylight.

11. A driver monitoring system comprising: the rearview mirror assembly of any one of claims 1-10 including the image sensor and the controller, wherein the controller is further configured to monitor and / or identify the driver based on images received from the image sensor.

12. A rearview mirror assembly for a vehicle, the vehicle including an image sensor associated with the vehicle to capture images of an exterior scene, the image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein, upon capturing the exterior scene, the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel, the rearview mirror assembly comprising: an electro-optic mirror element disposed to provide a driver of the vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity; and a controller communicatively connected to the electro-optic mirror element and the image sensor, the controller configured to: determine the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel, determine the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor, determine an amount of natural daylight detected by the image sensor based on the ratios, and determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

13. The rearview mirror assembly of claim 12, wherein the exterior scene captured by the image sensor is a rearward scene relative to the vehicle.

14. The rearview mirror assembly of any one of claims 12 and 13, wherein the image sensor is located in the rearview mirror assembly.

15. The rearview mirror assembly of any one of claims 12 and 13, wherein the image sensor is located remote from the rearview mirror assembly.

16. The rearview mirror assembly of any one of claims 12-15, wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are time-averaged values of at least some of the pixels of the image sensor.

17. The rearview mirror assembly of any one of claims 12-16, wherein the values detected in the IR only channel, the Red+IR channel, the Green+IR channel, and the Blue+IR channel are received from only those pixels that are within a sub-region of the array of pixels.

18. The rearview mirror assembly of any one of claims 12-17, wherein the controller determines the reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight by determining if the amount of natural daylight is below a threshold level and then adjusting the reflectivity based on glare light from the rear of the vehicle.

19. A method for determining an amount of natural daylight using an image sensor having pixels sensitive to Red, Green, Blue (RGB) and infrared (IR) light, wherein the image sensor detects the Red+IR, Green+IR, Blue+IR and IR-only responses of the pixels that are respectively generated in a Red+IR channel, a Green+IR channel, a Blue+IR channel, and an IR only channel, the method comprising the steps of: determining the amount of each of Red, Green, and Blue light detected by the image sensor by subtracting a value detected by the IR only channel from each of the values detected in the Red+IR channel, the Green+IR channel, and the Blue+IR channel, determining the relative ratios of the amounts of Red, Green, Blue, and IR light detected by the image sensor,determining an amount of natural daylight detected by the image sensor based on the ratios.

20. The method of claim 19, wherein the method is executed by a controller communicatively connected to the image sensor and to an electro-optic mirror element disposed to provide a driver of a vehicle with a view rearward relative to the vehicle, the electro-optic mirror element having variable reflectivity, and wherein the controller is further configured to determine a reflectivity of the electro-optic mirror element based, at least in part, on the determined amount of natural daylight.

Images