Localized brightness detection for a vehicle

Abstract

A vehicle includes a passenger compartment, comprising a first compartment with a first environmental condition and a second compartment with a second environmental condition. At least one camera captures a first image of a first part of a region outside the vehicle and a second image of a second part of the region. The first part of the region is aligned with the first compartment, and the second part of the region is aligned with the second compartment. The vehicle's environmental control system includes at least one lighting control and one climate control. The control circuitry is configured to determine the brightness conditions of the region based on the first and second images and to control the environmental control system to adjust the environmental conditions based on these brightness conditions.

Description

AREA OF REVELATION

[0001] The present disclosure relates generally to localized brightness detection for a vehicle, and more particularly to climate and / or lighting control using imaging-based brightness tracking. BACKGROUND OF REVELATION

[0002] Conventional brightness detection can be achieved via a solar load sensor inside the vehicle. SUMMARY OF REVELATION

[0003] According to a first aspect of the present disclosure, a vehicle includes a passenger compartment having a first space and a second space. The first space has a first environmental condition and the second space has a second environmental condition. The vehicle further includes at least one camera that captures at least a first image of a first portion of a region exterior to the vehicle and that captures at least a second image of a second portion of the region exterior to the vehicle. The first portion of the region is aligned with the first space and the second portion of the region is aligned with the second space. An environmental control system of the vehicle includes at least one of lighting control and climate control for the vehicle. A control circuit is in communication with the at least one camera and the environmental control system.The control circuit is configured to determine a first brightness condition of the first portion of the region based on the at least one first image. The control circuit is further configured to determine a second brightness condition of the second portion of the region based on the at least one second image. The control circuit is further configured to control the environmental control system to adjust the first and second environmental conditions based on the first and second brightness conditions.

[0004] Embodiments of the first aspect of the present disclosure may include any one or a combination of the following features: - Adaptation to the first environmental condition is independent of adaptation to the second environmental condition; - the adaptations to the first and second environmental conditions are adaptations to a climate of the first and second space respectively; - the first and second brightness conditions each include a brightness angle and a brightness level; - the control circuit is configured to process the first and second brightness conditions in a spatial brightness model that is trained to determine a response for the environmental control system based on the first and second brightness conditions; - the at least one camera includes a first camera having a vehicle forward orientation and a second camera having a different orientation than the vehicle forward orientation, and the first camera captures the at least one first image and the second camera captures the at least one second image; - the control circuit is configured to process a plurality of first images of the at least one first image sequentially captured by the first camera, detect a cluster of dark pixels whose size increases over the plurality of first images sequentially captured, and activate headlights of the vehicle in response to the increasing size of the cluster of dark pixels; - the spatial brightness model maps the brightness conditions to the first and second room; - the response includes an adjustment of at least one of the vehicle's interior lighting and the passenger compartment climate control; - the control circuit is configured to train the spatial brightness model based on user feedback on previous responses of the environmental control system; and - further includes headlights, and wherein the response consists in controlling the headlights of the vehicle.

[0005] According to a second aspect of the present disclosure, a vehicle includes a passenger compartment having a first compartment and a second compartment. The first compartment has a first climate, and the second compartment has a second climate. The vehicle further includes at least one camera that captures at least a first image of a first portion of a region outside the vehicle aligned with the first compartment and a second portion of the region outside the vehicle aligned with the second compartment.

[0006] The vehicle further includes a climate control system. The vehicle further includes a control circuit in communication with the at least one camera and the climate control system. The control circuit is configured to determine a first brightness condition of the first portion of the region based on the images. The control circuit is further configured to determine a second brightness condition of the second portion of the region based on the images. The control circuit is further configured to control the climate control system to adjust the first climate based on the first and second brightness conditions. The control circuit is further configured to control the climate control system to adjust the second climate based on the first and second brightness conditions.

[0007] Embodiments of the second aspect of the present disclosure may include any one or a combination of the following features: - Adaptation to the first climate is independent of adaptation to the second climate; - the control circuit is configured to process the first and second brightness conditions in a spatial brightness model that is trained to determine a response for the climate control system based on the first and second brightness conditions; - the spatial brightness model maps the first and second brightness conditions to the first and second space; - the control circuit is configured to train the spatial brightness model based on user feedback on previous responses of the climate control system; and - further includes a ventilation system, and wherein the response consists in controlling the ventilation system.

[0008] According to a third aspect of the present disclosure, a vehicle includes a passenger compartment having a first space and a second space. The first space has a first climate and the second space has a second climate. The vehicle further includes at least one camera that captures at least a first image of a first portion of a region exterior to the vehicle aligned with the first space and a second portion of the region exterior to the vehicle aligned with the second space. The vehicle further includes a climate control system. The vehicle further includes a control circuit in communication with the at least one camera and the climate control system. The control circuit is configured to determine a first brightness condition of the first portion of the region based on the images.The control circuit is further configured to determine a second brightness condition of the second portion of the region based on the images. The control circuit is further configured to process the first and second brightness conditions in a spatial brightness model trained to determine a response for the climate control system based on the first and second brightness conditions. The control circuit is further configured to control the climate control system to adjust the first climate based on the response. The control circuit is further configured to control the climate control system to adjust the second climate based on the response.

[0009] Embodiments of the second aspect of the present disclosure may include any one or a combination of the following features: - Adaptation to the first climate is independent of adaptation to the second climate; and - the first and second brightness conditions each include a brightness angle and a brightness level.

[0010] These and other features, advantages, and objects of the present disclosure will be better understood and appreciated by those skilled in the art by reference to the following description, claims, and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the drawings: Fig. 1 is a plan view of a vehicle incorporating a management system of the present disclosure; Fig. 2 is a block diagram of a management system according to one aspect of the present disclosure; Fig. 3 is a flowchart of a process for adapting an environmental control system of a vehicle by a management system of the present disclosure; Fig. 4 is a flow diagram of a process for managing passive and active modes for monitoring a region outside the vehicle using a management system according to one aspect of the present disclosure; Fig. 5 is a flow diagram of a process for detecting brightness conditions and employing a spatial brightness model to control an environmental control system for the vehicle; Fig. 6 is an image taken by an imaging device on a vehicle of a tunnel which the vehicle is approaching; and Fig. 7 is an image taken by an imaging device on a vehicle of trees on both lateral sides of the vehicle. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0012] Reference will now be made in detail to the presently preferred embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to designate the same or similar parts. In the drawings, the depicted structural elements may or may not be to scale, and certain components may or may not be enlarged relative to other components for the purpose of emphasis and clarity.

[0013] For the purposes of description in this document, the terms “upper”, “lower”, “right”, “left”, “back”, “front”, “vertical”, “horizontal” and derivatives thereof refer to the concepts in their orientation in Fig. 1. However, it is to be understood that the concepts may assume various alternative orientations unless expressly stated otherwise. It is also to be understood that the specific devices and processes illustrated in the accompanying drawings and described in the following description are merely exemplary embodiments of the inventive concepts defined in the appended claims. Thus, specific dimensions and other physical characteristics associated with the embodiments disclosed herein are not to be considered limiting unless the claims expressly state otherwise.

[0014] The presently illustrated embodiments consist primarily of combinations of method steps and device components related to localized brightness detection for a vehicle. Accordingly, the device components and method steps have been represented in the drawings by conventional symbols where appropriate, showing only those specific details relevant to an understanding of the embodiments of the present disclosure, so as not to obscure the disclosure with details that would be readily apparent to one of ordinary skill in the art in light of the present description. Furthermore, like reference numerals represent like elements throughout the description and the drawings.

[0015] In the present context, the term "and / or," when used in a list of two or more elements, means that any of the listed elements may be employed individually, or any combination of two or more of the listed elements may be employed. For example, if a composition is described as containing components A, B, and / or C, the composition may contain only A; only B; only C; A and B in combination; A and C in combination; B and C in combination; or A, B, and C in combination.

[0016] As used herein, the term "about" means that amounts, sizes, formulations, parameters, and other quantities and properties are not and need not be exact, but may be approximate and / or greater or lesser, and may reflect tolerances, conversion factors, rounding off, measurement errors, and the like, as well as other factors known to those skilled in the art. When the term "about" is used to describe a value or range endpoint, the disclosure should be understood to include the specific value or range endpoint referenced. Regardless of whether a numerical value or range endpoint includes "about" in the description, the numerical value or range endpoint is intended to include two embodiments: one modified by "about" and one unmodified by "about."It is further understood that the endpoints of each of the domains are significant both relative to the other endpoint and independently of the other endpoint.

[0017] The terms "substantially," "substantially," and variations thereof, as used herein, are intended to indicate that a described characteristic is equal to or approximately equal to a value or description. For example, a "substantially planar" surface is intended to mean that a surface is flat or approximately flat. Furthermore, "substantially" is intended to mean that two values ​​are equal to or approximately equal. In some embodiments, "substantially" may refer to values ​​within about 10% of each other, such as within about 5% of each other, or within about 2% of each other.

[0018] As used herein, the terms "the," "the," "a," or "a" or "an" mean "at least one" and should not be limited to "only one" unless expressly stated otherwise. Thus, for example, a reference to "a component" includes embodiments having two or more such components, unless the context clearly indicates otherwise.

[0019] Generally with reference to Fig. 1-7, reference numeral 10 denotes a system for managing environmental control in a vehicle environment. The management system 10 is incorporated into a vehicle 12, such as an automobile, for detecting brightness conditions by monitoring illumination in a region exterior 14 of the vehicle 12. The system 10 may provide enhanced monitoring and detection techniques for detecting sunlight or other light sources illuminating the region exterior 14 of the vehicle 12 and / or an interior 16 of the vehicle 12. Light detection may be achieved via imaging devices 18, 20, 22, 24 inside or outside the vehicle 12 that communicate image data for processing by the management system 10 to determine lighting conditions and control an environmental control system 26 of the vehicle 12.By employing the imaging devices 18, 20, 22, 24 to detect brightness conditions instead of or in addition to the sun load and / or ambient light sensing in the cabin 40, interference with brightness detection, such as objects on the windshield of the vehicle 12 (e.g., windshield wipers, ice, snow, leaves, other objects), may be limited or otherwise restricted. Control may include adjustments to exterior lighting, including headlights 27 (e.g., low beams and / or high beams), interior lighting (e.g., cabin lights 72, optical transparency of the window of the vehicle 12), displays within the vehicle 12 (e.g., a head-up display (HUD), a control panel backlight, etc.)), and climate control (e.g., fan speed, heating, cooling, circulation modes, other aspects of the climate(s), etc.).In this way, the management system 10 uses images 28 of the region outside 14 of the vehicle 12 instead of or in addition to other sensors to detect lighting conditions and optimize climate and lighting responses for the vehicle 12. By utilizing the imaging devices 18, 20, 22, 24, other sensors intended for brightness detection can be omitted or used as a redundancy check for determinations made by the management system 10.

[0020] In general, the management system 10 may provide enhanced detection and response by adjusting the sampling of the images 28 captured by the imaging devices 18, 20, 22, 24 via the control circuit 38. Such adjustments are based on operating conditions of the vehicle 12 and / or the light levels or directions (e.g., angles, distributions, etc.). The management system 10 may additionally or alternatively provide enhanced detection and response by capturing images 28 of the area exterior 14 of the vehicle 12 from multiple angles (e.g., a front 30, a rear 32, and the sides 34, 36 of the vehicle 12) and localizing the environmental response inside or outside the vehicle 12 (e.g., adjusting interior lighting (e.g., lighting conditions) in one area of ​​the vehicle 12 and not in another area of ​​the vehicle 12).The management system 10 may also include an artificial intelligence algorithm that can train models based on feedback from a user or from other systems in the vehicle 12 or in surrounding vehicles. In this way, brightness thresholds can be fine-tuned over time to effect the target response of the environmental control system 26 when desired (e.g., not turning on the headlights 27 or not turning on the headlights 27 at an appropriate time).

[0021] Now with special reference to Fig. 1, the vehicle 12 is depicted as a truck having two passenger compartments 40, 42. The two passenger compartments 40, 42 include a cab 40 and a truck bed 42. The vehicle 12 includes a plurality of sensing devices for detecting conditions of the area exterior 14 of the vehicle 12. For example, the sensing devices include the imaging devices 18, 20, 22, 24, which may include cameras and / or infrared sensors (e.g., a rain sensor), as well as thermocouples (e.g., a temperature sensor 116), pressure sensors, humidity sensors, and the like. The vehicle 12 includes a plurality of windows, such as a left window 39a, a right window 39b, a rear window 39c, and a front window 39d (e.g., a windshield).

[0022] The imaging devices 18, 20, 22, 24 may include a front imaging device 18, a rear imaging device 20, a right imaging device 24, and / or a left imaging device 22, each configured to capture images 28 of the region exterior 14 of the vehicle 12. For example, the front imaging device 18 has a field of view capturing a vehicle forward view of the region exterior 14, the rear imaging device 20 has a field of view capturing a vehicle rearward view of the region exterior 14, etc. In this way, each of the imaging devices 18, 20, 22, 24 may capture a different portion of the region exterior 14 (e.g., a front portion 44, a rear portion 46, a left portion 48, and a right portion 50).In some examples, only the front imaging device 18 is used to capture the front portion 44, although it is contemplated that any combination of imaging devices may be employed to capture one or more portions of the region exterior 14 of the vehicle 12. In one non-limiting example, a rotating imaging device is employed to capture / stitch a panoramic or 360° view of the region exterior 14.

[0023] The position of the imaging devices 18, 20, 22, 24 is exemplary and not limiting. For example, while illustrated at a front end 52 of the vehicle 12, the front imaging device 18 may be located within the interior 16 of the vehicle 12 behind the front window 39d. Similarly, while the rear imaging device 20 is depicted at a rear end 54 of the vehicle 12 (e.g., a rear parking assist imaging device), the rear imaging device 20 may be located in any position configured to capture the rear portion 46 of the region exterior 14 of the vehicle 12 (e.g., within the interior 16 and aligned with the rear window 39c).The side imaging devices 22, 24 are coupled to the rearview mirror assemblies 56 in the present example, although it is contemplated that other positions for the side imaging devices 22, 24 may be utilized (e.g., a top wall 58 of the vehicle 12, doors of the vehicle 12, any position where the imaging devices 18, 20, 22, 24 face the side windows). By allowing dynamic placement of the imaging devices 18, 20, 22, 24, the management system 10 can utilize pre-installed imaging devices 18, 20, 22, 24 to detect the brightness levels or light directions of the region exterior to the vehicle 12. In this way, reorganization or physical relocation of the imaging devices 18, 20, 22, 24 can be limited to allow the software of the management system 10 to improve brightness detection.

[0024] Now with reference to Fig. 1 and Fig. 2, the environmental control system 26 may include one or both of the lighting control systems 62, 64 and a climate control system 66 to control the environmental conditions in or around the vehicle 12. For example, the lighting control systems 62, 64 include an interior lighting system 62 and an exterior lighting system 64. The exterior lighting system 64 includes the headlamps 27 of the vehicle 12, including high-beam light sources 68 and low-beam light sources 70. The interior lighting system 62 includes lights 72 in the cabin, such as dome lights, control panel lights, gauge lights (e.g., windshield lights), accent lights, dome lights, and other interior lighting mechanisms. The climate control system 66 includes a thermal control unit 74, an air conditioning unit 76, and a ventilation control unit 78, which may be controlled individually or together to adjust the environmental conditions of the cabin 40.For example, the ventilation control unit 78 may include one or more fans 80 that may be controlled by the climate control system 66 to control airflow through the cabin 40. Fan speed and circulation modes (e.g., recirculation mode, venting mode) may be selectively controlled by the climate control system 66. Further, heating or cooling of the air may be adjusted by the climate control system 66 in response to signals communicated by the control circuit 38, as discussed in more detail below.

[0025] The environmental control system 26 may provide a spatial Fig. the cabin 40 ( Fig. 1) based on a localized control over the environmental conditions of the cabin 40. For example, the spatial Fig. outline a plurality of spaces 84, 86 in the cabin 40 that can individually control the climate control system 66 and / or the interior lighting system 62. For example, a first space 84 corresponding to an operator position may have individual temperature and / or lighting control relative to a second space 86 corresponding to a passenger position other than the operator. Although depicted as two spaces 84, 86, it is contemplated that the number of spaces 84, 86 may be spatially Fig. higher than two to allow for more uniform lighting and climate control throughout the vehicle 12. For example, sunlight may illuminate and therefore heat one area of ​​the vehicle 12 more than another, while a desired temperature for the entire cabin 40 is common to all areas in the vehicle 12. In such a case, the individual control provided by the management system 10 may improve passenger comfort.

[0026] Now with special reference to Fig. 2, the control circuit 38 of the management system 10 logically interconnects the sensing devices of the vehicle 12 and the environmental control system 26 to process the image data and / or other sensor data to improve the timing of the response to the brightness detection. The control circuit 38 may include a controller 88 having a processor 90 and a memory 92. The processor 90 includes one or more digital processing devices, including, for example, a central processing unit (CPU) with one or more processor cores, a graphics processing unit (GPU), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and / or the like.In some configurations, multiple processor 90 devices are combined into a system-on-a-chip (SoC) configuration, while in other configurations, the processor 90 devices are discrete components.

[0027] Memory 92 is formed from one or more data storage devices, including, for example, magnetic or solid-state drives and random-access memory (RAM) devices that store digital data. Memory 92 contains stored program instructions, sensor data from the sensing device (e.g., image data, temperature data, humidity data, light-sheet information, etc.), and image processing instructions. For operations involving neural network training or machine learning operations, the image processing module may additionally store training data for brightness detection.

[0028] In the present example, the processor 90 includes an image processor 94 that processes the image data from the imaging devices 18, 20, 22, 24 to detect brightness levels and / or light directions or angles based on patterns in the image data, such as clusters of bright pixels or clusters of dim or dark pixels in the images 28. The image processor 94 may perform various processing tasks on the image data, including preprocessing, filtering, masking, cropping, and various enhancement techniques to improve detection and efficiency. In addition to, or as a subset of, the brightness detection, the image data may include color information, such as pixel values ​​in a red-green-blue (RGB) array, which is processed by the image processor 94 to detect brightness. Examples of how the image processing may be operated are further described with respect to Fig. 3-5 shown.

[0029] Further with reference to Fig. 2, the control circuit 38 may include an artificial intelligence engine 96 that uses machine learning processes to generate trained neural networks to optimize the detection of brightness and the position of the brightness relative to the various sides of the vehicle 12 (e.g., the front 30, the rear 32, the right and left sides 34, 36). For example, in addition to or alternatively to including neural networks for scanning the images 28 captured by the imaging devices 18, 20, 22, 24 to detect pixel patterns, the control circuit 38 may include machine learning models trained to determine threshold brightness conditions that cause responses by the environmental control system 26.For example, the brightness conditions determined by control circuitry 38 may be compared to a threshold brightness condition preprogrammed into controller 88 or actively adjusted by machine learning techniques. In some examples, manual feedback may be used to train the machine learning models to modify the threshold brightness conditions. Digital switches (e.g., a touchscreen) or physical buttons or switches (see . Fig. 1) can be manipulated by a user, such as a passenger of the vehicle 12, to reverse the response initiated by the control circuit 38 on the environmental control system 26. In this way, the machine learning models can be trained through supervised learning to update the brightness condition thresholds.

[0030] Brightness conditions may include indications of light angles (e.g., sun or other light source low in the sky vs. high in the sky) and intensities. For example, a map of the overall light distribution of the vehicle 12 may indicate brighter aspects on some of the sides of the vehicle 12 and darker aspects on other sides of the vehicle 12. Further, the position of light sources may be detected based on areas of the images 28 captured by the at least one imaging device 18, 20, 22, 24 that have higher brightness levels. By detecting the position of the light sources (e.g., the sun, oncoming vehicle headlights, streetlights, etc.), light angles may be determined by the control circuit 38 to further refine the response by the system 10.

[0031] Still referring to Fig. 2, a sampling unit 98 may be in communication with or otherwise included within the processor 90 to allow the control circuit 38 to adjust a sampling rate of the images 28 captured by one or more of the imaging devices 18, 20, 22, 24. The sampling unit 98 may operate passively or actively with the imaging devices 18, 20, 22, 24 by either selectively reading the image data or actively initiating the image capture process. In either example, the number of frames of the region exterior 14 of the vehicle 12 analyzed by the image processor 94 may be controlled by the control circuit 38 via the sampling unit 98.

[0032] Further with reference to Fig. 2, the control circuit 38 may be in communication with auxiliary systems and sensing devices other than the imaging devices 18, 20, 22, 24 to provide redundancy and / or logical checks of brightness conditions. For example, a powertrain 100 of the vehicle 12, an ignition system 102, a positioning system 104 such as a global positioning system (GPS), a vehicle-to-vehicle (V2V) communication system 106, a rain detection system 108, at least one microphone 110, a sun load sensor 112, an ambient light sensor 114, a temperature sensor 116, and / or any other sensing device or vehicle system may be communicatively coupled to the control circuit 38.In one example, a speed sensor 118 is in communication with control circuitry 38 to allow control circuitry 38 to modify detection techniques based on the speed of vehicle 12. Data from these auxiliary systems and sensing devices may be used by control circuitry 38 to verify or assist in determining brightness based on images 28, as described in more detail below.

[0033] Now with reference to Fig. 3-5, exemplary processes S300, S400 by which the management system 10 performs the adaptation to the environmental control system 26 using the hardware of Fig. 2 can be determined. With particular reference to Fig. 3, a process S300 for controlling the environmental control system 26 includes processing a first set of images of a plurality of images 28 acquired by at least one of the imaging devices 18, 20, 22, 24 of the region exterior 14 of the vehicle 12 at a first sampling rate at step S302. For example, at least one imaging device 18, 20, 22, 24 (e.g., one or more of the imaging devices 18, 20, 22, 24) may be employed to acquire images 28 of the region exterior 14 of the vehicle 12, and the sampling unit 98 may be controlled by the processor 90 to sample the images 28 at a relatively fast rate.

[0034] In some examples, the first sample rate is between 10 and 60 frames per second (fps). In some examples, the first sample rate is approximately 30 to 60 fps.

[0035] Process S300 further includes determining the brightness level of the region exterior 14 of the vehicle 12 based on processing the first set of images at S304. In one example, the first sampling rate is applied during an initialization mode in which one or more of the imaging devices 18, 20, 22, 24 capture initial conditions, including brightness conditions, temperature, humidity, etc. As described in more detail below, the initialization mode may further classify the position of the vehicle 12 as "inside" or "outside" based on the environment captured based on the image data and provide a baseline brightness condition threshold against which changes are to be compared.

[0036] Once the brightness level of the region outside 14 is determined, the control circuit 38 may process a second set of images at a second sampling rate that is different from the first sampling rate at S306. The second images may be captured at this time. In this way, the second set of images may follow the first set of images in time. For example, following the initialization mode, the control circuit 38 may enter an active mode in which the sampling rate of the sampling module is actively controlled based on one or more conditions of the region outside 14 of the vehicle 12 or operating conditions of the vehicle 12. For example, the control circuit 38 may receive an indication that the vehicle 12 is moving (e.g., in forward gear, in reverse gear, etc.) based on communication with the powertrain 100 or a speed sensor 118.) as opposed to being in the park position or not moving. In response to movement of the vehicle 12 and / or termination of the initialization mode, the control circuit 38 may decrease the sampling rate to the second sampling rate. Thus, following the initialization mode, which may provide position data of the vehicle 12 and / or other information to provide a confidence value of the brightness level of the region exterior 14 of the vehicle 12, the active mode may be operational.

[0037] In some examples, control circuitry 38 may adjust to the second sampling rate in response to detecting a rate of change in brightness level. In this example, the first set of images refers to images 28 captured during the active mode. If the rate of change in brightness level increases beyond a rate of change limit, control circuitry 38 adjusts to the second sampling rate to provide a more stable prediction of brightness conditions. For example, if vehicle 12 is moving along a road at night and streetlights periodically illuminate the road, if detected at 60 fps, the brightness level may be adjusted between a low level and a very high level based on when vehicle 12 is far from or near a streetlight.In response to this rate of change in the brightness level, the control circuit 38 may reduce the sampling rate to provide a more consistent estimate of the brightness for the purpose of initiating the response of the environmental control system 26. In some examples, the second frame rate may be between 1 and 20 fps. In some examples, the first frame rate is between 1 and 10 fps.

[0038] Further with reference to Fig. 3, at step S308, the control circuit 38 detects a shift in the brightness level based on the second set of images. For example, the shift may be an actual change in brightness conditions that suggests a response by the environmental control system 26. In the example of driving at night on roads partially illuminated by streetlights, the shift may refer to a more uniform or average brightness change, which may or may not cause a response of activating the headlights 27 of the vehicle 12. In this way, the shift is an indication to the control system to adjust the environmental control system 26.

[0039] The control circuit 38 can then control the environmental control system 26 in response to the shift in the brightness level at step S310. Accordingly, based on the detected shift in the brightness level, the environmental control system 26 can turn on the headlights 27, increase the brightness of an instrument panel or the HUD in the cabin 40, adjust a temperature of the cabin 40 by controlling the ventilation, or adjust any other parameter of the environmental conditions. Such adjustments can be made to the individual compartments 84, 86 in the vehicle 12 or to the entire cabin 40 of the vehicle 12.

[0040] Now with reference to Fig. 4, another process S400, which may be a sub-process of the first process S300 or separate from the first process S300, provides for switching between the initialization and active modes to control the headlights 27 of the vehicle 12. The process S400 includes the control circuit 38 detecting the ignition of the vehicle 12 via the ignition system 102 at step S402. At S404, an automatic light mode is detected by the control circuit 38. The automatic light mode may refer to an operating mode for the headlights 27 that allows the management system 10 to control whether the headlights 27 are activated or not based on brightness conditions. Once an engine of the vehicle 12 is turned on and the automatic light mode is enabled, initial conditions of the interior 16 and / or the region exterior 14 of the vehicle 12 are determined by the control circuit 38 at step S406.

[0041] Based on the initial conditions, the control circuit 38 may classify the region outside 14 of the vehicle 12 as "inside" or "outside" at step S408. In this way, the control circuit 38 may assign an initialization value to the brightness conditions inside a structure (e.g., under a carport or garage, in a tunnel, etc.) and further turn on the headlights 27 based on the classification that the vehicle 12 is inside at step S410. If the vehicle 12 is "outside" (e.g., not covered by a structure), the control circuit 38 classifies the brightness conditions when the vehicle 12 is outside at step S412 and assigns an initialization value to them. In the present example, two imagers (e.g.,The front imaging device 18 and the rear imaging device 20 may image the region exterior 14 of the vehicle 12 to capture images 28 indicative of brightness conditions in a forward and rearward vehicle direction. The fields of view of these imaging devices 18, 20, 22, 24 may also cover at least a portion of the side view of the vehicle 12. In the present example, if either the front or rear brightness conditions are below the brightness threshold, the control circuit 38 may deactivate the headlights 27 in response. In other examples, the control circuit 38 may combine or blend the front and rear brightness conditions to determine an overall brightness level of the region exterior 14.

[0042] While the vehicle 12 is idling, the brightness conditions are continuously processed. Once the speed of the vehicle 12 is non-zero (step S414), the sampling rate of the sampling module is set to the first sampling rate at step S416 to track brightness conditions while the vehicle 12 is maneuvering (e.g., driving). During operation of the vehicle 12, the control circuit 38 actively monitors the brightness conditions to control the activation or deactivation of the headlights 27 at step S418.

[0043] During step 418, the machine learning models may be applied and trained based on automatic changes to the headlights 27 effected by the control circuit 38 and the user's response. For example, manually turning on the headlights while the processor 90 calculates the brightness levels may change the control circuit 38 to turn on the headlights 27 by default during future events where the brightness conditions are similar or identical. Using the images 28 to detect the headlights of other vehicles may also be used to train the machine learning models. For example, a pair of clusters of pixels may be detected in the images 28, and the control circuit 38 may determine in response that other vehicles have their headlights 27 turned on. The control circuit 38 may then activate the headlights 27.

[0044] To improve power management, the control circuit 38 further dynamically adjusts the sampling rate based on whether the vehicle 12 is parked, stationary, or otherwise still in place for an extended period of time, and upon such a determination, decreases the sampling rate. For example, in step S420, the control circuit 38 classifies the stop of the vehicle 12 as a "stop-and-go" condition or a "full stop" condition (step S422). In the stop-and-go condition, such as in traffic or when stopped for a relatively short duration (seconds or minutes), the control circuit 38 continues to sample at the first sampling rate. If the vehicle 12 is stopped for more than a relatively short duration (e.g., at least 3 minutes, at least 5 minutes, at least 10 minutes, etc.), the control circuit 38 may adjust to a lower sampling rate to conserve power in step S424.Because the vehicle 12 is not moving, the brightness conditions may be tracked less actively (e.g., low sampling rates). In some examples, the gear status of the powertrain 100 being park and / or a parking brake for the vehicle 12 being engaged may also or alternatively provide an adjustment to the lower sampling rate.

[0045] At step S426, the user may be notified via a human-machine interface (HMI), such as the HUD or touchscreen control panel, that the headlights 27 should be deactivated. For example, the user may be notified to prompt the user to deactivate the headlights 27. If the vehicle 12 begins to move again (step S414), the process may repeat steps S416-S420. Otherwise, the process ends once the engine is turned off, as detected by the control circuit 38 at step S428.

[0046] The above-described adjustment of frame sampling by control circuit 38 may provide improved savings in computing power and / or electrical power. For example, a reduced sampling rate may result in less image data being processed, thus allowing other vehicle monitoring systems to process data under a common power and / or computing level.

[0047] Now with reference to Fig. 5, a process S500 for extracting brightness conditions from the images 28 acquired by one or more imaging devices 18, 20, 22, 24 and generating / training a spatial brightness model 120 may be used independently or in conjunction with the previously described sampling rate adaptation processes S300, S400. The process S500 includes capturing the images 28 from one or more angles at S502. For example, while the image in step S502 is of a field of view in a vehicle forward direction, the vehicle rearward direction and / or vehicle transverse directions may be captured.

[0048] The images 28 and metadata corresponding to the images 28 (e.g., exposure time, brightness, saturation, etc.) are then communicated to the control circuit 38, which may combine or merge the image data and metadata from multiple imaging devices 18, 20, 22, 24 to represent brightness conditions of one or more of the portions 44, 46, 48, 50 of the region exterior 14 of the vehicle 12 at step S504. For example, the control circuit 38 may process pixel values, such as red, green, and blue color values, to estimate the brightness conditions. The image processor 94 may determine lux values ​​for the images 28 and combine the lux values. For example, the Fig. The table illustrated in Figure 5 illustrates exemplary lux values ​​for localized areas of the vehicle 12 (e.g., the spaces 84, 86 of the cabin 40).

[0049] At step S506, the brightness condition(s) is (are) compared with information collected by the auxiliary sensors and systems in communication with the control circuit 38, as shown in Fig. 2. For example, if the solar load sensor 112 is present, the brightness condition(s) may be compared to solar load information collected via a photodiode or other device in the vehicle 12 to serve as a logical check against the brightness condition. For example, an ON / OFF signal from the solar load sensor 112 or the ambient light sensor 114 may be compared to the brightness condition estimated by the control circuit 38. If the brightness condition estimate is misaligned or does not match the signal from the auxiliary sensing devices, the brightness condition estimates may be modified to be lower or higher based on the mismatch.For example, if the solar load sensor 112 produces a very high light (lux) reading and the brightness condition estimate is average, the control circuit 38 may adjust the estimate higher. It is contemplated that the auxiliary sensing devices may be omitted in various examples, and only one or more of the imaging devices 18, 20, 22, 24 may be used to determine the brightness conditions.

[0050] At step S508, the control circuit 38 may compare the brightness condition estimate(s) to threshold data to determine whether a response is requested from the environmental control system 26 and, if so, what that response may be. At step S510, the control circuit 38 may communicate an instruction to adjust the environmental control system 26 based on the brightness conditions exceeding the threshold data. For example, if the brightness condition estimate is high and the headlights 27 are off, no adjustment may be initiated for the exterior lighting system 64. However, the climate control system 66 may adjust temperature controls within the cabin 40 to cool the cabin 40, and / or the interior lighting system 62 may adjust the lighting in the cabin 40 to reduce interior lighting.The responses can be localized to the spaces 84, 86 aligned with the sides 34, 36 of the vehicle 12, which have higher lux readings.

[0051] For example, the table in Fig. 5 different lux values ​​corresponding to different parts of the vehicle 12, such as the front 30, the rear 32, and the sides 34, 36. The first space 84 may be aligned with the left front and the second space 86 may be aligned with the right front of the cabin 40. Since the left side 34 has lower lux values, the control circuit 38 may control the interior lighting to provide more illumination to the left side 34 of the interior 16 and less illumination to the right side 36 of the interior 16. Additionally or alternatively, if both the first and second spaces 84, 86 are set by the climate control system 66 to maintain a common temperature, the climate control system 66 may increase the fan speed for greater cooling of the right side 36 than the left side to account for more heat (e.g., from sunlight) on the right side 36 than on the left side 34.It is contemplated that this example is not limiting. In general, by correlating the brightness conditions with areas of the cabin 40, the interior lighting and climate of the cabin 40 can be individually controlled with precision by the control circuit 38.

[0052] The table in Fig. 5 further illustrates the generation and training of the spatial brightness model 120. For example, the control circuit 38 may compare the various measurements to a brightness marker. The brightness marker may be stored in memory 92 as a predefined value generated based on the use of a ground-truth light sensor to compare it to tested measurements of lux values ​​from various parts of the region exterior 14 of the vehicle 12. For example, the spatial brightness model 120 may determine a heavier weighting for the lux values ​​of images 28 captured by the front imaging device 18 relative to the lux values ​​of images 28 captured by the rear imaging device 20. Further, the brightness marker may be dynamic and change based on user feedback.At step S512, the process captures user feedback in response to the adjustment of the environmental control system 26. For example, if the temperature controls were modified in response to the brightness conditions, the user may manually activate controls of the climate control system 66 to undo the change. Based on this feedback, the spatial brightness model 120, the brightness marker, and the threshold information may be updated. In another example, the headlights 27 are automatically activated in response to the brightness level being below a threshold brightness level, and the user may manually adjust the headlights 27 to keep the headlights 27 on. Such adjustments may provide user feedback to the control circuit 38 to update the trained models and the threshold data.

[0053] Now with reference to Fig. 6 and Fig. 7 illustrate various examples of the management system 10 and the associated processes S300, S400, S500 applied to the captured images 28, an improved control of the environmental control system 26. For example, the front imaging device 18 may capture an image of a tunnel 122 in front of the vehicle 12 ( Fig. 6). The management system 10 may provide active adjustment of the headlights 27 before the vehicle 12 enters the tunnel 122. For example, the image processor 94 may either correlate the image with other images 28 of tunnels 122 using trained models to detect the tunnel 122, or otherwise determine via pixel analysis that a dark spot on the image is getting larger as more images 28 are sampled. Since brightness levels may decrease as the vehicle 12 approaches the tunnel 122, the management system 10 may decrease or increase the sampling rate to detect that such changes in brightness levels are actually occurring and control the exterior lighting system 64 to turn on the headlights 27. In another example, a pair of oncoming lights may be detected in the image captured by the forward imaging device 18.In response, the control circuit 38 can switch off the high beam of the headlights 27 without switching off the low beam.

[0054] With reference to Fig. 7, the brightness conditions on one or both of the right and left sides 34, 36 can be detected by the management system 10 to allow localized control over the space in the cabin 40. In the example from Fig. 7, there are tall trees 124 on both sides 34, 36 of the vehicle 12, resulting in the lux level of the right and left imaging devices 22, 24 being lower than the front and rear imaging devices 18, 20. In this example, the control circuit 38 may adjust any subsystem of the environmental control system 26 to cause the vehicle 12 to have consistent lighting and heating / cooling in the interior 16.

[0055] It is contemplated that in any of the preceding examples, the position of the vehicle 12, the time of day, the temperature, the direction of travel, or any other factor related to conditions of the region external 14 of the vehicle 12 may be accessed by the control circuit 38 to further enhance the brightness condition estimation. For example, the GPS may indicate that the position of the vehicle 12 is in a region (e.g., the Northern Hemisphere) that has less sunlight than other regions. In other examples, the GPS may track the vehicle 12 along a travel route that may be predefined by the user or predicted based on the position of the vehicle 12. Thus, the travel route may refer to an expected route for the vehicle 12 to travel.The time of day may further be used to track the position of the sun relative to the direction of travel and path of the vehicle 12. In these examples, the control circuit 38 adjusts the brightness threshold information, the brightness marker, and / or the spatial brightness model 120 to provide improved estimates that limit manual adjustments. For example, if it is determined based on GPS information and time of day information that the sun is over the position of the vehicle 12 in . Fig.7, the lux values ​​of the surrounding sides 34, 36 (e.g., the left and / or right sides 34, 36) may be immaterial for the brightness estimation for the purpose of activating the headlights 27. In contrast, if the sun is aligned with the right side 36 of the vehicle 12, the direction of the light may be determined by the control circuit 38. Accordingly, in this example, the control circuit 38 may give greater weight to the lux values ​​of the front and rear imaging devices 18, 20. In this way, image processing for classifying the environment (e.g., trees, buildings, mountains) may be combined with processing to determine the light direction and magnitude.

[0056] In one example, data from one or more of the microphones 110 may be used to detect rain (e.g., the sound of rain). In response to such a detection, the control circuit 38 may turn on the headlights 27 by heavily weighting the auxiliary sensor data from the microphone 110. For example, in a neural network or other mechanism of the spatial brightness model 120, a neuron corresponding to audio data captured by the microphone 110 may be heavily weighted to cause the brightness estimate to exceed the threshold information and to cause the headlights 27 to turn on. In this way, the management system 10 and the processes of the disclosure may provide software techniques that improve the responsiveness of the exterior lighting.

[0057] In general, the management system 10 can improve brightness detection and response by using machine learning and sampling rate control, among other mechanisms. The system 10 can increase response time and more accurately predict events to adjust lighting and / or climate.

[0058] It is to be understood that variations and modifications may be made to the foregoing structure without departing from the concepts of the present disclosure, and it is further understood that such concepts are intended to be covered by the following claims unless these claims expressly state otherwise by their language.

[0059] According to the present invention, there is provided a vehicle comprising: a passenger compartment comprising a first space and a second space, the first space having a first environmental condition and the second space having a second environmental condition; at least one camera that captures at least a first image of a first portion of a region exterior to the vehicle and that captures at least a second image of a second portion of the region exterior to the vehicle, the first portion of the region being aligned with the first space and the second portion of the region being aligned with the second space; an environmental control system of the vehicle comprising at least one of lighting control and climate control for the vehicle;and a control circuit in communication with the at least one camera and the environmental control system, the control circuit configured to: determine a first brightness condition of the first portion of the region based on the at least one first image; determine a second brightness condition of the second portion of the region based on the at least one second image; and control the environmental control system to adjust the first and second environmental conditions based on the first and second brightness conditions.

[0060] According to one embodiment, adaptation to the first environmental condition is independent of adaptation to the second environmental condition.

[0061] According to one embodiment, the adaptations to the first and second environmental conditions are adaptations to a climate corresponding to the first and second spaces.

[0062] According to one embodiment, the first and second brightness conditions each include a brightness angle and a brightness level.

[0063] According to one embodiment, the control circuit is configured to: process the first and second brightness conditions in a spatial brightness model trained to determine a response for the environmental control system based on the first and second brightness conditions.

[0064] According to one embodiment, the at least one camera includes a first camera having a vehicle forward orientation and a second camera having an orientation different from the vehicle forward orientation, wherein the first camera captures the at least one first image and the second camera captures the at least one second image.

[0065] According to one embodiment, the control circuit is configured to process a plurality of first images of the at least one first image sequentially captured by the first camera; detect a cluster of dark pixels whose size increases over the plurality of first images sequentially captured; and activate headlights of the vehicle in response to the increasing size of the cluster of dark pixels.

[0066] According to one embodiment, the spatial brightness model maps the brightness conditions to the first and second spaces.

[0067] According to one embodiment, the response includes adjusting at least one of an interior lighting of the vehicle and a climate control of the passenger compartment.

[0068] According to one embodiment, the control circuit is configured to: train the spatial brightness model based on user feedback to previous responses of the environmental control system.

[0069] According to one embodiment, the invention is further characterized by: headlights, and wherein the reaction consists in controlling the headlights of the vehicle.

[0070] According to the present invention, a vehicle is provided comprising: a passenger compartment having a first space and a second space, the first space having a first climate and the second space having a second climate; at least one camera capturing images of a first portion of a region exterior to the vehicle aligned with the first space and a second portion of the region exterior to the vehicle aligned with the second space; a climate control system; and a control circuit in communication with the at least one camera and the climate control system, the control circuit configured to: determine a first brightness condition of the first portion of the region based on the images; determine a second brightness condition of the second portion of the region based on the images;Controlling the climate control system to adjust the first climate based on the first and second brightness conditions; and controlling the climate control system to adjust the second climate based on the first and second brightness conditions.

[0071] According to one embodiment, adaptation to the first climate is independent of adaptation to the second climate.

[0072] According to one embodiment, the control circuit is configured to: process the first and second brightness conditions in a spatial brightness model trained to determine a response for the climate control system based on the first and second brightness conditions.

[0073] According to one embodiment, the spatial brightness model maps the first and second brightness conditions to the first and second spaces.

[0074] According to one embodiment, the control circuit is configured to: train the spatial brightness model based on user feedback on previous responses of the climate control system.

[0075] According to one embodiment, the invention is further characterized by: a ventilation system, and wherein the response consists in controlling the ventilation system.

[0076] According to the present invention, a vehicle is provided comprising: a passenger compartment having a first space and a second space, the first space having a first climate and the second space having a second climate; at least one camera capturing images of a first portion of a region exterior to the vehicle aligned with the first space and a second portion of the region exterior to the vehicle aligned with the second space; a climate control system; and a control circuit in communication with the at least one camera and the climate control system, the control circuit configured to: determine a first brightness condition of the first portion of the region based on the images; determine a second brightness condition of the second portion of the region based on the images;Processing the first and second brightness conditions in a spatial brightness model trained to determine a response for the climate control system based on the first and second brightness conditions; controlling the climate control system to adjust the first climate based on the response; and controlling the climate control system to adjust the second climate based on the response.

[0077] According to one embodiment, adaptation to the first climate is independent of adaptation to the second climate.

[0078] According to one embodiment, the first and second brightness conditions each include a brightness angle and a brightness level.

Claims

[1] Vehicle comprising: a passenger compartment having a first space and a second space, the first space having a first environmental condition and the second space having a second environmental condition; at least one camera that captures at least a first image of a first portion of a region outside the vehicle and that captures at least a second image of a second portion of the region outside the vehicle, the first portion of the region being aligned with the first space and the second portion of the region being aligned with the second space; an environmental control system of the vehicle comprising at least one of lighting control and climate control for the vehicle; and a control circuit in communication with the at least one camera and the environmental control system, the control circuit being configured to: Determining a first brightness condition of the first part of the region based on the at least one first image; Determining a second brightness condition of the second part of the region based on the at least one second image; and Controlling the environmental control system to adjust the first and second environmental conditions based on the first and second brightness conditions. [2] The vehicle of claim 1, wherein the adaptation to the first environmental condition is independent of the adaptation to the second environmental condition. [3] The vehicle of claim 2, wherein the adjustments to the first and second environmental conditions are adjustments to a climate corresponding to the first and second spaces. [4] The vehicle of claim 1, wherein the first and second brightness conditions each include a brightness angle and a brightness level. [5] The vehicle of claim 1, wherein the control circuit is configured to: to process the first and second brightness conditions in a spatial brightness model that is trained to determine a response for the environmental control system based on the first and second brightness conditions. [6] The vehicle of claim 5, wherein the at least one camera includes a first camera having a vehicle forward orientation and a second camera having an orientation different from the vehicle forward orientation, the first camera capturing the at least one first image and the second camera capturing the at least one second image. [7] The vehicle of claim 6, wherein the control circuit is configured to: process a plurality of first images of the at least one first image sequentially captured by the first camera; to detect a cluster of dark pixels whose size increases over the plurality of initial images taken sequentially; and Activate the vehicle's headlights in response to the increasing size of the cluster of dark pixels. [8] The vehicle of claim 6, wherein the spatial brightness model maps the brightness conditions to the first and second spaces. [9] The vehicle of claim 8, wherein the response includes adjusting at least one of an interior lighting of the vehicle and a climate control of the passenger compartment. [10] A vehicle according to any one of claims 5-9, wherein the control circuit is configured to: to train the spatial brightness model based on user feedback on previous responses of the environmental control system. [11] A vehicle according to any one of claims 5-9, further comprising: Headlights, and the response consists in controlling the vehicle's headlights.

Images