Local brightness detection for vehicles

Abstract

The invention provides local brightness detection for a vehicle. A vehicle includes a cabin having a first volume and a second volume, the first volume having a first environmental condition and the second volume having a second environmental condition. At least one camera captures a first image of a first portion of an area outside the vehicle and a second image of a second portion of the area. A first portion of the region is aligned with the first space and a second portion of the region is aligned with the second space. An environmental control system of a vehicle has at least one of lighting control and climate control. The control circuit is configured to determine a brightness condition of the area based on the first image and the second image and control the environmental control system to adjust the environmental condition based on the brightness condition.

Description

Technical Field

[0001] The present disclosure relates generally to local brightness detection for vehicles, and more particularly to climate control and / or lighting control using imaging-based brightness tracking. Background Art

[0002] Conventional brightness detection can be achieved via a sunlight sensor inside the vehicle. Summary of the invention

[0003] According to a first aspect of the present disclosure, a vehicle includes a cabin having a first space and a second space. The first space has a first environmental condition, and the second space has a second environmental condition. The vehicle also includes at least one camera, the at least one camera capturing at least one first image of a first portion of an area outside the vehicle and capturing at least one second image of a second portion of an area outside the vehicle. The first portion of the area is aligned with the first space, and the second portion of the area is aligned with the second space. The environmental control system of the vehicle has at least one of lighting control and climate control for the vehicle. The control circuit communicates with the at least one camera and the environmental control system. The control circuit is configured to determine a first brightness condition of a first portion of the area based on at least one first image. The control circuit is also configured to determine a second brightness condition of a second portion of the area based on at least one second image. The control circuit is also configured to control the environmental control system to adjust the first environmental condition and the second environmental condition based on the first brightness condition and the second brightness condition.

[0004] Embodiments of the first aspect of the present disclosure may include any one or a combination of the following features:

[0005] – the adjustment of the first environmental condition is independent of the adjustment of the second environmental condition;

[0006] - the adjustment of the first environmental condition and the second environmental condition is an adjustment of the climate of the first space and the second space respectively;

[0007] - the first brightness condition and the second brightness condition each include a brightness angle and a brightness level;

[0008] - the control circuit is configured to process the first brightness condition and the second brightness condition in a spatial brightness model, the spatial brightness model being trained to determine a response to the environmental control system based on the first brightness condition and the second brightness condition;

[0009] – at least one camera includes a first camera having a vehicle-forward orientation and a second camera having an orientation different from the vehicle-forward orientation, and the first camera captures at least one first image and the second camera captures at least one second image;

[0010] – the control circuit is configured to process a plurality of first images of at least one first image sequentially captured by the first camera, detect dark pixel clusters of increasing size across the plurality of first images captured sequentially, and activate headlights of the vehicle in response to the increasing size of the dark pixel clusters;

[0011] – A spatial brightness model maps brightness conditions to a first space and a second space;

[0012] - the response includes adjustment of at least one of the vehicle's interior lighting and cabin climate control;

[0013] - the control circuit is configured to train the spatial brightness model based on user feedback of previous responses of the environmental control system; and

[0014] - also includes headlights, and wherein the response is controlling the headlights of the vehicle.

[0015] According to a second aspect of the present disclosure, a vehicle includes a cabin having a first space and a second space. The first space has a first climate, and the second space has a second climate. The vehicle also includes at least one camera that captures an image of a first portion of an area outside the vehicle aligned with the first space and an image of a second portion of an area outside the vehicle aligned with the second space. The vehicle also includes a climate control system. The vehicle also includes a control circuit that communicates with the at least one camera and the climate control system. The control circuit is configured to determine a first brightness condition of a first portion of the area based on the image. The control circuit is also configured to determine a second brightness condition of a second portion of the area based on the image. The control circuit is also configured to control the climate control system to adjust the first climate based on the first brightness condition and the second brightness condition. The control circuit is also configured to control the climate control system to adjust the second climate based on the first brightness condition and the second brightness condition.

[0016] Embodiments of the second aspect of the present disclosure may include any one or a combination of the following features:

[0017] – Adjustment of the first climate is independent of adjustment of the second climate;

[0018] - the control circuit is configured to process the first brightness condition and the second brightness condition in a spatial brightness model, the spatial brightness model being trained to determine a response to the climate control system based on the first brightness condition and the second brightness condition;

[0019] – a spatial brightness model maps the first brightness condition and the second brightness condition to the first space and the second space;

[0020] - the control circuitry is configured to train the spatial brightness model based on user feedback of previous responses of the climate control system; and

[0021] - also including a ventilation system, and wherein the response is controlling the ventilation system.

[0022] According to a third aspect of the present disclosure, a vehicle includes a cabin having a first space and a second space. The first space has a first climate, and the second space has a second climate. The vehicle also includes at least one camera that captures an image of a first portion of an area outside the vehicle aligned with the first space and an image of a second portion of an area outside the vehicle aligned with the second space. The vehicle also includes a climate control system. The vehicle also includes a control circuit that communicates with the at least one camera and the climate control system. The control circuit is configured to determine a first brightness condition of a first portion of the area based on the image. The control circuit is also configured to determine a second brightness condition of a second portion of the area based on the image. The control circuit is also configured to process the first brightness condition and the second brightness condition in a spatial brightness model that is trained to determine a response to the climate control system based on the first brightness condition and the second brightness condition. The control circuit is also configured to control the climate control system to adjust the first climate based on the response. The control circuit is also configured to control the climate control system to adjust the second climate based on the response.

[0023] Embodiments of the second aspect of the present disclosure may include any one or a combination of the following features:

[0024] – the adjustment of the first climate is independent of the adjustment of the second climate; and

[0025] - The first brightness condition and the second brightness condition each include a brightness angle and a brightness level.

[0026] These and other features, advantages and objectives of the present disclosure will be further understood and appreciated by those skilled in the art by referring to the following description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In the attached picture:

[0028] Figure 1 is a top view of a vehicle incorporating the management system of the present disclosure;

[0029] Figure 2 is a block diagram of a management system according to one aspect of the present disclosure;

[0030] Figure 3 is a flow chart of a process for adjusting an environmental control system of a vehicle via the management system of the present disclosure;

[0031] Figure 4 is a flow chart of a process for managing passive and active modes for monitoring an area outside a vehicle using a management system according to one aspect of the present disclosure;

[0032] Figure 5 is a flow chart of a process for detecting brightness conditions and employing a spatial brightness model to control an environmental control system of a vehicle;

[0033] Figure 6 is an image captured by an imaging device on the vehicle of the tunnel that the vehicle is approaching; and

[0034] Figure 7 are images captured by an imaging device on the vehicle of trees on either lateral side of the vehicle. DETAILED DESCRIPTION

[0035] Reference will now be made in detail to the present preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. In the drawings, the depicted structural elements may or may not be drawn to scale, and certain parts may or may not be enlarged relative to other parts for the purpose of emphasis and understanding.

[0036] For the purpose of this description, the terms "upper", "lower", "right", "left", "rear", "front", "vertical", "horizontal" and their derivatives shall be used in conjunction with Figure 1 The present invention relates to the concept of orientation in the present invention. However, it should be understood that the concept can present various alternative orientations unless expressly indicated to the contrary. It should also be understood that the specific devices and processes shown in the drawings and described in the following specification are merely exemplary embodiments of the inventive concepts defined in the appended claims. Therefore, unless the claims expressly state otherwise, specific dimensions and other physical characteristics related to the embodiments disclosed herein should not be considered limiting.

[0037] The embodiments shown herein exist primarily in a combination of method steps and device components related to local brightness detection for a vehicle. Therefore, device components and method steps have been indicated by conventional symbols in the drawings where appropriate, and only those specific details relevant to understanding the embodiments of the present disclosure are shown so as not to obscure the present disclosure by details that are readily apparent to those of ordinary skill in the art having benefited from the description herein. In addition, the same reference numerals in the specification and drawings represent the same elements.

[0038] As used herein, the term "and / or" when used with a list of two or more items means that any one of the listed items may be employed alone, or any combination of two or more of the listed items may be employed. For example, if a composition is described as containing components A, B, and / or C, the composition may contain: only A; only B; only C; a combination of A and B; a combination of A and C; a combination of B and C; or a combination of A, B, and C.

[0039] As used herein, the term "about" means that the amount, size, formula, parameter and other quantities and characteristics are not exact, nor do they need to be exact, but may be approximate and / or larger or smaller as needed to reflect tolerances, conversion factors, rounding, measurement errors, etc. and other factors known to those skilled in the art. When the term "about" is used to describe the endpoints of a value or range, the disclosure should be understood to include the specific value or endpoint involved. Regardless of whether the endpoints of a numerical value or range in this specification are described as "about", the endpoints of the numerical value or range are intended to include two embodiments: one modified by "about" and one not modified by "about". It should also be understood that the endpoints of each range in the range are significant in relation to the other endpoint and independently of the other endpoint.

[0040] As used herein, the terms "substantially," "substantially," and variations thereof are intended to indicate that the feature being described is equal to or approximately equal to a value or description. For example, a "substantially planar" surface is intended to indicate a planar or approximately planar surface. Additionally, "substantially" is intended to mean that two values ​​are equal or approximately equal. In some embodiments, "substantially" may indicate that values ​​are within about 10% of each other, such as within about 5% of each other, or within about 2% of each other.

[0041] Unless expressly indicated to the contrary, as used herein, the terms "the," "a," or "an" mean "at least one" and should not be limited to "only one." Thus, for example, reference to "a component" includes embodiments having two or more such components unless the context clearly indicates otherwise.

[0042] Overall reference Figures 1 to 7, reference numeral 10 generally designates a system for managing environmental control in a vehicle environment. The management system 10 is coupled to a vehicle 12, such as an automobile, for detecting brightness conditions by monitoring lighting in an area 14 outside the vehicle 12. The system 10 may provide enhanced monitoring and detection techniques to detect sunlight or other light sources illuminating the area 14 outside the vehicle 12 and / or the interior 16 of the vehicle 12. Light sensing may be accomplished via imaging devices 18, 20, 22, 24 inside or outside the vehicle 12 that transmit image data for processing by the management system 10 to determine lighting conditions and control an environmental control system 26 of the vehicle 12. By employing the imaging devices 18, 20, 22, 24 to detect brightness conditions in lieu of or in addition to sunlight and / or ambient light sensing in the vehicle cabin 40, interference with brightness detection, such as objects on the windshield of the vehicle 12 (e.g., windshield wipers, ice, snow, leaves, other objects), may be constrained or otherwise limited. The controls may include adjustments to exterior lighting, including headlamps 27 (e.g., low beam and / or high beam), interior lighting (e.g., lights 72 within the cabin, optical transmittance of windows of the vehicle 12, displays in the vehicle 12 (e.g., heads-up display (HUD), control panel backlights, etc.), and climate control (e.g., fan speed, heating, cooling, cycle modes, other aspects of climate, etc.). In this manner, the management system 10 utilizes the image 28 of the area 14 outside the vehicle 12 to detect lighting conditions and optimize the climate and lighting response of the vehicle 12 in lieu of or in addition to other sensors. By utilizing the imaging devices 18, 20, 22, 24, other sensors dedicated to brightness detection may be omitted or used as a redundant check on determinations made by the management system 10.

[0043] In general, the management system 10 can provide enhanced detection and response by adjusting the sampling of the images 28 captured by the imaging devices 18, 20, 22, 24 via the control circuit 38. Such adjustments are based on the operating conditions and / or light levels or directions (e.g., angles, distributions, etc.) of the vehicle 12. The management system 10 can also or alternatively provide enhanced detection and response by capturing images 28 of the area 14 outside the vehicle 12 from multiple angles (e.g., the front 30, rear 32, and sides 34, 36 of the vehicle 12) and localizing the environmental response inside or outside the vehicle 12 (e.g., adjusting the interior lighting (e.g., lighting conditions) in one area of ​​the vehicle 12 without adjusting the interior lighting (e.g., lighting conditions) in another area of ​​the vehicle 12). The management system 10 can also incorporate an artificial intelligence algorithm that can train a model based on feedback from a user or from other systems in the vehicle 12 or in surrounding vehicles. In this way, the brightness threshold can be fine-tuned over time to cause a target response of the environmental control system 26 when needed (e.g., not turning on the headlights 27 or turning on the headlights 27 at an appropriate time).

[0044] Now more specifically refer to Figure 1 , the vehicle 12 is shown as a truck having two compartments 40, 42. The two compartments 40, 42 include a truck bed 40 and a cargo bed 42. The vehicle 12 includes a plurality of sensing devices for detecting conditions of an area 14 outside the vehicle 12. For example, the sensing devices include imaging devices 18, 20, 22, 24, which may include cameras and / or infrared sensors (e.g., rain sensors), as well as thermocouples (e.g., temperature sensor 116), pressure sensors, humidity sensors, etc. The vehicle 12 includes a plurality of windows, such as a left window 39a, a right window 39b, a rear window 39c, and a front window 39d (e.g., a windshield).

[0045] The imaging devices 18, 20, 22, 24 may include a front imaging device 18, a rear imaging device 20, a right imaging device 24, and / or a left imaging device 22, each of which is configured to capture an image 28 of an area 14 outside the vehicle 12. For example, the front imaging device 18 has a field of view that captures a vehicle front view of the exterior area 14, the rear imaging device 20 has a field of view that captures a vehicle rear view of the exterior area 14, and so on. In this manner, each of the imaging devices 18, 20, 22, 24 may capture a different portion of the exterior area 14 (e.g., the front 44, the rear 46, the left 48, and the right 50). In some examples, only the front imaging device 18 is used to capture the front 44, but it is contemplated that any combination of imaging devices may be employed to capture one or more portions of the area 14 outside the vehicle 12. In a non-limiting example, a rotating imaging device is employed to capture / stitch a panoramic view or 360° view of the exterior area 14.

[0046] The locations of the imaging devices 18, 20, 22, 24 are exemplary and non-limiting. For example, while shown at the front end 52 of the vehicle 12, the front imaging device 18 may be located in the interior 16 of the vehicle 12 and behind the front window 39d. Similarly, while the rear imaging device 20 is depicted at the rear end 54 of the vehicle 12 (e.g., a rear parking assist imaging device), the rear imaging device 20 may be located in any location configured to capture the rear portion 46 of the area 14 exterior to the vehicle 12 (e.g., located in the interior 16 and oriented toward the rear window 39c). In this example, the side imaging devices 22, 24 are coupled to the rearview mirror assembly 56, but it is contemplated that other locations for the side imaging devices 22, 24 may be utilized (e.g., a top wall 58 of the vehicle 12, a door of the vehicle 12, any location where the imaging devices 18, 20, 22, 24 face the side windows). By allowing dynamic placement of the imaging devices 18, 20, 22, 24, the management system 10 can utilize pre-installed imaging devices 18, 20, 22, 24 to detect brightness levels or light directions of the area 14 outside the vehicle 12. In this way, reorganization or physical repositioning of the imaging devices 18, 20, 22, 24 can be limited to allow the software of the management system 10 to enhance brightness detection.

[0047] Reference now Figure 1 and Figure 2 , the environmental control system 26 may include any one or both of a lighting control system 62, 64 and a climate control system 66 to control environmental conditions in or around the vehicle 12. For example, the lighting control systems 62, 64 include an interior lighting system 62 and an exterior lighting system 64. The exterior lighting system 64 includes the headlights 27 of the vehicle 12, including a high beam light source 68 and a low beam light source 70. The interior lighting system 62 includes lights 72 within the vehicle cabin, such as overhead lighting, control panel backlights, display lighting (e.g., head-up display lighting), highlight lights, dome lights, and other interior lighting mechanisms. The climate control system 66 includes a heat control unit 74, an air conditioning unit 76, and a ventilation control unit 78, which can be controlled individually or together to adjust the environmental conditions of the cabin 40. For example, the ventilation control unit 78 may include one or more fans 80, which can be controlled by the climate control system 66 to control the airflow through the cabin 40. The fan speed and circulation mode (e.g., recirculation mode, ventilation mode) can be selectively controlled by the climate control system 66. Additionally, heating or cooling of the air may be adjusted by the climate control system 66 in response to signals transmitted by the control circuit 38 discussed in greater detail below.

[0048] The climate control system 26 may include a cabin 40 ( Figure 1) or the spatial map is generated based on local control of the environmental conditions of the cabin 40. For example, the spatial map 82 may outline a plurality of spaces 84, 86 in the cabin 40 that may be individually controlled by the climate control system 66 and / or the interior lighting system 62. For example, a first space 84 corresponding to an operator position may have separate temperature and / or lighting controls relative to a second space 86 corresponding to a non-operator passenger position. Although shown as two spaces 84, 86, it is contemplated that the number of spaces 84, 86 of the spatial map 82 may be higher than two to allow for more uniform lighting and climate control throughout the vehicle 12. For example, sunlight may illuminate one area of ​​the vehicle 12 and thereby make the one area warmer than another area, while the desired temperature of the entire cabin 40 is the same for all areas of the vehicle 12. In such cases, the individual control provided by the management system 10 may enhance passenger comfort.

[0049] Now more specifically refer to Figure 2 , the control circuit 38 of the management system 10 is logically inserted into the sensing devices of the vehicle 12 and the environmental control system 26 to process image data and / or other sensor data to enhance the timing of the response to the brightness detection. The control circuit 38 may include a controller 88 having a processor 90 and a memory 92. The processor 90 includes one or more digital processing devices, including, for example, a central processing unit (CPU) with one or more processing cores, a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc. In some configurations, multiple processor 90 devices are combined into a system on chip (SoC) configuration, while in other configurations, the processor 90 devices are discrete components.

[0050] The memory 92 is formed by one or more data storage devices, including, for example, magnetic or solid-state drives and random access memory (RAM) devices that store digital data. The memory 92 holds stored program instructions, sensor data from sensing devices (e.g., image data, temperature data, humidity data, light segment information, etc.), and image processing instructions. In operations including training neural network or machine learning operations, the image processing module may additionally store training data for brightness detection.

[0051] In the present example, the processor 90 includes an image processor 94 that processes image data from the imaging devices 18, 20, 22, 24 to detect brightness levels and / or light directions or angles based on patterns in the image data, such as clusters of bright pixels or clusters of dim or dark pixels in the image 28. The image processor 94 may perform various processing tasks on the image data, including pre-processing, filtering, masking, cropping, and various enhancement techniques to enhance detection and efficiency. In addition to or as a subset of brightness detection, the image data may include color information, such as pixel values ​​in a red-green-blue (RGB) array, that is processed by the image processor 94 to detect brightness. Reference Figures 3 to 5 Examples of how image processing may operate are further described.

[0052] Continue to refer Figure 2 , the control circuitry 38 may include an artificial intelligence engine 96 that uses a machine learning process to generate a trained neural network to optimize the detection of brightness and the location of brightness relative to the various sides of the vehicle 12 (e.g., the front 30, the rear 32, the right side 34, and the left side 36). For example, in addition to or in lieu of including a neural network to scan the image 28 captured by the imaging devices 18, 20, 22, 24 to detect pixel patterns, the control circuitry 38 may cause a machine learning model to be trained to determine a threshold brightness condition that causes the environmental control system 26 to respond. For example, the brightness condition determined by the control circuitry 38 may be compared to a threshold brightness condition that is preprogrammed in the controller 88 or actively adjusted through machine learning techniques. In some examples, manual feedback may be used to train the machine learning model to modify the threshold brightness condition. A digital switch (e.g., a touch screen) or a physical button or switch (see Figure 1 ) can be manipulated by a user, such as an occupant of the vehicle 12, to override a response initiated by the control circuit 38 at the environmental control system 26. In this way, a machine learning model can be trained via supervised learning to update the brightness condition thresholds.

[0053] The brightness conditions may include an indication of light angle (e.g., sun or other light source low in the sky versus high in the sky) and intensity. For example, a map of the overall light distribution of the vehicle 12 may indicate brighter aspects on some of the sides of the vehicle 12 and darker aspects on other sides of the vehicle 12. Additionally, the location of the light source may be detected based on areas of the image 28 captured by at least one imaging device 18, 20, 22, 24 having higher brightness levels. By detecting the location of the light source (e.g., sun, oncoming vehicle headlights, street lights, etc.), the control circuitry 38 may determine the light angle to further refine the response of the system 10.

[0054] Still reference Figure 2, a sampling unit 98 may be in communication with or otherwise included within the processor 90 to allow the control circuit 38 to adjust the sampling rate of the images 28 captured by one or more of the imaging devices 18, 20, 22, 24. The sampling unit 98 may passively or actively operate with the imaging devices 18, 20, 22, 24 by selectively reading image data or actively initiating an image capture process. In either example, the number of frames of the area 14 outside the vehicle 12 analyzed by the image processor 94 may be controlled by the control circuit 38 via the sampling unit 98.

[0055] Continue to refer Figure 2 , the control circuit 38 may communicate with auxiliary systems and sensing devices other than the imaging devices 18, 20, 22, 24 to provide redundancy and / or logic checks of brightness conditions. For example, the powertrain 100, ignition system 102, positioning system 104 (such as a global positioning system (GPS)), vehicle-to-vehicle communication system (V2V system) 106, rain detection system 108, at least one microphone 110, sunlight sensor 112, ambient light sensor 114, temperature sensor 116 and / or any other sensing device or vehicle system of the vehicle 12 may be communicatively coupled to the control circuit 38. In one example, a speed sensor 118 communicates with the control circuit 38 to allow the control circuit 38 to modify the detection technique based on the speed of the vehicle 12. The control circuit 38 may use data from these auxiliary systems and sensing devices to check or support the determination of brightness based on the image 28, as will be further described below.

[0056] Reference now Figures 3 to 5 , the management system 10 may now be described Figure 2 The exemplary process S300, S400 by which the adjustment of the environmental control system 26 is determined by the hardware of the embodiment of the present invention. Figure 3 , the process S300 for controlling the environment control system 26 includes processing a first set of images of a plurality of images 28 captured by at least one of the imaging devices 18, 20, 22, 24 of the area 14 outside the vehicle 12 at a first sampling rate at step S302. For example, at least one imaging device 18, 20, 22, 24 (e.g., one or more of the imaging devices 18, 20, 22, 24) may be employed to capture the images 28 of the area 14 outside the vehicle 12 and the sampling unit 98 may be controlled by the processor 90 to sample the images 28 at a relatively fast rate. In some examples, the first sampling rate is between 10 frames per second (fps) and 60 frames per second (fps). In some examples, the first sampling rate is approximately 30fps to 60fps.

[0057] The process S300 also includes determining, at S304, a brightness level of an area 14 outside the vehicle 12 based on processing the first set of images. In one example, the first sampling rate is applied during an initialization mode in which one or more of the imaging devices 18, 20, 22, 24 capture initial conditions, including brightness conditions, temperature, humidity, etc. As will be further described herein, the initialization mode may also classify the location of the vehicle 12 as "inside" or "outside" based on the surrounding environment detected based on the image data, and provide a basic brightness condition threshold for comparing changes.

[0058] Once the brightness level of the external area 14 is determined, the control circuit 38 can process the second set of images at a second sampling rate different from the first sampling rate at S306. The second image can be captured at this time. In this way, the second set of images can be temporally after the first set of images. For example, after the initialization mode, the control circuit 38 can enter an active mode in which the sampling rate of the sampling module is actively controlled based on one or more conditions of the area 14 outside the vehicle 12 or the operating conditions of the vehicle 12. For example, based on communication with the powertrain 100 or the speed sensor 118, the control circuit 38 can receive an indication that the vehicle 12 is moving (e.g., driving, reversing, etc.) rather than parked or not moving. In response to the movement of the vehicle 12 and / or the termination of the initialization mode, the control circuit 38 can reduce the sampling rate to the second sampling rate. Therefore, after the initialization mode (which can provide the position data of the vehicle 12 and / or other information to provide a confidence value of the brightness level of the area 14 outside the vehicle 12), the active mode can be operable.

[0059] In some examples, the control circuit 38 can be adjusted to a second sampling rate in response to detecting the rate of change of the brightness level. In this example, the first group of images refers to the image 28 captured during the active mode. When the rate of change of the brightness level increases beyond the rate of change limit, the control circuit 38 adjusts to the second sampling rate to provide a more stable prediction of the brightness condition. For example, if the vehicle 12 moves along the road at night and the street lights periodically illuminate the street, if the brightness level is detected at 60fps, the brightness level can be adjusted between a low level and an extremely high level based on when the vehicle 12 is away from the street lights or near the street lights. In response to this rate of change of the brightness level, the control circuit 38 can reduce the sampling rate to provide a more uniform estimate of the brightness for the purpose of initiating a response of the environmental control system 26. In some examples, the second frame rate can be between 1fps and 20fps. In some examples, the second frame rate is between 1fps and 10fps.

[0060] Continue to refer Figure 3At step S308, the control circuit 38 detects a transition in brightness level based on the second set of images. For example, a transition may be an actual change in brightness conditions that suggests a response by the environmental control system 26. In the example of driving on a street partially illuminated by street lights at night, a transition may refer to a more uniform or average brightness change that may or may not cause a response of activating the headlights 27 of the vehicle 12. In this way, the transition is an indication to the control system to adjust the environmental control system 26.

[0061] The control circuit 38 may then control the environmental control system 26 in response to the transition in brightness level at step S310. Thus, based on the detected transition in brightness level, the environmental control system 26 may turn on the headlights 27, increase the brightness of the instrument panel or HUD in the cabin 40, adjust the temperature of the cabin 40 via controlling ventilation, or adjust any other parameter of the environmental conditions. Such adjustments may be made to individual spaces 84, 86 in the vehicle 12 or to the entire cabin 40 of the vehicle 12.

[0062] Reference now Figure 4 Another process S400, which may be a sub-process of the first process S300 or separate from the first process S300, provides for switching between an initialization mode and an activation mode to control the headlights 27 of the vehicle 12. The process S400 includes the control circuit 38 detecting the ignition of the vehicle 12 via the ignition system 102 at step S402. At S404, the control circuit 38 detects an automatic lighting mode. The automatic lighting mode may refer to an operating mode of the headlights 27 that allows the management system 10 to control whether the headlights 27 are activated based on brightness conditions. Once the engine of the vehicle 12 is turned on and the automatic lighting mode is enabled, the initial conditions of the interior 16 and / or exterior area 14 of the vehicle 12 are determined by the control circuit 38 at step S406.

[0063] Based on the initial conditions, at step S408, the control circuit 38 may classify the area 14 outside the vehicle 12 as "inside" or "outside". In this way, at step S410, the control circuit 38 may assign initialization values ​​to the brightness conditions inside the structure (e.g., under a carport or garage, in a tunnel, etc.), and also energize the headlights 27 based on the classification of the vehicle 12 as inside. If the vehicle 12 is "outside" (e.g., not covered by a structure), then at step S412, the control circuit 38 classifies the brightness conditions when the vehicle 12 is outside and assigns the initialization values ​​to the brightness conditions. In this example, two imagers (e.g., a front imaging device 18 and a rear imaging device 20) scan the area 14 outside the vehicle 12 to capture images 28 indicating brightness conditions in the vehicle forward direction and the vehicle rearward direction. The field of view of these imaging devices 18, 20, 22, 24 may also cover at least a portion of the side view from the vehicle 12. In this example, if the front or rear brightness conditions are below the brightness threshold, the control circuit 38 can, in response, de-energize the headlights 27. In other examples, the control circuit 38 can blend or combine the front and rear brightness conditions to determine the overall brightness level of the exterior area 14.

[0064] The brightness conditions are continuously processed while the vehicle 12 is idling. Once the speed of the vehicle 12 is non-zero (step S414), the sampling rate of the sampling module is set to a first sampling rate at step S416 to track the brightness conditions during the maneuvering (e.g., driving) of the vehicle 12. During the operation of the vehicle 12, at step S418, the control circuit 38 actively monitors the brightness conditions to control the activation or deactivation of the headlights 27.

[0065] During step 418, a machine learning model may be applied and trained based on automatic changes to the headlights 27 implemented by the control circuitry 38 and the user's response. For example, manually turning on the headlights while the processor 90 is calculating the brightness level may change the control circuitry 38 to default to energizing the headlights 27 in future events with similar or identical brightness conditions. Using the image 28 to detect the headlights of other vehicles may also be used to train a machine learning model. For example, a pair of pixel clusters may be detected in the image 28, and in response, the control circuitry 38 may determine that the other vehicle has turned on the headlights 27. The control circuitry 38 may then activate the headlights 27.

[0066] To enhance power management, the control circuit 38 also dynamically adjusts the sampling rate based on whether the vehicle 12 is parked, not moving, or otherwise stationary for a longer period of time, and reduces the sampling rate after such a determination. For example, at step S420, the control circuit 38 classifies the stop of the vehicle 12 as a "stop-and-go" condition or a "full stop" condition (step S422). In a stop-and-go condition, such as in traffic or stopped for a relatively short duration (a few seconds or minutes), the control circuit 38 continues to sample at the first sampling rate. At step S424, if the vehicle 12 stops for more than a relatively short duration (e.g., at least 3 minutes, at least 5 minutes, at least 10 minutes, etc.), the control circuit 38 can adjust to a lower sampling rate to save power. Because the vehicle 12 is not moving, the brightness condition may be less actively tracked (e.g., a low sampling rate). In some examples, the gear state of the powertrain 100 is a park gear and / or the parking brake of the vehicle 12 is activated, which may also or alternatively provide an adjustment to the lower sampling rate.

[0067] At step S426, the user may be notified via a human machine interface (HMI) such as a HUD or a touch screen panel that the headlights 27 should be disabled. For example, the user may be notified that the user is requested to disable the headlights 27. If the vehicle 12 begins moving again (step S414), the process may repeat steps S416 to S420. Otherwise, once the engine shutdown is detected by the control circuit 38 at step S428, the process ends.

[0068] Adjusting frame sampling by the control circuitry 38 described above may provide enhanced computational and / or power savings. For example, a reduced sampling rate may result in less image data to be processed and thus allow other vehicle monitoring systems to process data at a common power and / or computational level.

[0069] Reference now Figure 5 , a process S500 for extracting brightness conditions from images 28 captured from one or more imaging devices 18, 20, 22, 24 and generating / training a spatial brightness model 120 may be used independently or in combination with the previously described sampling rate adjustment processes S300, S400. The process S500 includes capturing images 28 from one or more angles at S502. For example, while the image in step S502 has a field of view in the vehicle forward direction, the vehicle rearward direction and / or the vehicle lateral direction may be captured.

[0070] The image 28 and metadata corresponding to the image 28 (e.g., exposure time, brightness, saturation, etc.) are then transmitted to the control circuit 38, which can mix or fuse the image data and metadata from the multiple imaging devices 18, 20, 22, 24 to present the brightness conditions of one or more of the portions 44, 46, 48, 50 of the area 14 outside the vehicle 12 at step S504. For example, the control circuit 38 can process pixel values, such as red values, green values, and blue values, to estimate the brightness conditions. The image processor 94 can determine the lux value of the image 28 and mix the lux value. For example, Figure 5 The table shown in shows exemplary lux values ​​for localized areas of the vehicle 12 (eg, the spaces 84 , 86 of the cabin 40 ).

[0071] At step S506, the brightness condition is compared with the Figure 2 The control circuit 38 shown in the figure can be used to compare the information collected by the auxiliary sensors and systems in communication with the auxiliary sensors and systems. For example, if there is a sunlight sensor 112, the brightness condition can be compared with the sunlight information generated by a photodiode or other device in the vehicle 12 to serve as a logic check for the brightness condition. For example, the on / off signal from the sunlight sensor 112 or the ambient light sensor 114 can be compared with the brightness condition estimated by the control circuit 38. If the brightness condition estimate is not aligned or consistent with the signal from the auxiliary sensing device, the brightness condition estimate can be modified to be lower or higher based on the inconsistency. For example, if the sunlight sensor 112 produces a very high light (lux) reading and the brightness condition estimate is average, the control circuit 38 can adjust the estimate higher. It is contemplated that the auxiliary sensing device can be omitted in various examples, and only one or more of the imaging devices 18, 20, 22, 24 are used to determine the brightness condition.

[0072] At step S508, the control circuitry 38 may compare the brightness condition estimate to the threshold data to determine whether a response from the environmental control system 26 is requested, and if so, what that response may be. At step S510, the control circuitry 38 may transmit instructions to adjust the environmental control system 26 based on the brightness condition exceeding the threshold data. For example, if the brightness condition estimate is high and the headlights 27 are off, no adjustment may be initiated to the exterior lighting system 64. However, the climate control system 66 may adjust the temperature control within the cabin 40 to cool the cabin 40, and / or the interior lighting system 62 may adjust the lighting in the cabin 40 to reduce interior lighting. The response may be localized to the spaces 84, 86 aligned with the sides 34, 36 of the vehicle 12 having the higher lux readings.

[0073] For example, Figure 5The table in shows the changing lux values ​​corresponding to different parts of the vehicle 12, such as the front 30, rear 32, and sides 34, 36. The first space 84 can be aligned with the left front of the cabin 40, and the second space 86 can be aligned with the right front of the cabin. Because the left side 34 has a lower lux value, the control circuit 38 can control the interior lighting to provide more lighting to the left side 34 of the interior 16 and less lighting to the right side 36 of the interior 16. In addition or alternatively, if both the first space 84 and the second space 86 are set to maintain a common temperature by the climate control system 66, the climate control system 66 can increase the fan speed to cool the right side 36 more than the left side to take into account that there is more heat on the right side 36 than on the left side 34 (e.g., from sunlight). It is contemplated that this example is non-limiting. In general, by associating brightness conditions with areas of the cabin 40, the interior lighting and climate of the cabin 40 can be individually and precisely controlled by the control circuit 38.

[0074] Figure 5 The table in also shows the generation and training of the spatial brightness model 120. For example, the control circuit 38 can compare various measurements with the brightness flag. The brightness flag can be stored in the memory 92 as a predefined value generated based on the use of a real value light sensor to compare with the test measurement of the lux value of different parts of the area 14 outside the vehicle 12. For example, the spatial brightness model 120 can determine the lux value of the image 28 captured by the front imaging device 18 relative to the lux value of the image 28 captured by the rear imaging device 20. In addition, the brightness flag can be dynamic and can be changed based on user feedback. At step S512, the process detects user feedback in response to the adjustment of the environmental control system 26. For example, if the temperature control is modified in response to the brightness condition, the user can manually control the climate control system 66 to undo the change. Based on the feedback, the spatial brightness model 120, the brightness flag and the threshold information can be updated. In another example, the headlights 27 are automatically activated in response to the brightness level being lower than the threshold brightness level, and the user can manually adjust the headlights 27 to keep the headlights 27 turned on. Such adjustments may provide user feedback to the control circuitry 38 to update the trained model and threshold data.

[0075] Reference now Figure 6 and Figure 7 , various examples of the management system 10 and associated processes S300, S400, S500 applied to the captured images 28 demonstrate enhanced control of the environmental control system 26. For example, the front imaging device 18 may capture an image of the tunnel 122 in front of the vehicle 12 ( Figure 6). The management system 10 may provide proactive adjustments to the headlights 27 before the vehicle 12 enters the tunnel 122. For example, the image processor 94 may use a trained model to correlate the image with other images 28 of the tunnel 122 to detect the tunnel 122, or may otherwise determine via pixel analysis that dark spots on the image become larger as more images 28 are sampled. Because the brightness level may decrease as the vehicle 12 approaches the tunnel 122, the management system 10 may reduce or increase the sampling rate to detect that such a brightness level change is actually occurring and control the exterior lighting system 64 to energize the headlights 27. In another example, a pair of oncoming lights may be detected in an image captured by the front imaging device 18. In response, the control circuit 38 may de-energize the high beams of the headlights 27 without de-energizing the low beams.

[0076] refer to Figure 7 , the management system 10 can detect the brightness conditions of one or both of the right side 34 and the left side 36 to allow local control of the space in the cabin 40. Figure 7 In the example of FIG. 1 , tall trees 124 are located on both sides 34, 36 of the vehicle 12, which causes the lux levels of the right imaging device 22 and the left imaging device 24 to be lower than the lux levels of the front imaging device 18 and the rear imaging device 20. In this example, the control circuit 38 can adjust any subsystem of the climate control system 26 to cause the vehicle 12 to have consistent lighting and heating / cooling in the interior 16.

[0077] It is contemplated that in any of the foregoing examples, the location of the vehicle 12, the time of day, the temperature, the direction of travel, or any other factor related to the conditions of the area 14 outside the vehicle 12 may be accessed by the control circuitry 38 to further enhance the brightness condition estimate. For example, the GPS may indicate that the location of the vehicle 12 is in an area that has less sunlight than other areas (e.g., the Northern Hemisphere). In other examples, the GPS may track the vehicle 12 along a driving route, which may be predefined by a user or predicted based on the location of the vehicle 12. Thus, the driving route may refer to an expected route that the vehicle 12 will travel. The time of day may also be used to track the position of the sun relative to the direction of travel and path of the vehicle 12. In these examples, the control circuitry 38 adjusts the brightness threshold information, the brightness flag, and / or the spatial brightness model 120 to provide an enhanced estimate that limits manual adjustments. For example, if it is determined based on the GPS information and the time of day information that the brightness condition of the area 14 is less than that of the vehicle 12, the control circuitry 38 may adjust ... Figure 7In an example, if the sun is above the position of the vehicle 12, the lux values ​​of the surrounding sides 34, 36 (e.g., the left side 34 and / or the right side 36) may be insignificant for the brightness estimation used to activate the headlights 27. In contrast, if the sun is aligned with the right side 36 of the vehicle 12, the direction of the light may be determined by the control circuit 38. Therefore, in this example, the control circuit 38 may weight the lux values ​​of the front imaging device 18 and the rear imaging device 20 more heavily. In this way, image processing for classifying the surrounding environment (e.g., trees, buildings, mountains) can be combined with processing for determining light direction and magnitude.

[0078] In one example, data from one or more of the microphones 110 may be employed to detect rain (e.g., the sound of rainfall). In response to such detection, the control circuitry 38 may cause the headlights 27 to be powered on by heavily weighting the auxiliary sensor data from the microphones 110. For example, in another mechanism of the neural network or spatial brightness model 120, neurons corresponding to audio data captured by the microphones 110 may be heavily weighted to cause the brightness estimate to exceed the threshold information and cause the headlights 27 to be powered on. In this manner, the management system 10 and processes of the present disclosure may provide software techniques for enhancing the responsiveness of exterior lighting.

[0079] In general, the management system 10 can enhance brightness detection and response by using machine learning and controlling sampling rates and other mechanisms. The system 10 can increase response time and more accurately predict events to adjust lighting and / or climate.

[0080] It will be understood that changes and modifications may be made to the foregoing constructions without departing from the concepts of the present disclosure, and it will be further understood that such concepts are intended to be encompassed by the appended claims unless the claims expressly state otherwise by their language.

[0081] According to the present invention, a vehicle is provided, the vehicle comprising: a cabin having a first space and a second space, the first space having a first environmental condition and the second space having a second environmental condition; at least one camera, the at least one camera capturing at least one first image of a first portion of an area outside the vehicle and capturing at least one second image of a second portion of the area outside the vehicle, wherein the first portion of the area is aligned with the first space and the second portion of the area is aligned with the second space; an environmental control system of the vehicle, the environmental control system having at least one of lighting control and climate control for the vehicle; and a control circuit communicating with the at least one camera and the environmental control system, the control circuit being configured to: determine a first brightness condition of the first portion of the area based on at least one first image; determine a second brightness condition of the second portion of the area based on at least one second image; and control the environmental control system to adjust the first environmental condition and the second environmental condition based on the first brightness condition and the second brightness condition.

[0082] According to one embodiment, the adjustment of the first environmental condition is independent of the adjustment of the second environmental condition.

[0083] According to one embodiment, the adjustment of the first environmental condition and the second environmental condition is an adjustment of the climate of the first space and the second space, respectively.

[0084] According to one embodiment, the first brightness condition and the second brightness condition each include a brightness angle and a brightness level.

[0085] According to one embodiment, the control circuit is configured to process the first brightness condition and the second brightness condition in a spatial brightness model trained to determine a response to the environmental control system based on the first brightness condition and the second brightness condition.

[0086] According to one embodiment, the at least one camera includes a first camera having a vehicle forward orientation and a second camera having an orientation different from the vehicle forward orientation, wherein the first camera captures at least one first image and the second camera captures at least one second image.

[0087] According to one embodiment, the control circuit is configured to: process multiple first images of at least one first image captured sequentially by a first camera, detect dark pixel clusters of increasing size across the multiple first images captured sequentially; and activate the headlights of the vehicle in response to the increased size of the dark pixel clusters.

[0088] According to one embodiment, the spatial brightness model maps brightness conditions to the first space and the second space.

[0089] According to one embodiment, the response includes adjustment of at least one of interior lighting of the vehicle and climate control of the cabin.

[0090] According to one embodiment, the control circuit is configured to train the spatial brightness model based on user feedback of previous responses of the environmental control system.

[0091] According to one embodiment, the invention is further characterized by: headlights, and wherein the response is controlling the headlights of the vehicle.

[0092] According to the present invention, a vehicle is provided, comprising: a cabin having a first space and a second space, the first space having a first climate and the second space having a second climate; at least one camera, the at least one camera capturing an image of a first portion of an area outside the vehicle aligned with the first space and an image of a second portion of an area outside the vehicle aligned with the second space; a climate control system; and a control circuit communicating with the at least one camera and the climate control system, the control circuit being configured to: determine a first brightness condition of a first portion of the area based on an image; determine a second brightness condition of a second portion of the area based on the image; control the climate control system to adjust the first climate based on the first brightness condition and the second brightness condition; and control the climate control system to adjust the second climate based on the first brightness condition and the second brightness condition.

[0093] According to one embodiment, the adjustment of the first climate is independent of the adjustment of the second climate.

[0094] According to one embodiment, the control circuit is configured to process the first brightness condition and the second brightness condition in a spatial brightness model that is trained to determine a response to the climate control system based on the first brightness condition and the second brightness condition.

[0095] According to one embodiment, the spatial brightness model maps the first brightness condition and the second brightness condition to the first space and the second space.

[0096] According to one embodiment, the control circuit is configured to train the spatial brightness model based on user feedback of previous responses of the climate control system.

[0097] According to one embodiment, the invention is further characterized by: a ventilation system, and wherein the response is controlling the ventilation system.

[0098] According to the present invention, a vehicle is provided, comprising: a cabin having a first space and a second space, the first space having a first climate and the second space having a second climate; at least one camera, the at least one camera capturing an image of a first portion of an area outside the vehicle aligned with the first space and an image of a second portion of an area outside the vehicle aligned with the second space; a climate control system; and a control circuit communicating with the at least one camera and the climate control system, the control circuit being configured to: determine a first brightness condition of a first portion of the area based on an image; determine a second brightness condition of a second portion of the area based on the image; process the first brightness condition and the second brightness condition in a spatial brightness model, the spatial brightness model being trained to determine a response to the climate control system based on the first brightness condition and the second brightness condition; and control the climate control system based on the response to adjust the second climate.

[0099] According to one embodiment, the adjustment of the first climate is independent of the adjustment of the second climate.

[0100] According to one embodiment, the first brightness condition and the second brightness condition each include a brightness angle and a brightness level.

Claims

1. A vehicle comprising: a cabin having a first space and a second space, the first space having a first environmental condition and the second space having a second environmental condition; at least one camera that captures at least one first image of a first portion of an area exterior to the vehicle and captures at least one second image of a second portion of the area exterior to the vehicle, wherein the first portion of the area is aligned with the first space and the second portion of the area is aligned with the second space; an environmental control system for the vehicle, the environmental control system having at least one of lighting control and climate control for the vehicle; as well as a control circuit in communication with the at least one camera and the environmental control system, the control circuit being configured to: determining a first brightness condition of the first portion of the area based on the at least one first image; determining a second brightness condition of the second portion of the area based on the at least one second image; and The environmental control system is controlled to adjust the first environmental condition and the second environmental condition based on the first brightness condition and the second brightness condition. 2 . The vehicle of claim 1 , wherein the adjustment to the first environmental condition is independent of the adjustment to the second environmental condition.

3. The vehicle of claim 2, wherein the adjustments to the first and second environmental conditions are adjustments to the climate of the first and second spaces, respectively.

4. The vehicle of claim 1, wherein the first brightness condition and the second brightness condition each include a brightness angle and a brightness level.

5. The vehicle of claim 1, wherein the control circuit is configured to: The first brightness condition and the second brightness condition are processed in a spatial brightness model that is trained to determine a response to the environmental control system based on the first brightness condition and the second brightness condition.

6. The vehicle of claim 5, wherein the at least one camera comprises a first camera having a vehicle forward orientation and a second camera having an orientation different from the vehicle forward orientation, wherein the first camera captures the at least one first image and the second camera captures the at least one second image.

7. The vehicle of claim 6, wherein the control circuit is configured to: processing a plurality of first images of the at least one first image sequentially captured by the first camera; detecting dark pixel clusters of increasing size across the plurality of sequentially captured first images; and Headlights of the vehicle are activated in response to the increasing size of the dark pixel clusters.

8. The vehicle of claim 6, wherein the spatial brightness model maps the brightness conditions to the first space and the second space.

9. The vehicle of claim 8, wherein the response comprises an adjustment to at least one of interior lighting of the vehicle and climate control of the cabin.

10. The vehicle of any one of claims 5 to 9, wherein the control circuit is configured to: The spatial brightness model is trained based on user feedback of previous responses of the environmental control system.

11. The vehicle of any one of claims 5 to 9, further comprising: headlights, and wherein the response is controlling the headlights of the vehicle.

Images