Vehicle display system

Abstract

The invention provides a vehicle display system. A vehicle system includes a display unit and a computer communicatively coupled to a camera. The display unit includes: a display panel; the circuit board is fixed relative to the display panel; a plurality of visible light emitting diodes (LEDs) mounted to the circuit board; and a plurality of infrared LEDs, the plurality of infrared LEDs being mounted to the circuit board and being scattered together with the visible light LEDs. The circuit board is arranged to guide emitted light from the visible light LED and the infrared LED to the display panel. The computer is programmed to actuate the component based on an infrared image from the camera depicting an occupant illuminated by the infrared LED, and adjust the brightness of the infrared LED based on ambient brightness.

Description

Technical Field

[0001] This disclosure relates to a display system in a vehicle. Background Art

[0002] Modern vehicles typically include a display screen. The display screen can be of various types suitable for displaying content readable by the vehicle's occupants, such as light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), plasmas, digital light processing technology (DLPT), etc. The display screen can be mounted to the dashboard, where it is visible to the vehicle's occupants. Summary of the Invention

[0003] This disclosure describes a vehicle system in which a display unit of the vehicle can support imaging of the vehicle's occupants. The display unit includes a plurality of visible light-emitting diodes (LEDs) for displaying information on a display panel and a plurality of infrared LEDs for illuminating the occupants. The infrared LEDs are dispersed together with the visible LEDs on a circuit board of the display unit. Co-locating the infrared LEDs with the visible LEDs can reduce power consumption, package space, and wiring complexity compared to having a separate infrared illuminator in the passenger compartment. The location of the infrared LEDs and the distribution of the infrared LEDs among the visible LEDs can also provide a larger and more diffuse infrared illumination source than a separate infrared illuminator, thereby providing more complete illumination for the occupants.

[0004] Illuminating the occupants with infrared illumination can support the operation of vehicle components (e.g., a driver status monitoring system). The vehicle can include a camera capable of detecting infrared light and a computer communicatively coupled to the camera and the infrared LEDs. The computer is programmed to actuate components based on an infrared image of the occupants illuminated by the infrared LEDs from the camera, e.g., to output a message to the occupants if it is determined from the infrared image that the occupants have diverted their attention from the road. The computer is also programmed to adjust the brightness of the infrared LEDs based on the ambient brightness. For example, the infrared LEDs may be brighter during the day so that the infrared image is not overexposed by sunlight, and the infrared LEDs may be dimmer during the night to save energy.

[0005] A vehicle system includes a display unit and a computer communicatively coupled to the display unit. The display unit includes: a display panel; a circuit board fixed relative to the display panel; a plurality of visible light emitting diodes (LEDs), the plurality of visible light LEDs being mounted to the circuit board; and a plurality of infrared LEDs, the plurality of infrared LEDs being mounted to the circuit board and dispersed with the visible light LEDs. The circuit board is arranged to direct emitted light from the visible light LEDs and the infrared LEDs to the display panel. The computer is programmed to actuate components based on an infrared image of an occupant illuminated by the infrared LEDs from a camera, and to adjust the brightness of the infrared LEDs based on ambient brightness.

[0006] In an example, the computer may be programmed to increase the brightness of the infrared LEDs in response to an increase in the ambient brightness.

[0007] In an example, the vehicle system may further include the camera, the camera may be configured to detect visible light and infrared light, and the computer may be programmed to actuate the components based on a visible light image of the occupant from the camera.

[0008] In an example, the vehicle system may further include the camera, the camera may be configured to detect visible light and infrared light, and the computer may be programmed to actuate the components based on a visible light image of the occupant from the camera in response to the ambient brightness exceeding a threshold. In another example, the computer may be programmed to actuate the components based on the infrared image when the ambient brightness is below the threshold. In yet another example, the computer may be programmed to actuate the components based on the infrared image when the ambient brightness is above the threshold.

[0009] In yet another example, the computer may be programmed to inhibit actuating the components based on the visible light image in response to the ambient brightness being below the threshold. In yet another example, the computer may be programmed to adjust the brightness of the infrared LEDs by changing the pulse width modulation of the infrared LEDs.

[0010] In an example, the display unit may include a light guide plate fixed behind the display panel, and the circuit board may be arranged to direct emitted light from the visible light LEDs and the infrared LEDs into the light guide plate at an edge of the light guide plate. In another example, the visible light LEDs and the infrared LEDs may be arranged in rows along the edge of the light guide plate.

[0011] In an example, the number of the visible light LEDs may be greater than the number of the infrared LEDs.

[0012] In an example, the circuit board may be fixed behind the display panel and arranged parallel to the display panel. In another example, the display unit may include a plurality of optical domes encapsulating corresponding visible light LEDs, and the infrared LEDs may be encapsulated in corresponding ones of the optical domes. In yet another example, a subset of the optical domes may encapsulate corresponding infrared LEDs.

[0013] In yet another example, the visible light LEDs and the infrared LEDs may be arranged in a two-dimensional pattern on the circuit board.

[0014] In an example, the vehicle system may further include a dashboard, and the display unit may be mounted to the dashboard.

[0015] In an example, the vehicle system may further include the camera, and the camera may be spaced apart from the display unit.

[0016] A display unit includes: a display panel; a light guide plate fixed behind the display panel; a circuit board fixed relative to the display panel; a plurality of visible light light-emitting diodes (LEDs) mounted to the circuit board; and a plurality of infrared LEDs mounted to the circuit board and dispersed with the visible light LEDs. The circuit board may be arranged to direct emitted light from the visible light LEDs and the infrared LEDs into the light guide plate at an edge of the light guide plate.

[0017] In an example, the visible light LEDs and the infrared LEDs may be arranged in rows along the edge of the light guide plate.

[0018] In an example, the number of visible light LEDs may be greater than the number of infrared LEDs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a rear perspective view of an exemplary vehicle, where the passenger compartment is exposed for illustration.

[0020] Figure 2 is an illustrative exploded side cross-sectional view of an exemplary display unit of a vehicle.

[0021] Figure 3 is Figure 2 an illustrative plan view of the display unit of

[0022] Figure 4 is an illustrative exploded side cross-sectional view of another exemplary display unit of a vehicle.

[0023] Figure 5A is withFigure 4 Diagrammatic plan view of a consistent exemplary display unit.

[0024] Figure 5B is consistent with Figure 4 Diagrammatic plan view of another exemplary display unit.

[0025] Figure 6 Block diagram of an exemplary control system of a vehicle.

[0026] Figure 7 Flowchart of an exemplary process for determining when to output a message to an occupant of a vehicle. Detailed Description

[0027] Referring to the accompanying drawings, in which like reference numerals indicate like parts throughout the several views, vehicle system 105 of vehicle 100 includes a display unit 110 and a computer 600 communicatively coupled to display unit 110. Display unit 110 includes a display panel 205, a circuit board 210 fixed relative to display panel 205, a plurality of visible light light-emitting diodes (LEDs) 215 mounted to circuit board 210, and a plurality of infrared LEDs 220 mounted to circuit board 210 and dispersed with visible light LEDs 215. Circuit board 210 is arranged to direct emitted light from visible light LEDs 215 and infrared LEDs 220 to display panel 205. Computer 600 is programmed to actuate components based on an infrared image of an occupant illuminated by infrared LEDs 220 from camera 115, and to adjust the brightness of infrared LEDs 220 based on ambient brightness.

[0028] Referring to Figure 1 , vehicle 100 can be any passenger or commercial vehicle, such as a sedan, truck, sport utility vehicle, crossover vehicle, van, minivan, taxi, bus, etc.

[0029] Vehicle 100 includes a passenger compartment 125 for accommodating the occupants of vehicle 100. Passenger compartment 125 can include a plurality of seats 130. One or more of seats 130 can be disposed in the front row of passenger compartment 125, and one or more of seats 130 can be disposed in a second row behind the front row. Passenger compartment 125 can also include seats 130 in a third row (not shown) at the rear of passenger compartment 125. Seats 130 are shown as bucket seats in the front row and bench seats in the second row, but seats 130 can be other types. The position and orientation of seats 130 and their components can be adjusted by the occupants. Each seat 130 can define a corresponding occupant seating area 120, i.e., the space occupied or to be occupied by an occupant sitting in seat 130.

[0030] The passenger compartment 125 includes a dashboard 135. The dashboard 135 can be disposed at the front end of the passenger compartment 125 and face the front row seats 130. The dashboard 135 can include vehicle controls such as a steering wheel 145; instruments, dials, and information displays; heating and ventilation equipment; radios, and other electronics, etc.

[0031] The vehicle 100 includes a user interface 140. The user interface 140 presents information to and receives information from an operator of the vehicle 100. The user interface 140 can include dials, digital readouts, a screen such as the display unit 110, speakers, etc. for providing information to the operator, e.g., such as known human-machine interface (HMI) elements. The user interface 140 can include buttons, knobs, keypads, microphones, etc. for receiving information from the operator. Components of the user interface 140 can be mounted to the dashboard 135 or other locations in the passenger compartment 125.

[0032] The user interface 140 includes a display unit 110. The display unit 110 is mounted to the dashboard 135. The display unit 110 can be positioned to be visible to an operator of the vehicle 100 when the operator's head faces forward in the direction of travel of the vehicle 100, which means the operator can see the display unit 110 by moving their eyes without moving their head. The display unit 110 can also be positioned to irradiate an occupant of the vehicle 100, e.g., the operator, with an infrared LED 220, which will be described below. For example, the display unit 110 can be positioned above and in front of the steering wheel 145, e.g., to serve as an instrument panel. For another example, the display unit 110 can be positioned on the center console of the dashboard 135 and be visible to the operator as well as other occupants.

[0033] The display unit 110 can convey information to the occupants. For example, the display unit 110 can display information about the operation of the vehicle, such as speed, engine revolutions per minute, engine temperature, fuel level, or battery charge, etc.; passenger compartment status, such as open doors, seat positions, climate control settings, etc.; infotainment information, such as radio stations, media sources, volume, connection to user devices, etc.; and other information.

[0034] The camera 115 can detect electromagnetic radiation in a certain wavelength range. For example, the camera 115 can be configured to detect both visible light and infrared radiation. For example, the camera 115 can be a charge-coupled device (CCD), complementary metal-oxide semiconductor (CMOS), or any other suitable type.

[0035] The camera 115 is arranged such that the field of view of the camera 115 covers at least one of the occupant seating areas 120, e.g., the occupant seating area 120 for the operator of the vehicle 100, i.e., the occupant seating area 120 defined by the seat 130 directly behind the steering wheel 145. The camera 115 can be spaced apart from the display unit 110. For example, the camera 115 can be mounted to the rearview mirror, mounted to the instrument panel 135 at the center console, or positioned at the left or right lateral end of the instrument panel 135 below the front pillar of the vehicle 100.

[0036] Reference Figure 2 Referring to FIGS. 4 and 5, as an overall overview of the display unit 110, the display unit 110 includes a circuit board 210, visible light LEDs 215, infrared LEDs 220, a reflector 225, a display panel 205, and other layers. (For clarity, only some of the visible light LEDs 215 and infrared LEDs 220 are labeled). The components of the display unit 110 can be fixed relative to each other and fixed relative to the instrument panel 135; that is, the display unit 110 may lack moving parts. Figures 2 to 3 An example of a side-light arrangement of the visible light LEDs 215 and the infrared LEDs 220 is shown, and Figure 4 FIGS. 4 to 5 show examples of a direct backlight arrangement of the visible light LEDs 215 and the infrared LEDs 220. Generally and in both examples, the other layers can be sandwiched between the reflector 225 and the display panel 205. Generally and in both examples, the visible light LEDs 215 and the infrared LEDs 220 are mounted to the circuit board 210, the infrared LEDs 220 are scattered together with the visible light LEDs 215, and the circuit board 210 is arranged to direct the emitted light from the visible light LEDs 215 and the infrared LEDs 220 to the display panel 205.

[0037] The circuit board 210 serves as a substrate for physically mounting and electrically connecting the visible light LEDs 215 and the infrared LEDs 220. The circuit board 210 can be rigid. The circuit board 210 can have a generally flat shape, such as a rectangle. The circuit board 210 can include a plurality of electrical connections to the respective visible light LEDs 215 and infrared LEDs 220 such that the visible light LEDs 215 and the infrared LEDs 220 can be independently controlled.

[0038] Each LED 215, 220 is a semiconductor device that emits electromagnetic radiation via electroluminescence when current flows through it. Each LED 215, 220 includes a lead frame having an anvil and posts (not shown). The lead frame is connected to an anode pin and a cathode pin. The anvil includes a semiconductor die that generates electromagnetic radiation inside a reflective cavity. The lead frame may be housed in an epoxy lens or housing. Each LED 215, 220 may emit electromagnetic radiation at a wavelength defined by the construction and / or material of the semiconductor die. The visible light LED 215 may include multiple wavelengths, e.g., red visible light LEDs 215, green visible light LEDs 215, and blue visible light LEDs 215 arranged in a regular pattern above the circuit board 210. The infrared LED 220 may have a wavelength in the near-infrared spectrum, e.g., 940 nanometers. The number of visible light LEDs 215 may be greater than the number of infrared LEDs 220. This relative ratio may allow the visible light LEDs 215 to have sufficient density to fill the available space of the display panel 205, while the infrared LEDs 220 can still generate sufficient infrared radiation to illuminate the occupant for the camera 115.

[0039] The display panel 205 may be the outermost rigid layer of the display unit 110, e.g., it may be the outermost layer or may have one or more films thereon. The display panel 205 may have a generally flat shape, e.g., rectangular. The display panel 205 is transparent to the wavelengths of electromagnetic radiation emitted by the visible light LEDs 215 and the infrared LEDs 220, e.g., the display panel 205 may be polycarbonate. The display panel 205 may protect the internal components of the display unit 110 from the surrounding environment.

[0040] Reference Figures 2 to 3 , the display unit 110 may have a side-lit arrangement. As Figure 2 shown, the display unit 110 may include, for example, a stack of layers sandwiched together in a direction from the inside of the instrument panel 135 to the outside of the instrument panel 135: a reflector 225, a light guide plate 230, a diffuser film 235, a vertical prism film 240, a horizontal prism film 245, and a display panel 205. In other words, the reflector 225, the light guide plate 230, the diffuser film 235, the vertical prism film 240, and the horizontal prism film 245 are fixed behind the display panel 205. The layers may have a rectangular shape having generally the same length and width (but not necessarily thickness), and the rectangular shapes may be aligned with each other. The layers are parallel to each other and adjacent to each other. The circuit board 210 is outside the stack of layers, i.e., it is not one of the layers in the stack.

[0041] Continuing reference Figure 2, the circuit board 210 is fixed relative to the display panel 205 and the other stacked layers, and the circuit board 210 is arranged to direct the emitted light from the visible light LED 215 and the infrared LED 220 to the display panel 205. Specifically, the circuit board 210 is arranged to direct the emitted light from the visible light LED 215 and the infrared LED 220 into the light guide plate 230 at the edge of the light guide plate 230. The visible light LED 215 and the infrared LED 220 are arranged on the circuit board 210 along the edge of the light guide plate 230. The light guide plate 230 is configured to reflect visible light and infrared radiation from the edge to which the circuit board 210 is attached along the entire length or width of the light guide plate 230 to the opposite edge. For example, the light guide plate 230 may be made of poly(methyl methacrylate) (PMMA).

[0042] Reference Figure 3 , the visible light LED 215 and the infrared LED 220 are arranged in rows along the edge of the light guide plate 230. The infrared LED 220 is interspersed with the visible light LED 215 along the row. For example, the visible light LED 215 and the infrared LED 220 may be arranged in a repeating linear pattern, for example, alternating between three visible light LEDs 215 and one infrared LED 220.

[0043] Reference Figures 4 to 5B , the display unit 110 may have a direct backlight arrangement. As Figure 4 shown, the display unit 110 may include, for example, a stack of layers sandwiched together in a direction from the inside of the instrument panel 135 to the outside of the instrument panel 135: the circuit board 210, the reflector 225, the diffuser 405, the diffusion sheet 410, the prism 415, and the display panel 205. In other words, the circuit board 210, the reflector 225, the diffuser 405, the diffusion sheet 410, and the prism 415 are fixed behind the display panel 205. The layers (including the circuit board 210) may have a rectangular shape having substantially the same length and width (but not necessarily the thickness), and the rectangular shapes may be aligned with each other. The layers are parallel to each other and adjacent to each other.

[0044] Continue to refer to Figure 4 , the circuit board 210 is fixed relative to the display panel 205 and the other stacked layers, and the circuit board 210 is arranged to direct the emitted light from the visible light LED 215 and the infrared LED 220 to the display panel 205. Specifically, the circuit board 210 is fixed behind the display panel 205 and arranged parallel to the display panel 205. The visible light LED 215 and the infrared LED 220 are mounted on the side of the circuit board 210 facing the display panel 205 and aimed perpendicular to the circuit board 210 and the display panel 205.

[0045] ReferenceFigures 5A to 5B ,The visible light LEDs 215 and the infrared LEDs 220 can be arranged on the circuit board 210 in a two-dimensional pattern. The infrared LEDs 220 are interspersed with the visible light LEDs 215 along the length and width of the circuit board 210. For example, the visible light LEDs 215 and the infrared LEDs 220 can be arranged in a repeating two-dimensional pattern. For example, the visible light LEDs 215 are in the form of a grid, and there is an infrared LED 220 adjacent to every other visible light LED 215 in the length and width directions of the circuit board 210, as shown in FIG. 5.

[0046] Returning to Figure 4 ,The display unit 110 can include a plurality of optical domes 420 that encapsulate the corresponding visible light LEDs 215. The optical domes 420 can be mounted to the circuit board 210 or the reflector 225 above the visible light LEDs 215 and the infrared LEDs 220. The optical domes 420 can be transparent to the wavelengths of the electromagnetic radiation emitted by the visible light LEDs 215 and the infrared LEDs 220. The optical domes 420 can protect the LEDs 215, 220, and can diffuse and direct the light emitted by the visible light LEDs 215 to completely cover the display panel 205.

[0047] Referring to Figure 5A ,Each optical dome 420 can encapsulate one of the visible light LEDs 215, and each visible light LED 215 can be accommodated in one of the optical domes 420; that is, the visible light LEDs 215 and the optical domes 420 can have a one-to-one relationship. The infrared LEDs 220 can also be encapsulated in the corresponding optical domes among the optical domes 420 next to the visible light LEDs 215. For example, a subset of the optical domes 420 can encapsulate the corresponding infrared LEDs 220. In other words, some of the optical domes 420 each encapsulate a visible light LED 215 and an infrared LED 220, and the remaining optical domes 420 each encapsulate a visible light LED 215 without encapsulating an infrared LED 220.

[0048] Alternatively, referring to Figure 5B ,Each optical dome 420 can encapsulate a single LED 215, 220, that is, a visible light LED 215 or an infrared LED 220. The visible light LEDs 215 and the optical domes 420 that accommodate the visible light LEDs 215 can be arranged in a two-dimensional pattern (such as a rectangular grid), and the infrared LEDs 220 and the optical domes 420 that accommodate the infrared LEDs 220 can be positioned in the same slots of the two-dimensional pattern adjacent to the corresponding visible light LEDs 215.

[0049] Referring to Figure 6, the vehicle system 105 includes a computer 600 on the vehicle 100. The vehicle computer 600 is a microprocessor-based computing device, such as a general-purpose computing device (including a processor and a memory, an electronic controller, etc.), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a combination of the foregoing, etc. Generally, a hardware description language such as VHDL (VHSIC (Very High Speed Integrated Circuit) Hardware Description Language) is used in electronic design automation to describe digital and mixed-signal systems such as FPGAs and ASICs. For example, an ASIC is manufactured based on VHDL programming provided before manufacturing, and the logic components inside an FPGA can be configured based on, for example, VHDL programming stored in a memory electrically connected to the FPGA circuit. Thus, the computer 600 may include a processor, a memory, etc. The memory of the computer 600 may include a medium for storing instructions executable by the processor and for electronically storing data and / or databases, and / or the computer 600 may include a structure such as the foregoing structure that provides programming. The computer 600 may be multiple computers coupled together.

[0050] The vehicle system 105 may include a communication network 605. The computer 600 may transmit and receive data through the communication network 605. The communication network 605 may be, for example, a controller area network (CAN) bus, Ethernet, WiFi, local interconnect network (LIN), on-board diagnostic connector (OBD-II), and / or any other wired or wireless communication network. The computer 600 may be communicatively coupled via the communication network 605 to the visible light LEDs 215 and infrared LEDs 220 of the display unit 110, the camera 115, the user interface 140, the ambient light sensor 610, and other components.

[0051] The vehicle system 105 may include an ambient light sensor 610. The ambient light sensor 610 is a photodetector that detects the amount of ambient light present (i.e., the total light level from sources in the environment). The ambient light sensor 610 may be any suitable type, such as a phototransistor, a photodiode, a photonic integrated circuit, etc.

[0052] The computer 600 is programmed to receive a visible light image from the camera 115. The visible light image is an image within the field of view of the camera 115, and the visible light image depicts an occupant in the occupant seating area 120. Each visible light image is a two-dimensional matrix of pixels. The brightness or color of each pixel is represented as one or more numerical values, e.g., a scalar unitless value of photometric light intensity between 0 (black) and 1 (white), or a value for each of red, green, and blue, e.g., each on an 8-bit scale (0 to 255) or a 12-bit or 16-bit scale. The pixels can be a mixture of representations, e.g., a repeating pattern of three pixels and a scalar value of the intensity of a fourth pixel having three numerical color values, or some other pattern. The position within the visible light image, i.e., the position within the field of view of the camera 115 when the visible light image is recorded, can be specified in pixel dimensions or coordinates, e.g., a pair of ordered pixel distances such as a plurality of pixels from the top edge of the visible light image and a plurality of pixels from the left edge of the visible light image.

[0053] The computer 600 is programmed to receive an infrared image from the camera 115. The infrared image is an image within the field of view of the camera 115, and the infrared image depicts an occupant in the occupant seating area 120 illuminated by the infrared LEDs 220. Each infrared image is a two-dimensional matrix of pixels. Each pixel has a brightness of an infrared wavelength represented as a numerical value, e.g., a scalar unitless value of the light intensity of the infrared wavelength between 0 (black) and 1 (white), or a value on an 8-bit scale (0 to 255) or a 12-bit or 16-bit scale. The position within the infrared image, i.e., the position within the field of view of the camera 115 when the infrared image is recorded, can be specified in pixel dimensions or coordinates, e.g., a pair of ordered pixel distances such as a plurality of pixels from the top edge of the infrared image and a plurality of pixels from the left edge of the infrared image.

[0054] The computer 600 is programmed to adjust the brightness of the infrared LEDs 220. The computer 600 can control the brightness of the infrared LEDs 220 by varying the pulse width modulation of the infrared LEDs 220; i.e., the computer 600 can increase the brightness by increasing the proportion of time the infrared LEDs 220 are turned on, thereby increasing the average current received by the infrared LEDs 220, and the computer 600 can decrease the brightness by decreasing the proportion of time the infrared LEDs 220 are turned on, thereby decreasing the average current received by the infrared LEDs 220. With pulse width modulation, the average current can be varied even if the voltage across the infrared LEDs 220 is constant.

[0055] The computer 600 is programmed to adjust the brightness of the infrared LED 220 based on the ambient brightness. The computer 600 can receive the ambient brightness from the ambient light sensor 610. The computer 600 can increase the brightness of the infrared LED 220 in response to an increase in the ambient brightness and decrease the brightness of the infrared LED 220 in response to a decrease in the ambient brightness. In other words, there can be a positive relationship between the ambient brightness and the brightness of the infrared LED 220. For example, the brightness of the infrared LED 220 can be a mathematical function of the ambient brightness, i.e., I inf = f(I amb ), where I inf is the brightness of the infrared LED 220 and I amb is the ambient brightness, such as a positive linear relationship, i.e., I inf = mI amb , where m is a slope selected experimentally such that the infrared image has high contrast at different ambient brightness levels. For another example, the brightness of the infrared LED 220 can be selected from multiple preset brightness levels (e.g., two) based on the ambient brightness. In the example of two preset brightness levels, the computer 600 can select the larger preset brightness in response to the ambient brightness being greater than a threshold, and the computer 600 can select the smaller preset brightness in response to the ambient brightness being less than the threshold. The threshold can be selected to correspond to the transition between day and night.

[0056] The computer 600 can be programmed to determine the status of an occupant based on a visible light image or an infrared image. The status can be a classification of the apparent alertness of the occupant, e.g., attention on the road, attention elsewhere, no discernible attention, etc. The computer 600 can use conventional image recognition techniques (e.g., a convolutional neural network programmed to receive an image of the occupant as input and output the recognized status) to identify the status of the occupant. The convolutional neural network includes a series of layers, where each layer uses the previous layer as input. Each layer contains multiple neurons that receive as input data generated by a subset of the neurons in the previous layer and generate an output that is sent to the neurons in the next layer. The types of layers include: a convolutional layer that computes the dot product of weights and input data in small regions; a pooling layer that performs a downsampling operation along a spatial dimension; and a fully connected layer that is generated based on the outputs of all the neurons in the previous layer. The last layer of the convolutional neural network generates a score for each potential occupant status, and the final output is the status with the highest score.

[0057] The computer 600 is programmed to actuate components based on visible light images or infrared images depicting the occupant from the camera 115. For example, the computer 600 can output a message to the occupant based on visible light images or infrared images depicting the occupant from the camera 115. The computer 600 can output a message in response to the occupant's state being in a first set of states (e.g., attention elsewhere or no distinguishable attention, as determined from the visible light image or infrared image). The possible states of the occupant can be classified into one of a first group or a second group, where the first group includes the possible states for which a message should be output, and the second group includes the possible states for which a message should not be output. If the computer 600 determines different states of the occupant based on the visible light image and the infrared image, the computer 600 can output a message in response to at least one of the determined states being in the first group of states (e.g., attention elsewhere or no distinguishable attention), even if the other determined state is in the second group of states (e.g., attention on the road). The computer 600 can command the user interface 140 to output the message. The message can be visual, auditory, and / or tactile. As an example, the message can be a ringing and text displayed on the display unit 110 or another screen of the user interface 140. For another example, the computer 600 can determine the eye gaze direction of the occupant based on visible light images or infrared images depicting the occupant from the camera 115, e.g., by executing a machine learning program trained to recognize eye gaze and taking the visible light image or the infrared image as input. The position of the eye gaze on the user interface 140 or the display unit 110 can be considered an input provided by the occupant for controlling components of the vehicle 100.

[0058] The computer 600 can be programmed to actuate components, such as outputting a message, based on visible light images or infrared images according to the ambient brightness. For example, based on the visible light image in response to the ambient brightness being higher than a threshold, and based on the infrared image when the ambient brightness is lower than the threshold. For example, the computer 600 can be programmed to output a message to the occupant based on the infrared image when the ambient brightness is lower than the threshold, and the computer 600 can be programmed to output a message based on the visible light image in response to the ambient brightness exceeding the threshold, and to suppress outputting a message to the occupant based on the visible light image in response to the ambient brightness being lower than the threshold. In other words, when the ambient brightness is higher than the threshold, the computer 600 can rely on the visible light image or the infrared image to determine the state of the occupant and output a message in response to the state being in the first group, and when the ambient brightness is lower than the threshold, the computer 600 relies only on the infrared image and not on the visible light image to determine the state of the occupant and output a message in response to the state being in the first group. Thus, when the ambient light is too dim to use the visible light image, the infrared image provides data for determining the state of the occupant, and when the visible light image is available, the infrared image provides additional data.

[0059] Figure 7 is a flowchart showing an exemplary process 700 for determining when to output a message to an occupant. The memory of computer 600 stores executable instructions for performing the steps of process 700 and / or can be programmed in a structure such as described above. As an overall overview of process 700, computer 600 receives the ambient brightness and adjusts the brightness of infrared LED 220 based on the ambient brightness. In response to the ambient brightness exceeding a threshold, computer 600 receives a visible light image and an infrared image and determines the state of the occupant based on the images. In response to the ambient brightness being below the threshold, computer 600 receives an infrared image and determines the state of the occupant based on the infrared image. In response to the determined state being in a first set of states, computer 600 outputs a message to the occupant. Process 700 continues as long as vehicle 100 remains powered on.

[0060] Process 700 begins at block 705, where computer 600 receives the ambient brightness from ambient light sensor 610 via communication network 605, as described above.

[0061] Next, at block 710, computer 600 adjusts the brightness of infrared LED 220 based on the ambient brightness, as described above.

[0062] Next, at decision block 715, computer 600 determines whether the ambient brightness exceeds a threshold. In response to the ambient brightness exceeding the threshold, process 700 proceeds to block 720. In response to the ambient brightness being below the threshold, process 700 proceeds to block 730.

[0063] At block 720, computer 600 receives a visible light image and an infrared image from camera 115 via communication network 605, as described above.

[0064] Next, at block 725, computer 600 determines the state of the occupant based on the visible light image and the infrared image, as described above. After block 725, process 700 proceeds to decision block 740.

[0065] At block 730, computer 600 receives an infrared image from camera 115 via communication network 605, as described above.

[0066] Next, at block 735, computer 600 determines the state of the occupant based on the infrared image, as described above. After block 735, process 700 proceeds to decision block 740.

[0067] At decision block 740, computer 600 determines whether the determined state of the occupant is in a first set of states, as described above. In response to the determined state being in a first set of states, process 700 proceeds to block 745. In response to the determined state being in a second set of states, process 700 proceeds to decision block 750.

[0068] In block 745, computer 600 outputs a message to the occupant by commanding user interface 140 to output a message via communication network 605, as described above. After block 745, process 700 proceeds to decision block 750.

[0069] In decision block 750, computer 600 determines whether vehicle 100 is still on. In response to vehicle 100 still being on, process 700 returns to block 705 to continue evaluating the occupant's status. In response to vehicle 100 being off, process 700 ends.

[0070] In general, the described computing systems and / or devices may employ any of a variety of computer operating systems, including but not limited to the following versions and / or varieties: Ford Applications; AppLink / Smart Device Link middleware; Microsoft Operating System; Microsoft Operating System; Unix operating system (e.g., the operating system released by Oracle Corporation of Redwood Shores, California) Operating System); AIX UNIX operating system released by International Business Machines Corporation of Armonk, New York; Linux operating system; Mac OSX and iOS operating systems released by Apple Inc. of Cupertino, California; BlackBerry operating system released by BlackBerry Limited of Waterloo, Canada; and Android operating system developed by Google Inc. and the Open Handset Alliance; or the CAR Infotainment Platform provided by QNX Software Systems Limited. Examples of computing devices include but are not limited to in-vehicle computers, computer workstations, servers, desktops, notebooks, laptop computers, or handheld computers, or some other computing system and / or device.

[0071] Computing devices typically include computer-executable instructions, where the instructions can be executed by one or more computing devices such as those listed above. Computer-executable instructions can be compiled or interpreted from computer programs created using a variety of programming languages and / or technologies, which alone or in combination include but are not limited to Java TM, C, C++, Matlab, Simulink, Stateflow, Visual Basic, Java Script, Python, Perl, HTML, etc. Some of these applications can be compiled and executed on virtual machines such as Java Virtual Machine, Dalvik Virtual Machine, etc. Generally, a processor (e.g., a microprocessor) receives instructions from, for example, a memory, a computer-readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored and transmitted using a variety of computer-readable media. Files in a computing device are typically a collection of data stored on a computer-readable medium such as a storage medium, random access memory, etc.

[0072] A computer-readable medium (also referred to as a processor-readable medium) includes any non-transitory (e.g., tangible) medium that participates in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of the computer). Such media can take many forms, including but not limited to non-volatile media and volatile media. Instructions can be transmitted through one or more transmission media, which include optical fibers, wires, wireless communications, including internal components that make up a system bus coupled to a processor of the computer. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, FLASH-EEPROM, any other memory chip or cartridge, or any other medium from which a computer can read.

[0073] The databases, data repositories, or other data stores described herein can include various mechanisms for storing, accessing / retrieving, and retrieving various data, including hierarchical databases, sets of files in a file system, application databases in a dedicated format, relational database management systems (RDBMSs), non-relational databases (NoSQL), graph databases (GDBs), etc. Each such data store is typically included within a computing device that employs a computer operating system such as one of those mentioned above, and is accessed via a network in any one or more of a variety of ways. A file system can be accessed from a computer operating system and can include files stored in various formats. In addition to languages for creating, storing, editing, and executing stored programs such as the PL / SQL language mentioned above, an RDBMS typically also employs the Structured Query Language (SQL).

[0074] In some examples, system components may be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) and stored on a computer-readable medium associated therewith (e.g., disks, memories, etc.). A computer program product may include such instructions stored on a computer-readable medium for performing the functions described herein.

[0075] In the drawings, like reference numerals indicate like elements. Additionally, some or all of these elements may be varied. With respect to the media, processes, systems, methods, heuristics, etc. described herein, it should be understood that while the steps of such processes, etc. have been described as occurring in a certain ordered sequence, such processes may be practiced by performing the steps in an order different from that described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. The operations, systems, and methods described herein should always be implemented and / or performed in accordance with applicable owner / user manuals and / or safety guidelines.

[0076] The present disclosure has been described in an illustrative manner, and it should be understood that the terms used are of a descriptive nature and not restrictive. The use of “responsive to,” “after determining,” etc. indicates a causal relationship and not merely a temporal relationship. Given the above teachings, many modifications and variations of the present disclosure are possible, and the present disclosure may be practiced in other ways than specifically described.

[0077] According to the present invention, there is provided a vehicle system having: a display unit including: a display panel; a circuit board fixed relative to the display panel; a plurality of visible light-emitting diodes (LEDs) mounted to the circuit board; and a plurality of infrared LEDs mounted to the circuit board and dispersed with the visible light LEDs; the circuit board being arranged to direct emitted light from the visible light LEDs and the infrared LEDs to the display panel; and a computer communicatively coupled to the display unit; the computer being programmed to: actuate a component based on an infrared image of an occupant illuminated by the infrared LEDs from a camera; and adjust the brightness of the infrared LEDs based on ambient brightness.

[0078] According to an embodiment, the computer is programmed to increase the brightness of the infrared LEDs in response to an increase in the ambient brightness.

[0079] According to an embodiment, the present invention further features the camera, wherein: the camera is configured to detect visible light and infrared light; and the computer is programmed to actuate the component based on a visible light image of the occupant from the camera.

[0080] According to an embodiment, the invention is further characterized in that the camera, wherein: the camera is configured to detect visible light and infrared light; and the computer is programmed to actuate the component based on a visible light image of the occupant from the camera in response to the ambient brightness exceeding a threshold value.

[0081] According to an embodiment, the computer is programmed to actuate the component based on the infrared image when the ambient brightness is lower than the threshold value.

[0082] According to an embodiment, the computer is programmed to actuate the component based on the infrared image when the ambient brightness is higher than the threshold value.

[0083] According to an embodiment, the computer is programmed to inhibit actuating the component based on the visible light image in response to the ambient brightness being lower than the threshold value.

[0084] According to an embodiment, the computer is programmed to adjust the brightness of the infrared LED by changing the pulse width modulation of the infrared LED.

[0085] According to an embodiment, the display unit includes a light guide plate fixed behind the display panel; and the circuit board is arranged to guide the emitted light from the visible light LED and the infrared LED into the light guide plate at an edge of the light guide plate.

[0086] According to an embodiment, the visible light LED and the infrared LED are arranged in rows along the edge of the light guide plate.

[0087] According to an embodiment, the number of the visible light LEDs is greater than the number of the infrared LEDs.

[0088] According to an embodiment, the circuit board is fixed behind the display panel and arranged parallel to the display panel.

[0089] According to an embodiment, the display unit includes a plurality of optical domes encapsulating corresponding visible light LEDs, and the infrared LED is encapsulated in a corresponding optical dome among the optical domes.

[0090] According to an embodiment, a subset of the optical domes encapsulates corresponding infrared LEDs.

[0091] According to an embodiment, the visible light LED and the infrared LED are arranged in a two-dimensional pattern on the circuit board.

[0092] According to an embodiment, the invention is further characterized in a dashboard, to which the display unit is mounted.

[0093] According to an embodiment, the present invention is further characterized by the camera, wherein the camera is spaced apart from the display unit.

[0094] According to the present invention, there is provided a display unit having: a display panel; a light guide plate fixed behind the display panel; a circuit board fixed relative to the display panel; a plurality of visible light emitting diodes (LEDs) mounted to the circuit board; and a plurality of infrared LEDs mounted to the circuit board and dispersed together with the visible light LEDs; the circuit board is arranged to guide emitted light from the visible light LEDs and the infrared LEDs into the light guide plate at an edge of the light guide plate.

[0095] According to an embodiment, the visible light LEDs and the infrared LEDs are arranged in rows along the edge of the light guide plate.

[0096] According to an embodiment, the number of the visible light LEDs is greater than the number of the infrared LEDs.

Claims

1. A vehicle system, comprising: A display unit, the display unit comprising: Display panel; A circuit board, wherein the circuit board is fixed relative to the display panel; a plurality of visible light emitting diodes (LEDs), the plurality of visible light LEDs mounted to the circuit board; and a plurality of infrared LEDs mounted to the circuit board and interspersed with the visible light LEDs; The circuit board is arranged to direct emitted light from the visible light LED and the infrared LED to the display panel; and a computer communicatively coupled to the display unit; The computer is programmed to: actuating a component based on an infrared image from a camera depicting an occupant illuminated by the infrared LED; and The brightness of the infrared LED is adjusted based on the ambient brightness. 2 . The vehicle system of claim 1 , wherein the computer is programmed to increase the brightness of the infrared LED in response to an increase in the ambient brightness.

3. The vehicle system of claim 1, further comprising the camera, wherein: The camera is configured to detect visible light and infrared light; and The computer is programmed to actuate the component based on a visible light image from the camera depicting the occupant.

4. The vehicle system of claim 1, further comprising the camera, wherein: The camera is configured to detect visible light and infrared light; and The computer is programmed to actuate the component based on a visible light image from the camera depicting the occupant in response to the ambient brightness exceeding a threshold. 5 . The vehicle system of claim 4 , wherein the computer is programmed to actuate the component based on the infrared image when the ambient brightness is below the threshold. 6 . The vehicle system of claim 5 , wherein the computer is programmed to actuate the component based on the infrared image when the ambient brightness is above the threshold. 7 . The vehicle system of claim 4 , wherein the computer is programmed to refrain from actuating the component based on the visible light image in response to the ambient brightness being below the threshold.

8. The vehicle system of claim 1, wherein: The display unit includes a light guide plate fixed behind the display panel; and The circuit board is arranged to guide the emission light from the visible light LED and the infrared LED into the light guide plate at an edge of the light guide plate. 9 . The vehicle system of claim 8 , wherein the visible light LEDs and the infrared LEDs are arranged in a row along the edge of the light guide plate.

10. The vehicle system of claim 1, wherein the number of the visible light LEDs is greater than the number of the infrared LEDs. 11 . The vehicle system according to claim 1 , wherein the circuit board is fixed behind the display panel and arranged in parallel with the display panel.

12. The vehicle system of claim 11, wherein the display unit includes a plurality of optical domes enclosing respective visible light LEDs, and the infrared LEDs are enclosed in respective ones of the optical domes.

13. The vehicle system of claim 12, wherein a subset of the optical domes enclose respective infrared LEDs.

14. The vehicle system according to one of claims 1 to 13, further comprising an instrument panel, the display unit being mounted to the instrument panel.

15. A display unit comprising: Display panel; A light guide plate, the light guide plate being fixed behind the display panel; A circuit board, wherein the circuit board is fixed relative to the display panel; a plurality of visible light emitting diodes (LEDs), the plurality of visible light LEDs mounted to the circuit board; as well as a plurality of infrared LEDs mounted to the circuit board and interspersed with the visible light LEDs; The circuit board is arranged to guide the emission light from the visible light LED and the infrared LED into the light guide plate at an edge of the light guide plate.

Images