Controlling infrared (IR) cameras without passing messages between them to have non-overlapping illumination

Abstract

A system, such as a driver monitoring system (DMS) for a vehicle, includes a first infrared (IR) camera, a second IR camera, a sensor, and a controller. The first camera is configured to output first IR light, such as to illuminate a driver of the vehicle with the first IR light, so as to be used by the first camera to monitor the driver. The second camera is configured to output second IR light, such as to illuminate the driver with the second IR light, so as to be used by the second camera to monitor the driver. The sensor is configured to detect the first IR light being output by the first camera. The controller is configured to control the second camera so that it does not output the second IR light when the sensor detects the first IR light being output by the first camera.

Description

Technical Field

[0001] This disclosure relates to a plurality of infrared (IR) cameras controlled to have non-overlapping IR illumination. Background Technology

[0002] Infrared (IR) cameras use infrared light to monitor targets. In use, the IR camera's IR emitter outputs infrared light to illuminate the target, and the IR camera detects the infrared light reflected from the target, thus monitoring the target. Infrared light enhances visibility in low-light or nighttime conditions, making IR cameras ideal for continuous monitoring.

[0003] The vehicle's Driver Monitoring System (DMS) includes an infrared (IR) camera. The IR camera is oriented towards the driver, who is seated inside the vehicle. The IR camera uses infrared light to track and monitor the driver's body and behavioral cues. For example, the IR camera can track and monitor the driver's eye movements, head position, and facial expressions.

[0004] In addition to the DMS's IR camera, the vehicle may also have one or more other IR cameras to monitor other occupants and / or areas within the vehicle compartment.

[0005] The requirement for using multiple IR cameras in environments such as vehicle cabins is that the IR cameras cannot use infrared light simultaneously. That is, the IR cameras cannot have overlapping or simultaneous IR illumination and cannot output IR light at the same time. This requirement may be even more stringent when multiple IR cameras are monitoring the same target.

[0006] One option to meet this requirement is to have the IR camera controlled by a common controller that controls the IR camera to use infrared light in turn. This option is unavailable if the IR camera cannot be controlled by such a common controller.

[0007] Another option to meet this requirement is to allow the IR cameras to communicate with each other, transmitting messages such as those instructing on infrared light usage schedules, and allowing the IR cameras to determine when to take turns using infrared light based on these message transmissions. For example, in a vehicle, the IR cameras could be connected to the vehicle bus and communicate via that bus. Therefore, this option is not available when the IR cameras are not connected to the vehicle bus and / or are otherwise unable to transmit messages between them. Summary of the Invention

[0008] A system includes: a first infrared (IR) camera configured to output first IR light; a second IR camera configured to output second IR light; a first sensor configured to detect the first IR light being output by the first camera; and a controller configured to control the second camera to not output second IR light when the first sensor detects the first IR light being output by the first camera.

[0009] The system may further include: a second sensor configured to detect second IR light being output by a second camera; and a second controller configured to control the first camera to not output first IR light when the second sensor detects second IR light being output by the second camera.

[0010] The first sensor may be located remotely from both the first and second cameras, and / or the second sensor may be located remotely from both the first and second cameras. Alternatively, the first sensor may be an imaging device of the second camera, and / or the second sensor may be an imaging device of the first camera.

[0011] The system may further include: a third camera configured to output a third IR light; and a third controller configured to control the third camera to not output the third IR light when the first sensor detects (i) the first IR light being output by the first camera or (ii) the second IR light being output by the second camera.

[0012] The first and second cameras can be oriented to output their IR light toward different targets or toward a common target.

[0013] The first IR light can be a first IR light pulse sequence, and the second IR light can be a second IR light pulse sequence. The controller can also be configured to detect the pulse rate of the first IR light pulse sequence based on the first IR light output by the first camera detected by the first sensor camera, and control the second camera to make the second IR light pulse sequence have a pulse rate different from that of the first IR light pulse sequence.

[0014] The first and second cameras may not communicate directly with each other. The controllers may not communicate directly with each other either.

[0015] A system for monitoring a driver of a vehicle includes first and second infrared (IR) cameras, a first sensor, and a controller. The first camera is configured to illuminate the driver of the vehicle with first IR light for monitoring the driver. The second camera is configured to illuminate the driver with second IR light for monitoring the driver. The first sensor is configured to detect the presence of the first IR light. The controller is configured to control the second camera to refrain from illuminating the driver with the second IR light when the first sensor detects the presence of the first IR light.

[0016] The system may further include: a second sensor configured to monitor the presence of a second IR light; and a second controller configured to control a first camera to prevent the driver from being illuminated by the first IR light when the second sensor detects the presence of the second IR light.

[0017] The first sensor may be a first photodiode oriented toward the first camera, and the second sensor may be a second photodiode oriented toward the second camera.

[0018] The first and second cameras can communicate with each other without any vehicle bus between them. The controller can also communicate with each other without any vehicle bus.

[0019] One method includes: using a first sensor to detect first IR light being output by a first infrared (IR) camera; and using a controller to control a second IR camera so that it does not output second IR light when the first sensor detects the first IR light being output by the first camera.

[0020] The method may further include: using a second sensor to detect second IR light being output by a second camera; and using a second controller to control a first camera so that it does not output first IR light when the second sensor detects second IR light being output by the second camera.

[0021] The method may also include using a third controller to control a third IR camera so that it does not output third IR light when (i) the first sensor detects first IR light being output by the first camera or (ii) the second sensor detects second IR light being output by the second camera. Attached Figure Description

[0022] Figure 1 A functional block diagram of a vehicle is shown, which includes a driver monitoring system (DMS) with an infrared (IR) camera and a camera system that also has an IR camera; and

[0023] Figure 2 This is a flowchart illustrating the operation of the DMS and camera system when monitoring the driver of a vehicle. Detailed Implementation

[0024] This document discloses detailed embodiments of the present disclosure; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure that can be implemented in various and alternative forms. The accompanying drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show details of components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for instructing those skilled in the art to employ the present disclosure in different ways.

[0025] Now for reference Figure 1 The diagram shows a functional block diagram of vehicle 10. For example... Figure 1 The vehicle 10 depicted includes an automobile. However, vehicle 10 may include any number of other types of mobile platforms. Vehicle 10 typically includes a body 14, front wheels 16, rear wheels 18, and a chassis 20. The body 14 is mounted on the chassis 20 and substantially surrounds the components of vehicle 10. Wheels 16 and 18 are rotatably coupled to the chassis 12 near corresponding corners of the body 14. The body 14 includes a vehicle compartment 22. The driver and any other occupants of vehicle 10 sit within the vehicle compartment 22 during operation of the vehicle. Vehicle 10 also includes a propulsion system 24, a transmission system 26, and a steering system 28. The transmission system 26 transmits power from the propulsion system 24 to the wheels 16 and 18. The steering system 24 affects the position of the wheels 16 and 18.

[0026] The vehicle 10 also includes a display system 30. The display system 30 is operable to display information for occupants sitting in the vehicle compartment 22 to view and listen to.

[0027] Vehicle 10 also includes a Driver Monitoring System (DMS) 12. DMS 12 is operable to monitor the physical and behavioral cues of the driver seated within the vehicle compartment 22 and determine the driver's state based on the monitored driver information. DMS 12 is configured to perform actions based on the driver's state. Such actions include generating an alarm on display 30 to alert the driver and / or other occupants to the driver's state.

[0028] To monitor the driver, the DMS 12 includes an infrared (IR) camera 32. The camera 32 is positioned inside the vehicle compartment 22 and pointed towards the driver. The camera 32 uses infrared light to monitor the driver. More specifically, the IR emitter of the camera 32 outputs infrared light to illuminate the driver. The camera 32 detects the infrared light reflected from the driver, thereby monitoring the driver. In this way, the camera 32 can be used to track and monitor the driver's eye movements, head position, facial expressions, etc., so that the DMS 12 can determine the driver's state accordingly.

[0029] DMS 12 also includes a controller 34. The controller 34 is operable to control the camera 32 to output infrared light. That is, the controller 34 can control when the camera 32 outputs infrared light and when it does not. For example, the controller 34 is operable to set a schedule for the camera 32 to output infrared light. The schedule could be to continuously output infrared light for a defined duration, to output a sequence of infrared light pulses, wherein the pulse sequence has a defined pulse rate and pulse duration, etc.

[0030] The controller 34 is also operable to analyze reflected infrared light detected by the camera 32 to detect the driver's state. The controller 34 is also operable to execute actions of the DMS 12 in response to the detected driver state. For example, the controller 34 may be operable to cause the display 30 to generate an alarm to alert the driver and / or other occupants to the driver's state, as determined by the DMS.

[0031] Vehicle 10 also includes an infrared camera system 36. Camera system 36 includes one or more infrared cameras 40a, 40b, and 40n. Cameras 40a, 40b, and 40n are positioned within vehicle compartment 22 to point at the same or different targets. Targets can be occupants other than the driver within vehicle compartment 22. Targets can be a specific part of vehicle compartment 22. Targets can be the driver. Cameras 40a, 40b, and 40n use infrared light to detect their targets in a manner similar to how camera 32 of DMS 12 uses infrared light to detect the driver.

[0032] Camera system 36 includes at least one infrared camera positioned inside vehicle compartment 22 to be pointed at the driver and to monitor the driver using infrared light. For simplicity, camera system 36 will be considered as having only one camera for monitoring the driver (i.e., only one of cameras 40a, 40b, 40n is configured to monitor the driver), and this camera will be designated as camera 40.

[0033] Vehicle 10 also includes a controller 42 associated with camera system 36. Controller 42 is operable to control camera 40 to output infrared light. That is, controller 42 can control when camera 40 outputs infrared light and when it does not output infrared light. For example, controller 42 is operable to set a schedule for camera 40 to output infrared light. This schedule could be to continuously output infrared light for a certain duration, to output a sequence of infrared light pulses, wherein the pulse sequence has a certain pulse rate and pulse duration, etc. Controller 42 is also operable to analyze the reflected infrared light detected by camera 40 to detect the driver's state and cause an action to be performed based on the driver's state detected by camera 40.

[0034] Controllers 34 and 42 each include a data storage device that stores data for use by the controller when controlling the corresponding camera. The data storage device can be any type of direct-access memory and / or other memory device. Controllers 34 and 42 each also include a processor, a communication bus, and a computer-readable storage device or medium. The processor performs the computational and control functions of its controller. The processor can be any custom or commercially available processor, central processing unit (CPU), graphics processing unit (GPU), or an auxiliary processor among several processors associated with its controller. The computer-readable storage device or medium can include volatile and non-volatile storage devices capable of storing data, some of which represents executable instructions for use by its controller to control the camera. The bus is used to transmit sensor signals and control signals between the respective group of controllers and cameras.

[0035] like Figure 1 As shown, DMS 12 and camera system 36 are independent systems. Each system includes an IR camera and a controller. That is, DMS 12 includes camera 32 and controller 34, and camera system 36 includes camera 40 and controller 42. Since cameras 32 and 40 are part of independent systems, there is no means of direct communication between them, and there is also no means of direct communication between controllers 34 and 42.

[0036] Here, the camera 32 and controller 34 of DMS 12 may be referred to as "first camera" 32 and "first controller" 34; and the camera 40 and controller 42 of camera system 36 may be referred to as "second camera" 40 and "second controller" 42.

[0037] The requirement for using the first camera 32 and the second camera 40 to monitor the driver of vehicle 10 is that the cameras do not simultaneously output IR light. That is, the first camera 32 and the second camera 40 cannot have overlapping or simultaneous IR illumination and cannot output IR light at the same time.

[0038] According to this disclosure, the requirement is met by providing IR light sensors that monitor the IR light output by cameras 32 and 40 and communicate with controllers 34 and 42 so that the controllers know when the cameras output IR light, enabling the controllers to control the cameras so that only one camera outputs IR light at any given time.

[0039] For details, please refer to Figure 1The second controller 42 has an associated sensor 44. Sensor 44 is configured to monitor IR light output by the first camera 32. That is, sensor 44 is configured to detect the presence of IR light from the first camera 32. Thus, sensor 44 detects when the first camera 32 outputs IR light and when the first camera does not output IR light. Since sensor 44 monitors the first camera 32, sensor 44 may be referred to herein as the "first sensor" 44.

[0040] Similarly, the first controller 34 has an associated sensor 46. Sensor 46 is configured to monitor IR light output by the second camera 40. That is, sensor 46 is configured to detect the presence of IR light from the second camera 40. Thus, sensor 46 detects when the second camera 40 outputs IR light and when the second camera does not output IR light. Since sensor 46 monitors the second camera 40, sensor 46 may be referred to herein as the "second sensor" 46.

[0041] Sensors 44 and 46 can be photodetectors, photodiodes, etc. Therefore, in this case, the first sensor 44 is located away from the first camera 32, and the second sensor 46 is located away from the second camera 40. Alternatively, sensors 44 and 46 can be imaging devices for other cameras not being monitored. That is, the first sensor 44 can be an imaging device for the second camera 40; and / or the second sensor 46 can be an imaging device for the first camera 32.

[0042] The second controller 42 communicates with the first sensor 44 to determine when the first camera 32 outputs IR light. In general, based on the information monitored by the first sensor 44, the second controller 42 can determine when the first camera 32 illuminates the driver with IR light, i.e., when the first camera outputs IR light. More specifically, based on the monitored information, the second controller 42 can determine the specific properties of the IR light output by the first camera 32, including pulse rate and pulse duration (if the output IR light is a sequence of IR light pulses).

[0043] The second controller 42 controls the second camera 36 so that it does not output IR light when the first camera 32 is outputting IR light. Therefore, when monitoring the driver of the vehicle 10, the second camera 40 will not cause the first and second cameras to output IR light simultaneously.

[0044] Similarly, the first controller 34 communicates with the second sensor 46 to determine when the second camera 40 outputs IR light. Therefore, the first controller 34 can determine when the second camera 40 illuminates the driver with IR light, i.e., when the second camera outputs IR light. The first controller 34 controls the first camera 32 to not output IR light when the second camera 40 is outputting IR light. Therefore, when monitoring the driver of vehicle 10, the first camera 32 will not cause simultaneous IR light output by both the first and second cameras.

[0045] Now for reference Figure 2 and constantly refer to Figure 1 A flowchart 50 is shown depicting the operation of the DMS 12 and camera system 36 when monitoring the driver of vehicle 10. This operation includes illuminating a target (e.g., the driver of vehicle 10) with IR light output from the first camera 32, as shown in box 52. The operation continues until the first sensor 44 detects that the first camera 32 is outputting IR light, as shown in box 54. Then, the operation continues until the second controller 42 controls the second camera 40 to not output IR light when the first sensor 44 detects that the first camera 32 is outputting IR light, as shown in box 56.

[0046] The operation also includes enabling the second camera 40 to illuminate the target with IR light, as shown in box 58. The operation continues with the second controller 42 controlling the second camera 40 to illuminate the target with IR light, as shown in box 60. Then, the operation continues with the second sensor 46 detecting that the second camera 40 is outputting IR light, as shown in box 62; and the first controller 34 controlling the first camera 32 to not output IR light when the second sensor 46 detects that the second camera 40 is outputting IR light, as shown in box 64.

[0047] As previously mentioned, one scenario involves a vehicle with two cameras, both capable of actively irradiating light with infrared (IR). These two cameras are not connected via a bus network, in which they can exchange relevant messages for establishing a timing relationship. The first camera may be designated as the master camera due to its functional safety implications. This master camera does not transmit any messages to the communication bus or similar means indicating actual exposure or IR flash events. The second camera is intended to operate at a specific point in time while the first camera is operating. IR illumination from the second camera is not permitted to overlap with IR illumination from the first camera. That is, the second camera must operate without causing any cross-interference with the first camera. More generally, IR illumination from the second camera is not permitted to overlap with IR illumination from the other camera.

[0048] As previously described, this disclosure describes methods for desynchronizing cameras. Methods for desynchronizing cameras include detection. For example, introducing an IR sensing element (e.g., an IR photodiode) to detect IR pulses (IR light) from a first camera, and / or using an imager from a second camera to detect IR pulses from the first camera. Methods for desynchronizing cameras may also include determining the pulse rate of the IR pulses from the first camera, such as via interrupt-based or algorithm-based processing.

[0049] Although exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms of this disclosure. Rather, the terminology used herein is descriptive rather than restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. Furthermore, features of various implementations may be combined to form other embodiments of this disclosure.

Claims

1. A system comprising: A first camera, configured to output a first infrared (IR) light; A second camera, configured to output a second IR light; A first sensor, configured to detect the first IR light being output by the first camera; as well as A controller configured to control the second camera to not output the second IR light when the first sensor detects the first IR light being output by the first camera.

2. The system of claim 1, further comprising: A second sensor is configured to detect the second IR light being output by the second camera; as well as A second controller is configured to control the first camera to not output the first IR light when the second sensor detects the second IR light being output by the second camera.

3. The system as claimed in claim 1, wherein: The first sensor is located at a position far away from both the first camera and the second camera.

4. The system as claimed in claim 1, wherein: The first sensor is the imaging device of the second camera.

5. The system as claimed in claim 2, wherein: The first sensor is located at a position far away from both the first camera and the second camera, and / or the second sensor is located at a position far away from both the first camera and the second camera.

6. The system of claim 2, wherein: The first sensor is the imaging device of the second camera, and / or the second sensor is the imaging device of the first camera.

7. The system of claim 2, further comprising: A third camera, configured to output a third IR light; as well as A third controller is configured to control the third camera to not output the third IR light when the first sensor detects (i) the first IR light being output by the first camera or (ii) the second IR light being output by the second camera.

8. The system of claim 1, wherein: The first camera is oriented to output the first IR light toward the target; and The second camera is oriented to output the second IR light toward different targets.

9. The system of claim 1, wherein: The first camera is oriented to output the first IR light toward the target; and The second camera is oriented to output the second IR light toward the same target.

10. The system of claim 1, wherein: The first IR light is a first IR light pulse sequence; The second IR light is a second IR light pulse sequence; The controller is further configured to detect the pulse rate of the first IR light pulse sequence based on the first IR light output from the first camera detected by the first sensor; and The controller is also configured to control the second camera such that the second IR light pulse sequence has a pulse rate different from the pulse rate of the first IR light pulse sequence.

11. The system of claim 1, wherein: The first camera and the second camera do not communicate directly with each other.

12. The system of claim 2, wherein: The controllers do not communicate directly with each other.

13. A system for monitoring the driver of a vehicle, the system comprising: A first camera is configured to illuminate the driver of the vehicle with a first infrared (IR) light so that the first camera can be used to monitor the driver. A second camera is configured to illuminate the driver with a second IR light so that the second camera can be used to monitor the driver; A first sensor, configured to detect the presence of the first IR light; as well as A controller configured to control the second camera to prevent the driver from being illuminated by the second IR light when the first sensor detects the presence of the first IR light.

14. The system of claim 13, further comprising: A second sensor, configured to detect the presence of the second IR light; as well as A second controller is configured to control the first camera to prevent the driver from being illuminated by the first IR light when the second sensor detects the presence of the second IR light.

15. The system of claim 14, wherein: The first sensor is a first photodiode oriented toward the first camera, and the second sensor is a second photodiode oriented toward the second camera.

16. The system of claim 13, wherein: The first camera and the second camera do not communicate with each other via any vehicle bus.

17. The system of claim 13, wherein: The controllers do not communicate with each other via any vehicle bus.

18. A method comprising: The first IR light being emitted by the first infrared (IR) camera is detected using the first sensor; as well as The controller controls the second IR camera so that it does not output second IR light when the first sensor detects the first IR light being output by the first camera.

19. The method of claim 18, further comprising: The second sensor is used to detect the second IR light being output by the second camera; as well as The first camera is controlled by the second controller so that it does not output the first IR light when the second sensor detects the second IR light being output by the second camera.

20. The method of claim 19, further comprising: The third IR camera is controlled by a third controller so that it does not output third IR light when (i) the first sensor detects the first IR light being output by the first camera or (ii) the second sensor detects the second IR light being output by the second camera.

Images