Driver monitoring camera

Abstract

The invention relates to a driver monitoring camera comprising an optical sensor configured to take an image of a driver of a vehicle, a tilt sensor configured to determine a tilt of the driver monitoring camera with respect to a vehicle coordinate system, and a processor configured to determine a driver pose in a driver monitoring camera coordinate system from the image and configured to transform the driver pose in the driver monitoring camera coordinate system into a driver pose in the vehicle coordinate system using the determined tilt. The invention further relates to a vehicle comprising the driver monitoring camera.

Description

Technical Field

[0001] This invention relates to a driver monitoring camera. Background Technology

[0002] In vehicles, driver monitoring cameras can be used to monitor the driver and determine, for example, whether the driver is distracted or fatigued, by assessing their posture. A good place to mount a driver monitoring camera is the steering column, as drivers typically adjust the steering wheel so that their heads fall into the field of view of a camera mounted to the steering column. However, this means that the position of the driver monitoring camera changes when the driver adjusts the steering wheel. This needs to be taken into account when determining the driver's posture.

[0003] Image analysis can be used to calculate the camera's tilt angle relative to world coordinates (in this case, vehicle coordinates) via a transformation matrix. After adjusting the steering column tilt angle, image analysis of the environmental components in the camera coordinate system is correlated with the estimated tilt angle.

[0004] However, image analysis relies on known and predictable features, such as straight edges projected onto the image plane; otherwise, inaccuracy will be high. Therefore, the lack of interior vehicle landmarks limits the ability of driver monitoring cameras to determine their position relative to the vehicle's coordinates as they move. One such case is in convertibles where cameras are removed from the roof, thus limiting the number of interior landmarks—since most landmarks are located on the roof. Furthermore, image analysis for driver monitoring to determine camera position is computationally intensive (high computational resource consumption, especially when image analysis needs to be performed in the absence of clear vehicle landmarks). Since it is unknown when the driver will adjust the steering column tilt angle, camera position determination typically needs to be run continuously, and therefore continuously consumes computational resources. Summary of the Invention

[0005] Therefore, the object of this invention is to provide an efficient method that allows for taking into account changes in the position of a driver monitoring camera in the absence of vehicle landmarks.

[0006] According to the present invention, this objective is achieved by a vehicle cabin sensing system comprising: an optical sensor configured to capture an image of the driver of the vehicle; a tilt sensor configured to determine the tilt of a driver monitoring camera relative to a vehicle coordinate system; and a processor configured to determine the driver's posture in the driver monitoring camera coordinate system based on the image, and configured to transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system using the determined tilt.

[0007] According to one embodiment, the processor is configured to perform control actions based on the driver's posture in the vehicle coordinate system.

[0008] According to one embodiment, the optical sensor is configured to capture a series of images of the driver of a vehicle, wherein a tilt sensor is configured to monitor the tilt of the driver monitoring camera relative to the vehicle coordinate system and repeatedly indicate the tilt of the driver monitoring camera relative to the vehicle coordinate system to a processor, and wherein the processor is configured to determine the driver's posture in the driver monitoring camera coordinate system based on each of the series of images, and is configured to transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system using a first tilt indicated by the tilt sensor until the tilt sensor indicates a second tilt that differs from the first tilt by more than a predetermined threshold, and then transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system using the second tilt.

[0009] According to one embodiment, the sensor is configured to indicate the second tilt only if the difference between the second tilt and the first tilt exceeds a predetermined threshold.

[0010] According to one embodiment, the processor is configured to determine a transformation matrix based on a determined tilt angle, and use the transformation matrix to transform the driver's pose in the driver monitoring camera coordinate system to the driver's pose in the vehicle coordinate system.

[0011] According to one embodiment, the sensor is configured to trigger an update of the rotation matrix by signaling to the driver to monitor the tilt of the camera relative to the vehicle coordinate system.

[0012] According to one embodiment, the processor is configured to determine a transformation matrix based on a determined tilt angle, and to use the transformation matrix to transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system, wherein the processor is configured to update the current transformation matrix if the tilt angle change from a first tilt angle (on which the processor has determined the current transformation matrix) to a second tilt angle determined by a tilt angle sensor is higher than a predetermined threshold.

[0013] According to one embodiment, the driver monitoring camera includes a housing that holds the optical sensor and the tilt sensor.

[0014] According to one embodiment, the housing also accommodates the processor.

[0015] According to one embodiment, a vehicle is provided that includes a driver monitoring camera as described above.

[0016] According to one embodiment, a driver monitoring camera is mounted on the steering column of the vehicle.

[0017] According to one embodiment, the vehicle is a convertible. Attached Figure Description

[0018] The invention will now be described in more detail with reference to the accompanying drawings:

[0019] Figure 1 The vehicle is shown.

[0020] Figure 2 A driver monitoring camera is shown.

[0021] Figure 3 It shows Figure 2 The driver monitoring camera is placed on the steering column.

[0022] Figure 4 The output of the sensor signal representing the tilt angle of the driver's monitoring camera is shown. Detailed Implementation

[0023] Figure 1 Vehicle 100 is shown.

[0024] Driver 101 is driving vehicle 100.

[0025] For various reasons, it may be desirable to monitor driver 101. For example, it may be desirable to detect when the driver is drowsy or when driver 101 is distracted (e.g., using a mobile phone while driving). Monitoring driver 101 can specifically determine the driver 101's posture (position and orientation) (i.e., the position and orientation of the driver's body parts, such as arms and head). This can also be used for vehicle adjustments, such as the automatic positioning of the driver's headrest.

[0026] To this end, a driving monitoring system may be provided, which includes a driver monitoring camera (DMC) 102 that captures images of the driver 101 and, for example, allows determination of the driver's posture. Posture may include the driver 101 raising the phone to his ear or turning his head (and thus being distracted), or the driver lowering his head (which may be a sign of driver drowsiness).

[0027] The DMC 102 is mounted in the vehicle 102 in a position where the driver's eyes and face are visible during operation and are minimally obstructed by other components in their line of sight, such as the steering wheel 103. One potential location is on the steering column 104, on which the steering wheel 103 is mounted. The steering column 104 is typically adjustable in two directions: tilting up and down, and translating toward and away from the driver (here referred to as toward and away, respectively).

[0028] The image of the driver captured by the DMC 102 depends in particular on the tilt of the DMC 102, and therefore, if the DMC 102 is mounted on the adjustable steering column 104, it depends on the position of the steering column 104.

[0029] According to various embodiments, a tilt sensor (also referred to as a tilter or inclinometer) is incorporated into the DMC 102. This allows the position of the steering column 104 to be taken into account when monitoring the driver.

[0030] Figure 2 The driver monitoring camera 200 is shown.

[0031] The driver monitoring camera (DMC) 200 includes an optical module and a processing unit. The optical module consists of components required for imaging, including a lens group 201 and an image sensor (i.e., an optical sensor) 202 mounted on a camera printed circuit board (PCB) 203, which may also hold other components such as illuminators.

[0032] The processing unit includes components required for processing and analyzing images acquired by the optical module, including a microprocessor 204 on the processor PCB 205, which can hold other components similar to memory and additional components required to support camera functions, such as power supply, regulator and driver.

[0033] The optical module PCB 203 and the processor PCB 205 each include corresponding connectors 206 and 207, which are interconnected via connectors 206 and 207, for example by electrically coupling the wires or flexible circuit connection structure 208 of the two connectors 206 and 207.

[0034] exist Figure 2 In one embodiment, the tilter 209 is mounted on the processor PCB 205.

[0035] The tilt sensor 209 is a sensor that can measure the angle of the measured plane relative to a reference plane (e.g., in this case, the ground plane or a horizontal plane in the vehicle coordinate system). Although this embodiment shows the tilt sensor 209 as a separate and independent component, it can also be included as a feature in a sensor module, for example, placed on the camera PCB 203.

[0036] exist Figure 2In one embodiment, the tilter 209 is connected to the microprocessor 204 and sends the tilt value as a signal to the microprocessor 204. Signal filtering can be performed to eliminate or reduce the effects of noise and jitter on the signal. This can be implemented in the sensor module containing the tilter 209, or in the microprocessor 204 via hardware or software. Filtering may, for example, include considering variations in the determined tilt only if the determined tilt is sufficiently high (above, for example, a predetermined threshold of several degrees).

[0037] Figure 3 It shows Figure 2 The driver monitoring camera is placed on the steering column 301.

[0038] Steering column 301 corresponds, for example, to steering column 104. Steering wheel 302 (for example, corresponding to steering wheel 103) is attached to steering column 301.

[0039] The driver monitoring camera is housed in a camera housing 303 attached to the steering column 301.

[0040] Therefore, when the driver 304 tilts the steering column 301, the driver monitoring camera tilts along with the steering column 301, and the tilter 209 outputs a corresponding sensor signal representing (and thus indicating) the tilt angle.

[0041] Figure 4 Figure 401 shows the output of the sensor signal representing the tilt angle, denoted by α, of the driver monitoring camera 402. The pivot point 403 is located here at the left end of the driver monitoring camera 402, but it can also depend on the position of the driver monitoring camera housing 303 on the steering column 301.

[0042] Therefore, with the aid of the tilter 209, the driver monitoring camera 200 (specifically, the microprocessor 204) identifies its position and orientation relative to the world coordinate system (in this case, the vehicle coordinate system). The microprocessor 204 represents the position and orientation of the driver monitoring camera relative to the world coordinate system (i.e., the camera coordinate system) using camera extrinsic parameters (e.g., in matrix form). The camera coordinate system is transformed to the world coordinate system using the camera extrinsic parameters. Thus, the microprocessor 204 can detect features characterizing the driver's posture in the camera coordinate system (based on images captured by the driver monitoring camera) and then map them to the vehicle coordinate system.

[0043] Camera extrinsic parameters include, for example, a rotation matrix, which specifies the orientation of the camera (and therefore the camera coordinate system fixed to the camera) in the vehicle coordinate system. These extrinsic parameters are stored in memory, for example, and are used by the microprocessor 204 according to various algorithms for purposes such as determining the facial orientation of the driver 304 in the vehicle coordinate system after determining the camera coordinate system based on image analysis. The microprocessor 204 can use the camera extrinsic parameters in combination with other parameters (e.g., camera intrinsic parameters) to form a camera matrix, which is then used to map features in the camera coordinate system to the vehicle coordinate system.

[0044] Since frequent tilting of the steering column is not desired, the microprocessor 204 can be configured (e.g., by running corresponding software) to update the rotation matrix only when certain conditions are met. For example, an update is triggered when a change in tilt has occurred exceeding a threshold (since the last update of the rotation matrix), such that the rotation matrix is ​​determined to be not overly sensitive and, for example, not to be triggered by noise or vibration.

[0045] The method described above allows driver monitoring systems to determine the driver's posture in the vehicle coordinate system even in the absence of interior vehicle landmarks (e.g., in a convertible with the roof removed), because tilt detection (i.e., determination of camera tilt) does not rely on the analysis of known features (e.g., those located on the roof). Furthermore, this method eliminates the need for computationally intensive image analysis to detect these features and determine their locations through image analysis.

[0046] These can also be applied to vehicles with a fixed roof, where additional signals from the vehicle to notify the camera of its position are unavailable. This may be due to the lack of sensors on the steering column, steering wheel, or related components capable of detecting or measuring changes in position and angle.

[0047] List of reference numerals

[0048] 100 vehicles

[0049] 101 Driver

[0050] 102 Driver monitoring camera

[0051] 103 Steering Wheel

[0052] 104 Steering Column

[0053] 200 driver monitoring cameras

[0054] 201 Lens Group

[0055] 202 Image Sensor

[0056] 203 Camera PCB

[0057] 204 microprocessors

[0058] 205 Processor PCB

[0059] 206 and 207 connectors

[0060] 208 Connection Structure

[0061] 209 Inclinator

[0062] 301 Steering Column

[0063] 302 Steering Wheel

[0064] 303 Camera Housing

[0065] 304 Driver

[0066] 401 Chart

[0067] 402 Driver Surveillance Camera

[0068] 403 Pivot point.

Claims

1. Driver monitoring camera (102, 200), including: An optical sensor (202) is configured to capture an image of the driver of the vehicle (100); Tilt sensor (209) configured to determine the tilt of driver monitoring camera (102, 200) relative to vehicle coordinate system; as well as The processor (204) is configured to determine the driver's pose in the driver monitoring camera coordinate system from the image, and is configured to transform the driver's pose in the driver monitoring camera coordinate system to the driver's pose in the vehicle coordinate system using the determined tilt angle. The driver monitoring camera (102, 200) is mounted on the steering column (104, 301) of the vehicle (100). The processor (204) is configured to determine a transformation matrix based on a determined tilt angle, and to use the transformation matrix to transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system. The processor (204) is also configured to update the current transformation matrix if the change from a first tilt angle to a second tilt angle determined by a tilt sensor (209) exceeds a predetermined threshold. The processor (204) determines the current transformation matrix based on the first tilt angle. The optical sensor (202) is configured to capture a series of images of the driver of the vehicle (100), wherein the tilt sensor (209) is configured to monitor the tilt of the driver monitoring camera (102, 200) relative to the vehicle coordinate system and repeatedly indicate the tilt of the driver monitoring camera (102, 200) relative to the vehicle coordinate system to the processor (204), wherein the processor (204) is configured to determine the driver's posture in the driver monitoring camera coordinate system based on each of the series of images, and is configured to transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system using a first tilt indicated by the tilt sensor (209) until the tilt sensor (209) indicates a second tilt that differs from the first tilt by more than a predetermined threshold, and then transform the driver's posture in the driver monitoring camera coordinate system to the driver's posture in the vehicle coordinate system using the second tilt.

2. The driver monitoring camera (102, 200) according to claim 1, wherein, The processor (204) shown is configured to perform control actions based on the driver's posture in the vehicle coordinate system.

3. The driver monitoring camera (102, 200) according to claim 1 or 2, wherein, The sensor is configured to indicate the second tilt only when the difference between the second tilt and the first tilt is higher than a predetermined threshold.

4. The driver monitoring camera (102, 200) according to claim 1 or 2, wherein, The processor (204) is configured to determine a transformation matrix based on the determined tilt angle, and to use the transformation matrix to transform the driver's pose in the driver monitoring camera coordinate system to the driver's pose in the vehicle coordinate system.

5. The driver monitoring camera (102, 200) according to claim 4, wherein, The sensor is configured to trigger an update of the rotation matrix by signaling to the driver the tilt of the surveillance camera (102, 200) relative to the vehicle coordinate system.

6. The driver monitoring camera (102, 200) of claim 1 or 2, comprising a housing for holding the optical sensor (202) and the tilt sensor (209).

7. The driver monitoring camera (102, 200) according to claim 6, wherein, The housing also serves to hold the processor (204).

8. A vehicle (100), wherein, The vehicle (100) includes a driver monitoring camera according to any one of claims 1 to 7.

9. The vehicle (100) according to claim 8, wherein, The vehicle (100) is a convertible.

Images