Determination of the Distance between the Image Sensor and the Target Area

Abstract

A method for determining a distance D between an image sensor and a target area of a vehicle driver 传感器 The method includes: arranging a light source relative to the image sensor in a known geometric relationship such that a structure in the vehicle projects a shadow in the target area at least occasionally; determining a distance D between the light source and the target area based on an image of the target area including the shadow acquired by the image sensor and the geometric relationship (e.g., distance) between the light source and the structure 光源 ; and determining the distance D 光源 based on the distance D 传感器 and the geometric relationship between the light source and the image sensor. By this method, a reliable determination of the distance to the target area can be provided using only a single image sensor

Description

Field of the Invention

[0001] The present invention relates to determining the distance between a single image sensor and a target area, and more particularly to determining the distance between a single image sensor and a target area in an eye tracking system. Background of the Invention

[0003] Traditionally, in vehicle monitoring systems (such as vehicle occupant recognition devices or eye tracking systems), techniques for determining the distance from a sensor device to a vehicle occupant have focused on identifying the position of the body parts (e.g., the driver's head or face) of the vehicle occupant.

[0004] In most head or eye tracking systems, the user's face is illuminated with a light source having a central wavelength outside the visible spectrum, and an image of the face is acquired by one or several imaging sensors. In the case of two image sensors ("stereo imaging"), the distance to the face can be determined by triangulation. However, in a system with only one camera, it may be difficult to determine the distance to the face.

[0005] One method is to measure the interpupillary distance or other similar parameters (in the image) and estimate the distance based on the expected value of the parameter. A major problem with this distance determination method is the uncertainty in the estimate due to the natural distribution of the measured parameter, resulting in a distance estimation error of about 10%. Selecting different parameters, such as iris size or corneal curvature, can minimize the uncertainty in the measured value, but once the driver's eyes are covered, for example, by glasses, measuring these features is affected by light refraction and becomes unreliable.

[0006] Therefore, there is a need to improve the determination of the distance between the driver's face and a single image sensor.

[0007] General Disclosure of the Invention

[0008] The object of the present invention is to overcome the above problems and provide a feasible way to satisfactorily determine the distance between a single image sensor and a target area in a vehicle.

[0009] According to a first aspect of the present invention, this object and other objects are achieved by a method for determining the distance D between an image sensor and a target area of a vehicle driver 传感器 The method includes: arranging a point light source relative to the image sensor in a known geometric relationship such that the vehicle's steering wheel projects a shadow in the target area at least occasionally; determining the distance D between the light source and the target area based on the image of the target area including the shadow acquired by the image sensor and the geometric relationship (e.g., distance) between the light source and the steering wheel 光源 ; and based on the distance D 光源and determine the distance D based on the geometric relationship between the light source and the image sensor 传感器 .

[0010] According to a second aspect of the present invention, the object and other objects are achieved by a device for determining the distance D between an image sensor and a target area of a vehicle driver, the device comprising: a point light source arranged relative to the image sensor in a known geometric relationship and arranged such that at least occasionally the steering wheel of the vehicle projects a shadow in the target area; a processing circuit configured to determine the distance D between the light source and the target area based on an image of the target area including the shadow acquired by the image sensor and the geometric relationship between the light source and the steering wheel 传感器 and determine the distance D based on the distance D 光源 and the geometric relationship between the light source and the image sensor 光源 and determine the distance D based on the geometric relationship between the light source and the image sensor 传感器 .

[0011] The present invention is based on the fact that the steering wheel located between the light source and the target area will project a shadow, and if the geometric relationship between 1) the light source and the steering wheel and 2) the light source and the image sensor is known, this shadow can in turn be used to determine the distance between the light source and the target area.

[0012] By this method, it is possible to provide a reliable determination of the distance to the target area using only a single image sensor.

[0013] In principle, the known geometric relationship (e.g., distance) between the light source and the steering wheel enables the determination of a first distance between the light source and the target area based on the position and / or size of the shadow, and the known geometric relationship between the light source and the image sensor enables the conversion of this first distance into the required distance D 传感器 . (Of course, an explicit calculation of the first distance may not be necessary.)

[0014] Note that for illustrative purposes, the "known geometric relationship" between the light source and the steering wheel is referred to as "distance" in the following description. In many cases, this is also a correct description. However, in some cases, this is a simplification because the steering wheel may not be in the same plane as the target area (e.g., the face) and there is no single "distance". In such cases, the geometric relationship may be more complex than a scalar distance.

[0015] In some embodiments, the distance D can be determined by detecting the width of the shadow of a specific part of the steering wheel in the image and calculating the distance D as d×R / r 光源 to determine the distance D 光源 , where d is the distance between the light source and the steering wheel, r is the known width of this part, and R is the detected width.

[0016] In other embodiments, the distance D can be determined by detecting the position P of the shadow of a specific contour of the steering wheel in the image and calculating the distance D according to d×P / p 光源 to determine the distance D 光源 , where d is the distance between the light source and the steering wheel, p is the position of the contour relative to the optical axis of the light source, and P is the detected position, where both the position p and the detected position P are determined relative to the optical axis of the light source.

[0017] The typical design of a steering wheel with a rim and spokes and its rotational movement during driving make it highly suitable for the purposes of the present invention.

[0018] This part or contour can be part of a spoke of the steering wheel. One advantage of using a spoke is that the steering wheel is typically not laterally adjustable. Thus, the horizontal component of a radially extending spoke will not be affected by the up / down adjustment of the steering wheel.

[0019] The distance between the steering wheel and the image sensor can be pre-known or determined during an appropriate calibration process. For example, the steering wheel can be provided with reflective markers with known intervals such that the distance between the image sensor and the steering wheel can be detected based on the image acquired by the image sensor. Optionally, the distance between the image sensor and the steering wheel is determined based on the value of the geometric parameters identified in the image acquired by the image sensor and based on the pre-identified values of the same geometric parameters at a known distance.

[0020] In some embodiments, the geometry of the steering wheel (e.g., the width of the spokes) is desired for facilitating the determination of the distance. For example, such geometry can be known from CAD information related to the steering wheel of the vehicle. Optionally, in cases where such information is not available, the geometry of the steering wheel can be determined based on the image acquired by the image sensor and based on the known distance between the image sensor and the steering wheel.

[0021] In some embodiments, the angular position of the steering wheel is detected for facilitating the determination of the distance. If the geometry of the steering wheel and its angular position are known, the angle of the side of the spoke is known, which helps to identify the spoke shadow. Additionally, the position of the spoke is known, so it is not important to be able to acquire an image of the steering wheel. This allows for implementations where the image sensor cannot "see" the steering wheel.

[0022] Optionally, in some embodiments, the shadow pattern, the geometry of the steering wheel, the distance between the steering wheel and the image sensor, or the angular position of the steering wheel can be determined by providing appropriate outputs of measured values (e.g., geometric parameters identified in the acquired image) as a function of time to an artificial neural network that is trained according to changes in the geometric dimensions and positions of the steering wheel. Brief Description of the Drawings

[0024] The present invention will be described in more detail with reference to the drawings which illustrate currently preferred embodiments of the invention.

[0025] Figure 1 is a schematic diagram of an eye tracking system according to an embodiment of the present invention.

[0026] Figure 2a is Figure 1 a side view of the eye tracking system in.

[0027] Figure 2b is Figure 2a a schematic diagram of the geometric relationship in.

[0028] Figure 3a is Figure 1 another schematic diagram of the eye tracking system in.

[0029] Figure 3b is Figure 3a a schematic diagram of the geometric relationship in.

[0030] Figure 4 is a flowchart of a method according to an embodiment of the present invention.

[0031] Detailed Description of the Preferred Embodiments

[0032] In the following detailed description, some embodiments of the present invention will be described. However, it is to be understood that the features of different embodiments are interchangeable between the embodiments and can be combined in different ways, unless any other situation is specifically indicated. Even though many specific details are set forth in the following description to provide a more thorough understanding of the present invention, it will be apparent to those skilled in the art that the present invention can be practiced without these specific details.

[0033] Basic and conventional techniques in electronics, sensor systems, image analysis, signal processing, data communication systems, image acquisition systems, and other components implementing the present invention are considered to be readily understood by those skilled in the art, and thus further description and details will be omitted in this description for the sake of brevity.

[0034] In other instances, well-known structures or functions are not described in detail so as not to obscure the present invention.

[0035] Figure 1 , Figure 2a , Figure 3a shows an eye tracking system 1 for detecting a user's eyes. The system includes a light source 2, an image sensor 3, and a processing circuit 4. The image sensor 3 is configured to acquire an image of a target area, here the face 5 of the user 6. The processing circuit is configured to identify and detect the eyes 7 of the user 6.

[0036] Although a single light source is shown in Figure 1 , Figure 2a , Figure 3a more than one light source may be provided. For example, for the purpose of reducing / eliminating reflections, it may be advantageous to illuminate the target area alternately with at least two light sources. The light source 2 may be any suitable type of light source, including but not limited to light-emitting diodes (LEDs) operating in different wavelength domains (preferably having a central wavelength in the infrared (IR) or near-IR part of the spectrum), eye-safe laser sources with diverging lenses, etc. In this preferred embodiment, the light source 2 is an IR LED. To eliminate or minimize interference from ambient light, the light source is used in combination with a band-pass filter having a passband centered at the central IR wavelength. The central wavelength may be in the near-IR region, such as 840 nm or 940 nm. This filter has a passband capable of capturing most of the light emitted by the light source but at the same time blocking most of the ambient light.

[0037] The image sensor 3 may be a camera or a photodetector, such as a semiconductor image sensor, a photodetector, a thermal detector, a PIN diode or an avalanche diode. The camera 3 may also be a charge-coupled device (CCD) sensor array or a complementary metal-oxide-semiconductor (CMOS) sensor array, etc. In the case of a narrow-band light source such as the IR LED mentioned above, the image sensor 3 may be provided with a band-pass filter (not shown) having a passband corresponding to the spectrum of the light source 2.

[0038] In addition, the light source 2 and the image sensor 3 may be arranged such that the emitted signals are synchronously demodulated to filter out ambient IR noise, thereby improving the quality of the captured image even when using a low-intensity IR LED. The IR light emitted from the LED may be a continuous wave within a predetermined time period or pulsed light at a predetermined frequency. The sensitivity and resolution of the image sensor 3 may be selected such that the recorded image is suitable for further data processing such as image processing, or the recorded image is used as an input for machine vision algorithms.

[0039] The light source 2 and the image sensor 3 may be arranged to be time-synchronized. The processing circuit 4 may be configured to operate a computer vision system and image processing algorithms for processing and analyzing the images acquired from the image sensor 3.

[0040] In addition, the processing circuit 4 may be configured to perform other functions such as determining the position of the user's head. For example, when the driver moves his or her head, the processing circuit 4 may be configured to use the position information to determine the direction of the head movement and calculate the change in the distance between the target area 7 and the image sensor 4. The processing circuit 4 is also configured to send and / or receive data between different components of the vehicle or a remote data center. In addition, in this case, the processing circuit 4 is connected to the IR image sensor 3 and other components via a wired connection. However, it is conceivable to use a wireless connection instead of a wired connection to access remote components.

[0041] The steering wheel 8 is located between the light source and the face 5 such that the steering wheel projects a shadow 9 on the face 5.

[0042] The processing circuit may also be connected to receive various sensor inputs for processing. For example, information about the current steering wheel angle may be provided to the processing circuit.

[0043] It should be understood that although the light source 2, the image sensor 3, and the processing circuit 4 are shown as separate components in this example, this may or may not be the case in different examples. Other combinations are readily imaginable for those skilled in the art, such as integrating the light source 2 or the processing circuit 4 with the image sensor 3, or a remote processing circuit 4 being arranged to wirelessly receive data from the image sensor 3, etc. Thus, in different examples, some or all of these components may form part of a single unit.

[0044] Referring to FIG. 2, a specific part of the steering wheel 8 (here the rim 20) is used to determine the distance D 光源 . In this example, the specific part 20 has a known width "r", and the distance between the LED 2 and the steering wheel 8 is "d". When illuminated by the LED 2, the rim 20 forms a corresponding shadow part 22 with a width of "R" on the target area 7.

[0045] In this embodiment, the processing circuit 4 is arranged to detect and identify the size of the projected shadow part 22 in the acquired image. The incident light from the LED 2 is projected onto the rim 20 and the corresponding shadow 22, forming two similar triangles △abc~△ade, as Figure 2b shown. By using the triangle proportionality theorem, it is determined that the corresponding heights and sides of the two similar triangles are proportional to each other, and in this example, the processing circuit 4 determines the distance D according to 光源 D = d×R / r to determine the distance D 光源 .

[0046] In Figure 3aIn the embodiments herein, the position "p" of a specific contour (here, the edge 31 of the spoke 32 of the steering wheel 8) is determined relative to the optical axis A of the LED 2. The position "P" of the portion 34 of the shadow 9 corresponding to the position p is detected by the processing circuit 4 in the acquired image. As in Figure 2a the distance between the LED 2 and the steering wheel 8 is labeled as d.

[0047] Similarly, and with reference to Figure 3b , two similar triangles △apc~aPe are formed, where c and e are located on the optical axis A. If the positions p and P are represented as the normal distances to the optical axis A, then the distance D can be determined as D 光源 = d×P / p 光源 . It should be understood that it is also easy to think of determining the distance D by selecting other similar triangles 光源 .

[0048] The operation of the system 1 in Figure 4 will be described below with reference to the flowchart in Figure 1 , Figure 2a , Figure 3a .

[0049] In step S1, the light source 2 is arranged such that the shadow of the steering wheel 8 is projected onto the face 5 of the user 6. In some mounting configurations, a portion of the steering wheel 8 will always be located between the light source 2 and the face 5. In other mounting configurations, the shadow may only be visible for certain positions of the steering wheel.

[0050] In step S2, the image sensor 3 acquires one or several images including the face 5 with the shadow 9 of the steering wheel 8. The acquired images are provided to the processing circuit 4 for appropriate image processing.

[0051] In steps S3 - S5, the processing circuit 4 determines the distance between the image sensor 3 and the face 5 based on the relationships discussed above. First, in step S3, the processing circuit 4 extracts the width d and / or the position p of the shadow 9. For example, by providing the captured shadow of the steering wheel as input to an image processing algorithm, the corresponding pixel information can be extracted from the captured image. The extracted pixel information can include pixel brightness or color, pixel size, or pixel position on the target area, which can in turn be used to calculate the size or position of the shadow. Information about the current steering wheel angle can help identify the contour of the shadow, such as the edge of the spoke, which will have an inclination related to the steering wheel angle.

[0052] Then, in step S4, the information from S3 is used to determine the distance D between the light source 2 and the target area 5 光源Finally, using the known geometric relationship between the light source 2 and the image sensor 3, the distance D between the image sensor 3 and the target area 7 can be correspondingly calculated in step S5. 传感 device.

[0053] Note that Figure 4 the process in [reference] usually only needs to be executed once for each driving session. After the initial determination of the distance D 传感器 is made, continuous distance measurements can be made using some invariant parameters (such as iris size). By measuring this parameter at the initially determined distance, any change in the distance can be detected by monitoring this parameter.

[0054] Those skilled in the art will recognize that the present invention is in no way limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims. For example, the details of the eye tracking system disclosed herein are not critical to the operation of the present invention.

Claims

1. A method for determining a distance D between an image sensor and a target area of a driver of a vehicle 传感器 ​ Comprising: Arranging a point light source relative to the image sensor in a known geometric relationship such that the steering wheel of the vehicle projects a shadow in the target area at least occasionally; Based on the image acquired by the image sensor of the target area including the shadow and the geometric relationship between the point light source and the steering wheel, determine the distance D between the point light source and the target area 光源 ; And Based on the distance D 光源 and the known geometric relationship between the point light source and the image sensor, determine the distance D 传感器 , wherein, the distance D 光源 is determined by the following: Detecting the width (R) of the shadow of a specific part of the steering wheel in the image, and According to D 光源 Calculate the distance D according to D = d×R / r 光源 , where d is the known distance between the point light source and the steering wheel, r is the known width of the part, and R is the detected width Alternatively, the distance D 光源 is determined by the following: Detecting the position P of the shadow of a specific contour of a part of the steering wheel in the image, and According to D 光源 Calculate the distance D according to D = d×P / p 光源 , where d is the known distance between the point light source and the steering wheel, p is the known position of the contour, and P is the detected position Wherein, the position p and the detected position P are both determined relative to the optical axis of the point light source.

2. The method according to claim 1, Wherein, The part is the rim or spoke of the steering wheel.

3. The method according to claim 1 or 2, further Comprising: Determining the distance between the image sensor and the steering wheel, and Based on the distance between the image sensor and the steering wheel and the known geometric relationship between the point light source and the image sensor, determining the distance d between the point light source and the steering wheel.

4. The method according to claim 3, Wherein, The distance between the image sensor and the steering wheel is determined based on the value of the geometric parameter recognized in the image acquired by the image sensor and the pre-recognized value of the geometric parameter at a known distance.

5. The method according to claim 3, Wherein, Based on the positions of a set of markers in the image acquired by the image sensor, determining the distance between the image sensor and the steering wheel, the markers being arranged at predetermined positions on the steering wheel.

6. The method according to any one of claims 1-2 and 4-5, Wherein, The method further comprises determining the geometric structure of the steering wheel according to the image acquired by the image sensor and based on the distance between the image sensor and the steering wheel.

7. The method according to claim 3, Wherein, The method further comprises determining the geometric structure of the steering wheel according to the image acquired by the image sensor and based on the distance between the image sensor and the steering wheel.

8. The method according to any one of claims 1-2, 4-5 and 7, Wherein, The method further comprises detecting the angular position of the steering wheel.

9. The method according to claim 3, Wherein, The method further comprises detecting the angular position of the steering wheel.

10. The method according to claim 6, Wherein, The method further comprises detecting the angular position of the steering wheel.

11. An apparatus for determining a distance D between an image sensor and a target area of a driver of a vehicle 传感器 and Comprising: A point light source, the point light source being arranged relative to the image sensor in a known geometric relationship and being arranged such that the steering wheel of the vehicle projects a shadow in the target area at least occasionally, A processing circuit, the processing circuit being configured to: Determine the distance D between the point light source and the target area based on the image acquired by the image sensor of the target area including the shadow and the geometric relationship between the light source and the steering wheel 光源 , and Based on the distance D 光源 and the known geometric relationship between the point light source and the image sensor, determine the distance D 传感器 , wherein, the distance D 光源 is determined by the following items: Detect the width (R) of the shadow of a specific part of the steering wheel in the image, and According to D 光源 Calculate the distance D according to D = d×R / r 光源 , where d is the known distance between the point light source and the steering wheel, r is the known width of the part, and R is the detected width Alternatively, the distance D 光源 is determined by the following: Detect the position P of the shadow of a specific contour of a part of the steering wheel in the image, and According to D 光源 Calculate the distance D according to D = d×P / p 光源 , where d is the known distance between the point light source and the steering wheel, p is the known position of the contour, and P is the detected position Wherein, the position p and the detected position P are both determined relative to the optical axis of the point light source.

12. The device according to claim 11, Wherein, The device further comprises an angular sensor for detecting the angular position of the steering wheel.

Images