Deviation amount detection method, deviation amount detection device, and driver monitoring system

Abstract

To suppress degradation in calculation accuracy of a deviation amount following a state change of environmental light.SOLUTION: A deviation amount detection method comprises the steps of: making an illumination device 10 irradiate a structure 2 of a vehicle with light for suppressing influence of environmental light being light that enters the vehicle from the outside of the vehicle; making an imaging device 1 image the structure 2 irradiated with the light by the illumination device 10; and making a deviation amount detection device 3 calculate a deviation amount between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 on the basis of the image of the structure 2 imaged by the imaging device 1.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a deviation amount detection method, a deviation amount detection device, and a driver monitoring system. [Background technology]

[0002] A vehicle equipped with a driver monitoring system is equipped with a camera that captures images of occupants. The installation position of the camera may be shifted from the correct installation position due to a work error by a camera installer, etc. The correct installation position of the camera is, for example, the installation position indicated by the design value. If the installation position of the camera is shifted from the correct installation position, for example, the occupant in the image captured by the camera may become unclear, and as a result, the monitoring accuracy of the driver monitoring system may be degraded.

[0003] There is a deviation amount detection device that detects the amount of deviation between the installation position of an on-board camera attached to a vehicle and the correct installation position of the on-board camera (see Patent Document 1). In this deviation amount detection device, an electronic control unit extracts feature points of an object captured in an image taken by the on-board camera and calculates the amount of deviation from the feature points. Examples of the object include a window frame, a vehicle seat, or a headrest. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-181209 Summary of the Invention [Problem to be solved by the invention]

[0005] Changes in the state of ambient light, which is light entering the vehicle from outside, can cause changes in the color, brightness, or shadow of an object captured in an image captured by a camera. Changes in the color, etc. of an object can cause changes in the appearance of the object's shape. In the displacement amount detection device disclosed in Patent Document 1, the appearance of the shape of an object captured in an image changes as the ambient light changes, which can degrade the accuracy with which the electronic control unit extracts feature points of the object.This degradation in the accuracy with which the electronic control unit calculates the displacement amount can also degrade, which is a problem.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a deviation amount detection method and deviation amount detection device that can suppress deterioration in the accuracy of calculating the deviation amount due to changes in the ambient light conditions. [Means for solving the problem]

[0007] The deviation amount detection method according to the present disclosure includes a step in which a lighting device irradiates a structure of the vehicle with light to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle; a step in which an imaging device photographs the structure illuminated with light by the lighting device; and a step in which a deviation amount detection device calculates the amount of deviation between the installation position of the imaging device and the correct installation position of the imaging device based on an image of the structure photographed by the imaging device. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to suppress deterioration in the accuracy of calculating the amount of deviation due to changes in the state of ambient light. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to a first embodiment. [Figure 2] 2 is a hardware configuration diagram showing the hardware of a deviation amount detection device 3 according to the first embodiment. FIG. [Figure 3] 1 is a hardware configuration diagram showing hardware of a driver monitoring system 4 according to a first embodiment. [Figure 4] FIG. 10 is a hardware configuration diagram of a computer in the case where components of the deviation amount detection device 3 other than the illumination device 10 are realized by software, firmware, or the like. [Figure 5] Figure 5A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 5B is an explanatory diagram showing an image captured when the imaging device 1 is installed at an angle in the vehicle height direction. [Figure 6] Figure 6A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 6B is an explanatory diagram showing an image captured when the imaging device 1 is installed at an angle in the vehicle width direction. [Figure 7] Figure 7A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 7B is an explanatory diagram showing an image captured when the imaging device 1 is installed in a rotated state within the XY plane. [Figure 8] 4 is a flowchart showing a method for detecting a deviation amount according to the first embodiment. [Figure 9] FIG. 2 is an explanatory diagram showing an example of a captured image GP. [Figure 10] Figure 10A is an explanatory diagram showing a light source 10a whose light-emitting surface is installed in a position that makes contact with a window frame a, which is a structure 2, and Figure 10B is an explanatory diagram showing a light source 10a whose light-emitting surface is installed in a position that is away from the window frame a, which is a structure 2. [Figure 11] FIG. 11A is an explanatory diagram showing a state in which the shadow of an object caused by ambient light is cast inside the window frame a, and FIG. 11B is an explanatory diagram showing a captured image GP in which the shadow of an object caused by ambient light is reflected. [Figure 12] FIG. 12A is an explanatory diagram showing a state in which the lighting device 10 prevents shadows of objects caused by ambient light from entering the inside of the window frame a, and FIG. 12B is an explanatory diagram showing a captured image GP in which shadows of objects caused by ambient light are not reflected. [Figure 13] FIG. 10 is an explanatory diagram showing an example of light that creates a pattern on an irradiated surface. [Figure 14] FIG. 10 is an explanatory diagram showing an example in which light from a light source 10a is blinking. [Figure 15] 10 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to a second embodiment. FIG. [Figure 16] Figure 16A is an explanatory diagram showing a reflector 10c whose reflective surface is installed in a position where it comes into contact with a window frame a, which is a structure 2, and Figure 16B is an explanatory diagram showing a reflector 10c whose reflective surface is installed in a position away from the window frame a, which is a structure 2. [Figure 17] FIG. 10 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to a third embodiment. [Figure 18] FIG. 10 is a hardware configuration diagram showing the hardware of a deviation amount detection device 3 according to a third embodiment. [Figure 19] 1 is an explanatory diagram showing an example in which light-emitting points 60a, 60b, and 60c of a lighting device 60 are attached to the inside of a window frame a. [Figure 20] 10 is a flowchart showing a method for detecting a deviation amount according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0011] Embodiment 1 FIG. 1 is a configuration diagram showing a deviation amount detecting device 3 and a driver monitoring system 4 according to the first embodiment. FIG. 2 is a hardware configuration diagram showing the hardware of the deviation amount detection device 3 according to the first embodiment. FIG. 3 is a hardware configuration diagram showing the hardware of the driver monitoring system 4 according to the first embodiment.

[0012] In FIG. 1, the imaging device 1 is installed, for example, on the center console, steering column, A-pillar, rearview mirror, dashboard, instrument panel, or ceiling of a vehicle. The imaging device 1 is a camera that captures an image of a structure 2 inside a vehicle, a driver of the vehicle, etc. The camera may be, for example, a visible light camera or an infrared camera. The imaging device 1 outputs a captured image of a structure 2 to a displacement amount detection device 3, and outputs a captured image of a vehicle driver or the like to a driver monitoring system 4. The captured image output from the imaging device 1 to the deviation amount detection device 3 is an image used by the deviation amount detection device 3 when detecting the amount of deviation between the installation position of the imaging device 1 and the correct installation position. Therefore, the captured image is, for example, an image captured by the imaging device 1 when the imaging device 1 is attached to a vehicle. The captured image output from the imaging device 1 to the driver monitoring system 4 is an image that the driver monitoring system 4 uses to detect the driver's inattentive driving, etc. Therefore, the captured image is, for example, an image captured by the imaging device 1 while the driver is driving the vehicle.

[0013] The structure 2 is, for example, a window frame, an assist grip, a headrest, a B-pillar, a door frame, a shoulder anchor, or a sun visor of a vehicle. In the first embodiment, the captured image of the imaging device 1 is used in the driver monitoring system 4, so the imaging device 1 needs to be installed so that, for example, the face of the driver of the vehicle is captured when the driver is seated in the driver's seat, and the structure 2 is captured when the driver is not seated in the driver's seat. If the captured image of the imaging device 1 is not used in the driver monitoring system 4, the imaging device 1 does not need to be installed so that the face of the driver is captured.

[0014] The deviation amount detection device 3 detects the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1. The correct installation position of the imaging device 1 is, for example, the installation position indicated by the design value. The deviation amount detection device 3 outputs the deviation amount to the driver monitoring system 4.

[0015] The driver monitoring system 4 acquires the captured image from the imaging device 1 and acquires the amount of deviation from the deviation amount detection device 3. The driver monitoring system 4 monitors the state of the vehicle occupant based on the amount of deviation and the captured image. The state of the occupant may be, for example, the driver's inattentive driving or the driver's drowsy driving. For example, if the driver monitoring system 4 detects that the driver is looking away from the road or is drowsy at the wheel, it outputs warning information indicating a warning to the driver to an electronic control unit or the like of the vehicle.

[0016] In the first embodiment, an example will be described in which the driver monitoring system 4 detects the driver's inattentive driving or the driver's drowsy driving. However, the driver monitoring system 4 is not limited to detecting the driver's inattentive driving, etc., and may also detect the behavior of vehicle occupants other than the driver, for example. In this case, the vehicle occupants are photographed by the imaging device 1.

[0017] The deviation amount detection device 3 includes an illumination device 10, a position detection unit 11, and a deviation amount calculation unit 12. The lighting device 10 is installed on the exterior of a vehicle. The lighting device 10 includes a light source 10a that irradiates the structure 2 with light for suppressing the influence of ambient light, which is light that enters the vehicle from outside. The light source 10a is a surface light source, such as a display that emits white light or a display that emits infrared light. 1, it is assumed that the brightness of the light emitting surface of the light source 10a is uniform. Uniform brightness means, for example, that the difference between the maximum and minimum luminance measurement values on the light emitting surface of the surface light source is within ±10%. 1, the light source 10a is a surface light source. However, this is merely an example, and the light source 10a may be, for example, a point light source. 1 includes an illumination device 10. However, this is merely an example, and the illumination device 10 may be provided outside the misalignment amount detection device 3.

[0018] The position detection unit 11 is realized by, for example, a position detection circuit 21 shown in FIG. The position detection unit 11 acquires an image of the structure 2 captured by the imaging device 1. The position detection unit 11 detects the structure image position, which is the position of the structure 2 on the image, and outputs structure image position information indicating the structure image position to the displacement amount calculation unit 12.

[0019] The deviation amount calculation unit 12 is realized by, for example, a deviation amount calculation circuit 22 shown in FIG. The deviation amount calculation unit 12 acquires structure image position information from the position detection unit 11. The deviation amount calculation unit 12 also acquires, from an external source, reference image position information indicating a reference image position, which is the correct position on the image of the structure when the installation position of the imaging device 1 is correct. The deviation amount calculation unit 12 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position based on the reference image position indicated by the reference image position information and the structure image position indicated by the structure image position information. The deviation amount calculation unit 12 outputs the calculated deviation amount to the driver monitoring system 4.

[0020] The driver monitoring system 4 includes a sensing unit 31 and a warning output unit 32. The sensing unit 31 is realized by, for example, a sensing circuit 41 shown in FIG. The sensing unit 31 acquires the captured image from the imaging device 1 and acquires the amount of deviation from the deviation amount calculation unit 12 of the deviation amount detection device 3. The sensing unit 31 monitors the state of the vehicle occupant based on the amount of displacement and the captured image. The sensing unit 31 monitors the state of the occupants, and detects, for example, whether the driver is looking away from the vehicle or whether the driver is drowsy at the wheel.

[0021] The warning output unit 32 is realized by, for example, a warning output circuit 42 shown in FIG. If the sensing unit 31 detects, for example, that the driver is looking away from the road or is drowsy at the wheel, the warning output unit 32 outputs warning information indicating a warning to the driver to an electronic control unit of the vehicle or the like.

[0022] 1, it is assumed that the position detection unit 11 and the deviation amount calculation unit 12, which are components of the deviation amount detection device 3 other than the illumination device 10, are each realized by dedicated hardware as shown in Fig. 2. In other words, it is assumed that a part of the deviation amount detection device 3 is realized by a position detection circuit 21 and a deviation amount calculation circuit 22. Each of the position detection circuit 21 and the deviation calculation circuit 22 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0023] The components of the deviation amount detection device 3 other than the lighting device 10 are not limited to those realized by dedicated hardware, and the components other than the lighting device 10 may be realized by software, firmware, or a combination of software and firmware. Software or firmware is stored as a program in the memory of a computer. A computer refers to hardware that executes the program, such as a CPU (Central Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

[0024] FIG. 4 is a hardware configuration diagram of a computer in the case where the components of the deviation amount detection device 3 other than the illumination device 10 are realized by software, firmware, or the like. When the components of the deviation amount detection device 3 other than the lighting device 10 are realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the position detection unit 11 and the deviation amount calculation unit 12 is stored in the memory 51. Then, a processor 52 of the computer executes the program stored in the memory 51.

[0025] 2 shows an example in which the components of the misalignment detection device 3 other than the illumination device 10 are realized by dedicated hardware, and Fig. 4 shows an example in which the components of the misalignment detection device 3 other than the illumination device 10 are realized by software, firmware, or the like. However, this is merely an example, and some of the components of the misalignment detection device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, or the like.

[0026] FIG. 5A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and FIG. 5B is an explanatory diagram showing an image captured when the imaging device 1 is installed at an angle in the vehicle height direction. 5A and 5B, the X axis is parallel to the vehicle width direction, the Y axis is parallel to the vehicle height direction, and the Z axis is parallel to the vehicle front-rear direction. The driver's face is captured in each of the captured images in Figures 5A and 5B. If the installation position of the imaging device 1 is correct, the driver's face will be captured in the correct position in the captured image. In the example of Figure 5A, the correct position is approximately the center of the captured image, and the driver's face is captured in approximately the center of the captured image. If the imaging device 1 is installed with an angle in the vehicle height direction, the driver's face will appear in a position shifted in the vehicle height direction from the correct position in the captured image, as shown in FIG. 5B. If the angle in the vehicle height direction is large, the driver's face will appear to be facing upward or downward, which may make it difficult for the driver monitoring system 4 to detect the driver's distracted driving, etc.

[0027] Figure 6A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 6B is an explanatory diagram showing an image captured when the imaging device 1 is installed at an angle in the vehicle width direction. 6A and 6B, the X-axis, Y-axis, and Z-axis are the same as those in FIGS. 5A and 5B, respectively. The driver's face is captured in each of the captured images in Figures 6A and 6B. If the installation position of the imaging device 1 is correct, the driver's face will be captured in the correct position in the captured image. In the example of Figure 6A, the correct position is approximately the center of the captured image, and the driver's face is captured in approximately the center of the captured image. If the imaging device 1 is installed with an angle in the vehicle width direction, the driver's face will appear in a position shifted in the vehicle width direction from the correct position in the captured image, as shown in FIG. 6B. If the angle in the vehicle width direction is large, the driver's face will appear to be facing left or right, which may make it difficult for the driver monitoring system 4 to detect the driver's distracted driving, etc.

[0028] Figure 7A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 7B is an explanatory diagram showing an image captured when the imaging device 1 is installed in a rotated state within the XY plane. 7A and 7B, the X-axis, Y-axis, and Z-axis are the same as those in FIGS. 5A and 5B, respectively. The captured images in Figures 7A and 7B show the driver's face. If the installation position of the imaging device 1 is correct, the driver's face will be captured in the correct position in the captured image without rotation. In the example of Figure 7A, the correct position is approximately the center of the captured image, and the driver's face is captured in the correct position without rotation. The state without rotation means that the driver's face is captured so that the line connecting the driver's left eye and right eye is approximately parallel to the X-axis. If the imaging device 1 is installed in a state where it is rotated within the XY plane, the face of the driver will appear rotated in the captured image as shown in FIG. 7B. If the angle of rotation is large, the driver's face may appear to be facing upward, downward, left, right, or a combination of these directions, which may make it difficult for the driver monitoring system 4 to detect the driver's distracted driving, etc.

[0029] FIG. 8 is a flowchart showing a method for detecting the amount of deviation according to the first embodiment. For example, an installer of the imaging device 1 installs the imaging device 1 in a vehicle (step ST1 in FIG. 8). Here, an installation worker installs the imaging device 1 inside the vehicle. However, in a situation where the imaging device 1 has already been installed inside the vehicle, the installation worker does not need to install the imaging device 1 inside the vehicle. Next, the installer installs the lighting device 10 outside the vehicle (step ST2 in FIG. 8). Here, the installer installs the lighting device 10 on the exterior of the vehicle. However, in a situation where the lighting device 10 has already been installed on the exterior of the vehicle, the installer does not need to install the lighting device 10 on the exterior of the vehicle.

[0030] Next, the installer operates the lighting device 10. When the installer operates the lighting device 10, the lighting device 10 irradiates the structure 2 with light to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle (step ST3 in FIG. 8). The light for suppressing the influence of ambient light means light having a brightness that is capable of suppressing the influence of ambient light. Next, the installer operates the imaging device 1. The installer operates the imaging device 1, which then captures an image of the structure 2 illuminated by the lighting device 10 (step ST4 in FIG. 8). The deviation amount detection device 3 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the image of the structure 2 captured by the imaging device 1.

[0031] Next, the operations of the deviation amount detecting device 3 and the driver monitoring system 4 shown in FIG. 1 will be described. As described above, the imaging device 1 captures an image of the structure 2 inside the vehicle when light is irradiated onto the structure 2 by the lighting device 10, and outputs the captured image GP, which is an image of the structure 2, to the deviation amount detection device 3. Fig. 9 is an explanatory diagram showing an example of a captured image GP. The captured image GP shown in Fig. 9 is an image captured by an imaging device 1 installed on a dashboard, for example, and shows structures 2 such as a window frame a, an assist grip b, a headrest c, a sun visor d, a B-pillar e, a door frame f, and a shoulder anchor g of the vehicle. The captured image GP output from the imaging device 1 to the deviation amount detection device 3 is basically an image captured when the driver is not seated in the driver's seat. However, even if the driver is seated in the driver's seat, the captured image GP may be an image captured when the driver is seated in the driver's seat, as long as the driver is seated in a manner that does not block the structure 2.

[0032] The position detection unit 11 acquires the captured image GP from the imaging device 1 (step ST5 in FIG. 8). The position detection unit 11 detects the structure image position, which is the position of the structure 2 on the image, from the captured image GP (step ST6 in FIG. 8). The position detection unit 11 outputs structure image position information indicating the structure image position to the displacement amount calculation unit 12 .

[0033] The process of detecting the structure image position by the position detection unit 11 will be described in detail below. The position detection unit 11 receives from the outside reference image position information indicating the reference image position, which is the correct position on the image of the structure 2 when the installation position of the imaging device 1 is correct, and the reference image G REF and obtain the reference image G REF is an image taken when the imaging device 1 is installed in the correct position. In the first embodiment, the reference image position information and the reference image G REF are provided from the outside of the deviation amount detection device 3. However, this is only an example, and the reference image position information and the reference image G REF Each of these may be stored in the internal memory of the position detection unit 11. Here, an example will be described in which the position detection unit 11 detects the position of a structure image by performing template matching. In template matching, for example, SAD (Sum of Absolute Difference) shown in the following equation (1) is used.

[0034] In equation (1), I(x, y) is the pixel value of the captured image GP, and T(x, y) is the pixel value of the reference image G. REF (x, y) is the reference image position indicated by the reference image position information. w is the reference image G REF The width, h, of the reference image G REF The width w and height h are given as given values. d x is the scanning position of the x coordinate in the captured image GP, d y is the scanning position of the y coordinate in the captured image GP.

[0035] First, the position detection unit 11 detects whether the scanning position of the captured image GP is (d x ,d y ) and then, SAD(d x ,d y ) is calculated. The position detection unit 11 detects the reference image G REF or by rotating the scanning position (d x ,d y ) to change SAD(d x ,d y ) is calculated repeatedly. Next, the position detection unit 11 calculates the SAD(d x ,d y ) with the smallest SAD(d x ,d y ) to identify the The position detector 11 determines the minimum SAD (d x ,d y ) at the scanning position (d x ,d y ) outputs structure image position information indicating the structure image position to the displacement amount calculation unit 12 as the structure image position, which is the position of the captured image GP. The position detector 11 also calculates the minimum SAD (d x ,d y ) is calculated as the reference image G REF The amount of rotation of the imaging device 1 is calculated by the amount of deviation Δφ roll to the deviation amount calculation unit 12.

[0036] The deviation amount calculation unit 12 receives the structure image position information and the deviation amount Δφ in the rotation direction from the position detection unit 11. roll Obtain each of the following. The deviation amount calculation unit 12 receives reference image position information and reference image G REF Obtain each of the following. In the first embodiment, the reference image position information and the reference image G REF are provided from the outside of the deviation amount detection device 3. However, this is only an example, and the reference image position information and the reference image G REF may be stored in the internal memory of the deviation amount calculation unit 12.

[0037] The deviation amount calculation unit 12 calculates the reference image G so that the reference image position indicated by the reference image position information and the structure image position indicated by the structure image position information coincide with each other. REFis moved horizontally, vertically, or rotated. That is, the deviation amount calculation unit 12 calculates the deviation amount Δφ in the rotation direction. roll Only, the reference image G REF Rotate. The deviation amount calculation unit 12 also calculates the minimum SAD (d x ,d y ) at the scanning position (d x ,d y ) and the reference image G REF is moved horizontally or vertically. The deviation amount calculation unit 12 calculates the deviation amount of the reference image G when the reference image position and the structure image position coincide with each other. REF Horizontal displacement Δd x (=xd x ) and reference image G REF Vertical displacement Δd y (=yd y ) are calculated.

[0038] The deviation amount calculation unit 12 calculates the deviation amount of the reference image G REF Horizontal displacement Δd x From this, as shown in the following equation (2), the deviation amount Δφ between the installation position of the imaging device 1 and the correct installation position is x (Step ST7 in FIG. 8). x is the optical axis ΦX in the X-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position. REF and the amount of optical axis deviation between the optical axis of the imaging device 1 in the X-axis direction. The deviation amount calculation unit 12 also calculates the deviation amount of the reference image G REF Vertical displacement Δd y From this, as shown in the following equation (3), the deviation amount Δφ between the installation position of the imaging device 1 and the correct installation position is y (Step ST7 in FIG. 8). y is the optical axis ΦY in the Y-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position. REF and the amount of optical axis deviation between the optical axis of the imaging device 1 in the Y-axis direction.

[0039] TIFF0007720800000002.tif33166 In equations (2) and (3), FL is a known focal length of the imaging device 1. The focal length is, for example, the distance between an image sensor (not shown) included in the imaging device 1 and a lens (not shown) included in the imaging device 1.

[0040] The deviation amount calculation unit 12 calculates the deviation amount Δφ in the X-axis direction. x , deviation amount Δφ in the Y-axis direction y and the deviation in the rotational direction Δφ roll Each of the above is output to the driver monitoring system 4.

[0041] The sensing unit 31 of the driver monitoring system 4 acquires a captured image GP' from the imaging device 1. The captured image GP' is an image captured while the driver is driving the vehicle. The sensing unit 31 also receives the displacement amount Δφ in the X-axis direction from the displacement amount calculation unit 12. x , deviation amount Δφ in the Y-axis direction y and the deviation in the rotational direction Δφ roll Obtain each of the following. The sensing unit 31 performs passenger sensing to identify the position of the driver's head shown in the captured image GP', for example. When performing occupant sensing, the sensing unit 31 takes into consideration the possibility that the installation position of the image capturing device 1 may be displaced from the correct installation position. For this reason, when sensing the position of the driver's head, for example, the sensing unit 31 calculates a displacement amount Δφ x ,Δφ y ,Δφ roll The position that is shifted by only this amount is sensed. Specifically, if the coordinates of the driver's head position in the captured image GP' are (x, y, z), the coordinates (x', y', z') of the position sensed by the sensing unit 31 are expressed as follows in equation (4):

[0042] TIFF0007720800000003.tif44166

[0043] The sensing unit 31 monitors the state of the vehicle occupant based on the pixel value of the coordinates (x', y', z') in the captured image GP'. The sensing unit 31 monitors the state of the occupants, and detects, for example, whether the driver is looking away from the vehicle or whether the driver is drowsy at the wheel. The process of detecting whether the driver is looking away from the road or whether the driver is drowsy at the wheel is a known technique, and therefore a detailed description thereof will be omitted. If the sensing unit 31 detects, for example, that the driver is looking away from the road or is drowsy at the wheel, the warning output unit 32 outputs warning information indicating a warning to the driver to an electronic control unit of the vehicle or the like.

[0044] In the displacement amount detection device 3 shown in FIG. 1, a light source 10a of an illumination device 10 irradiates a structure 2 with light. Illuminating the structure 2 with light highlights the boundary between the structure 2 and other structures, thereby suppressing the influence of ambient light. Suppressing the influence of ambient light makes it easier to detect the position of the structure image from the captured image GP.

[0045] When the structure 2 inside the vehicle photographed by the imaging device 1 is, for example, a window frame a, the light source 10a of the lighting device 10 is installed on one of both sides of the window frame a, opposite the side facing the imaging device 1, as shown in Fig. 10. In the example of Fig. 10, the light source 10a is installed on the outside of the vehicle. In particular, in the example of FIG. 10A, the light source 10a is installed at a position where the light emitting surface thereof comes into contact with the window frame a, which is the structure 2. 10B, the light emitting surface of light source 10a is installed at a position away from window frame a, which is structure 2. When the light emitting surface of light source 10a is installed at a position away from window frame a, lighting device 10 may be supported by, for example, a tripod or may be suspended from the ceiling of a parking lot where the vehicle is parked. When the light emitting surface of light source 10a is installed at a position away from structure 2, it is desirable that the distance between the light emitting surface and structure 2 be as short as possible to prevent workers or the like from getting between the light emitting surface of light source 10a and structure 2. FIG. 10A is an explanatory diagram showing a light source 10a having a light-emitting surface disposed at a position where it comes into contact with a window frame a, which is a structure 2. FIG. FIG. 10B is an explanatory diagram showing a light source 10a whose light emitting surface is installed at a position away from a window frame a, which is the structure 2.

[0046] 11A, a shadow of an object caused by ambient light, such as a scene outside the vehicle, a worker, or the like, may appear inside the window frame a, which is the structure 2. In FIG. 11A, the area surrounded by a dashed line is the image capturing area of the imaging device 1. In this case, the image GP captured by the imaging device 1 may include the scenery outside the vehicle, a worker, or the shadow of an object. In a captured image GP in which shadows of objects caused by ambient light are reflected, it may be difficult to detect the position of the structure 2 because the boundary between the structure 2 and the shadows of the objects is unclear. FIG. 11A is an explanatory diagram showing a state in which a shadow of an object caused by ambient light is cast inside a window frame a. FIG. 11B is an explanatory diagram showing a captured image GP in which shadows of objects caused by ambient light are reflected. In Figure 11B, "0" represents the brightness value of the pixels that make up the window frame a, the pixels that make up the scenery outside the vehicle, the pixels that make up the worker, and the pixels that make up the shadow of the object. "1" represents the brightness value of the pixels other than these pixels. 11A and 11B, the shapes of the shadows of the objects and the like are deformed, so the shapes of the shadows of the objects and the like shown in Fig. 11A do not completely match the shapes of the shadows of the objects and the like shown in Fig. 11B.

[0047] On the other hand, when the lighting device 10 is installed, the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from entering the inside of the window frame a, as shown in FIG. 12A. In FIG. 12A, the area surrounded by a dashed line is the capture area of the imaging device 1. Therefore, as shown in FIG. 12B, the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from being captured in the captured image GP. Furthermore, the light source 10a of the lighting device 10 irradiates the structure 2 with light, thereby highlighting the boundary between the structure 2 and other structures. Therefore, the position of the structure 2 captured in the captured image GP can be easily detected. Therefore, by providing the deviation amount detection device 3 with the illumination device 10, the deviation amount Δφ x ,Δφ y ,Δφ roll This can prevent the deterioration of the calculation accuracy. FIG. 12A is an explanatory diagram showing a state in which the lighting device 10 prevents the shadow of an object caused by ambient light from entering inside the window frame a. FIG. 12B is an explanatory diagram showing a captured image GP in which shadows of objects caused by ambient light and the like are not reflected. 12B, "0" is the brightness value of the pixels that make up window frame a, and "1" is the brightness value of the pixels other than the pixels that make up window frame a. 12A and 12B, the shape of the window frame a is deformed, so the shape of the window frame a shown in Fig. 12A does not completely match the shape of the window frame a shown in Fig. 12B.

[0048] In the above-described first embodiment, the deviation amount detection method is configured to include a step in which the lighting device 10 irradiates the structure 2 of the vehicle with light for suppressing the influence of ambient light, which is light entering the vehicle from outside, and the imaging device 1 captures an image of the structure 2 illuminated with light by the lighting device 10. The deviation amount detection device 3 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the image of the structure 2 captured by the imaging device 1. Therefore, the deviation amount detection method can suppress deterioration in the calculation accuracy of the deviation amount due to changes in the state of ambient light.

[0049] In the deviation amount detection device 3 shown in Fig. 1, the brightness of the light emitting surface of the light source 10a is uniform. However, this is only one example, and the light irradiated from the light source 10a may be light that generates a pattern on the irradiated surface of the structure 2, as shown in Fig. 13. 13 is an explanatory diagram showing an example of light that produces a pattern on the irradiated surface. In the example of Fig. 13, a plurality of patterns each having a diamond shape are arranged vertically and horizontally. By drawing a pattern on the light-emitting surface of light source 10a, a pattern appears on the surface irradiated to structure 2. Furthermore, by engraving a pattern on the light-emitting surface of light source 10a, a pattern appears on the surface irradiated to structure 2. Furthermore, by disposing a sheet between the light-emitting surface of light source 10a and the structure 2, in which the transmittance of the patterned portion differs from the transmittance of the non-patterned portion, a pattern appears on the surface irradiated to structure 2. By making the light emitted from light source 10a to be light that generates a pattern on the irradiated surface of structure 2, it becomes easier to distinguish between the light emitted from light source 10a and ambient light, which is light other than the light emitted from light source 10a. As a result, the accuracy with which position detection unit 11 detects the position of the structure image is improved.

[0050] In the deviation amount detection device 3 shown in Fig. 1, the light source 10a continuously emits light. However, this is merely an example, and the light source 10a may also blink the light that it emits to the structure 2, as shown in Fig. 14. FIG. 14 is an explanatory diagram showing an example in which the light from the light source 10a is blinking. 14, among the imaging frames of the imaging device 1, in the first and third frames, the light source 10a irradiates the structure 2 with light. Among the imaging frames of the imaging device 1, in the second frame, the light source 10a does not irradiate the structure 2 with light. The position detection unit 11 can easily detect the structure 2 appearing in the captured image GP by detecting the difference between the captured image GP of the first frame and the captured image GP of the second frame, for example.

[0051] In the displacement amount detection device 3 shown in FIG. 1, the position detection unit 11 detects the structure image position, which is the position of the structure 2 on the image. However, this is merely an example, and instead of detecting the position of the structure 2 on the image, the position detection unit 11 may extract feature points of the structure 2 shown in the captured image GP using a method similar to that of the displacement amount detection device described in Patent Document 1. In this case, the displacement amount calculation unit 12 needs to calculate the amount of displacement from the feature points of the structure 2 using a method similar to that of the displacement amount detection device described in Patent Document 1. Note that in this case as well, the captured image GP is an image of the structure 2 captured by the imaging device 1 when light is irradiated onto the structure 2 by the lighting device 10.

[0052] In the displacement amount detection device 3 shown in Fig. 1, the structure 2 is assumed to be a window frame a, and the lighting device 10 irradiates light onto the window frame a. However, the structure 2 is not limited to the window frame a, and the structure 2 may be, for example, an assist grip b, a headrest c, a sun visor d, a B-pillar e, a door frame f, or a shoulder anchor g. Therefore, the lighting device 10 may be configured to irradiate light onto the assist grip b, etc.

[0053] Embodiment 2 In the second embodiment, a displacement amount detecting device 3 in which an illumination device 10 includes a light source 10b and a reflector 10c will be described.

[0054] Fig. 15 is a configuration diagram showing a deviation amount detecting device 3 and a driver monitoring system 4 according to embodiment 2. In Fig. 15, the same reference numerals as in Fig. 1 indicate the same or corresponding parts, and therefore description thereof will be omitted. In the displacement amount detecting device 3 shown in FIG. 15, the lighting device 10 includes a light source 10b and a reflector 10c. The light source 10b irradiates the window frame a, which is the structure 2, with light for suppressing the influence of ambient light. The light source 10b is a surface light source, and the brightness of the light-emitting surface of the surface light source is uniform. 15, the light source 10b is a surface light source. However, this is merely an example, and the light source 10b may be, for example, a point light source, or may be, for example, an incandescent lamp or an LED (Light Emitting Diode).

[0055] The reflector 10c is, for example, a screen, paper, cloth, a wooden board, a resin board, or a metal plate. However, this is only an example, and the reflector 10c may be a combination of the screen, paper, cloth, wooden board, resin board, and metal plate. The reflector 10c has a flat light reflecting surface. The reflecting plate 10c is installed on one of both sides of the structure 2, opposite to the side facing the imaging device 1. The reflector 10c reflects the light that has been emitted from the light source 10b and passed through the inside of the window frame a toward the window frame a. The light reflected by the reflector 10c passes through the inside of the window frame a and reaches the imaging device 1. 15, the reflecting surface of the reflector 10c is flat. However, this is just one example, and the reflecting surface of the reflector 10c may be, for example, a spherical surface or a surface having projections and recesses. In the displacement amount detection device 3 shown in Fig. 15, the light source 10b is installed inside the vehicle. However, this is just one example, and the light source 10b may be installed outside the vehicle. In this case, the reflector 10c reflects the light from the light source 10b in the direction of the window frame a. The light reflected by the reflector 10c passes through the window frame a and reaches the imaging device 1.

[0056] Next, the operation of the deviation amount detecting device 3 shown in FIG. 15 will be described. However, apart from the illumination device 10, the deviation amount detection device 3 is the same as that shown in Fig. 1. Therefore, only the operation of the illumination device 10 will be described here. When the structure 2 inside the vehicle that is photographed by the imaging device 1 is, for example, a window frame a, the reflector 10c of the lighting device 10 is installed on one of both sides of the window frame a, opposite the side facing the imaging device 1, as shown in Fig. 16. In the example of Fig. 16, the reflector 10c is installed on the outside of the vehicle.

[0057] 16A, a light source 10b, which is a part of the lighting device 10, is installed inside the vehicle, and a reflector 10c, which is also a part of the lighting device 10, is installed outside the vehicle. The reflective surface of the reflector 10c is installed in a position where it comes into contact with a window frame a, which is the structure 2. 16B, a light source 10b, which is a part of the lighting device 10, is installed outside the vehicle, and a reflector 10c, which is also a part of the lighting device 10, is installed outside the vehicle. The reflective surface of the reflector 10c is installed at a position away from a window frame a, which is a structure 2. When the reflective surface of the reflector 10c is installed at a position away from the window frame a, the reflector 10c may be supported by, for example, a tripod or may be suspended from the ceiling of a parking lot where the vehicle is parked. When the reflective surface of the reflector 10c is installed at a position away from the structure 2, it is desirable that the distance between the reflective surface and the structure 2 be as short as possible to prevent workers or the like from getting between the reflective surface of the reflector 10c and the structure 2. FIG. 16A is an explanatory diagram showing a reflector 10c having a reflecting surface placed at a position where it comes into contact with a window frame a, which is a structure 2. FIG. FIG. 16B is an explanatory diagram showing a reflector 10c whose reflecting surface is located at a position away from a window frame a, which is the structure 2.

[0058] When the lighting device 10 including the light source 10b and the reflector 10c is installed, the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from entering the inside of the window frame a, as shown in FIG. 12A. Therefore, as shown in FIG. 12B, the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from being reflected in the captured image GP. Furthermore, the reflector 10c reflects light toward the structure 2, thereby highlighting the boundary between the structure 2 and other structures, thereby suppressing the influence of ambient light. By suppressing the influence of ambient light, the accuracy of detection of the position of the structure 2 by the position detection unit 11 is improved. Therefore, by providing the deviation amount detection device 3 with the illumination device 10, the deviation amount Δφ x ,Δφ y ,Δφ roll This can prevent the deterioration of the calculation accuracy.

[0059] As described above, the misalignment amount detecting device 3 shown in FIG. 15 can suppress deterioration in the accuracy of calculating the misalignment amount due to changes in the state of the ambient light, similar to the misalignment amount detecting device 3 shown in FIG.

[0060] In the deviation amount detection device 3 shown in Fig. 15, the brightness of the light emitting surface of the light source 10b is uniform. However, this is just one example, and the light irradiated from the light source 10b may be light that generates a pattern on the irradiated surface of the structure 2, as shown in Fig. 13. By drawing a pattern on the light-emitting surface of light source 10b, a pattern appears on the surface irradiated to structure 2. Furthermore, by engraving a pattern on the light-emitting surface of light source 10b, a pattern appears on the surface irradiated to structure 2. Furthermore, by disposing a sheet between the light-emitting surface of light source 10b and the structure 2, in which the transmittance of the patterned portion differs from the transmittance of the non-patterned portion, a pattern appears on the surface irradiated to structure 2. By making the light emitted from light source 10b to be light that generates a pattern on the irradiated surface of structure 2, it becomes easy to distinguish between the light emitted from light source 10b and ambient light that is light other than the light emitted from light source 10b. As a result, the accuracy with which position detection unit 11 detects the position of structure 2 is improved. The light emitted from the light source 10b may be light that produces a pattern on the irradiated surface of the structure 2, but the reflector 10c may have a pattern. Even if the reflector 10c has a pattern, the pattern will be produced on the irradiated surface of the structure 2. The reflecting surface of the reflector 10c is made of a plurality of materials with different reflectivities, so that the reflector 10c has a pattern. Also, the reflecting surface of the reflector 10c has a spherical surface or an uneven surface, so that the reflector 10c has a pattern.

[0061] In the deviation amount detection device 3 shown in Fig. 15, the light source 10b continuously emits light. However, this is merely an example, and the light source 10b may blink the light that it emits to the structure 2, as shown in Fig. 14. The position detection unit 11 can easily detect the structure 2 appearing in the captured image GP by detecting the difference between the captured image GP of the first frame and the captured image GP of the second frame, for example.

[0062] 15, the structure 2 is assumed to be a window frame a, and the reflector 10c reflects light from the light source 10b toward the window frame a. However, the structure 2 is not limited to the window frame a, and the structure 2 may be, for example, an assist grip b, a headrest c, a sun visor d, a B-pillar e, a door frame f, or a shoulder anchor g. Therefore, the reflector 10c may reflect light from the light source 10b toward the assist grip b, etc.

[0063] Embodiment 3 In the third embodiment, a deviation amount detecting device 3 including an illumination device 60 having a plurality of light emitting points will be described.

[0064] Fig. 17 is a configuration diagram showing a deviation amount detecting device 3 and a driver monitoring system 4 according to embodiment 3. In Fig. 17, the same reference numerals as in Figs. 1 and 15 indicate the same or corresponding parts, and therefore description thereof will be omitted. Fig. 18 is a hardware configuration diagram showing the hardware of a deviation amount detection device 3 according to embodiment 3. In Fig. 18, the same reference numerals as in Fig. 2 denote the same or corresponding parts, and therefore description thereof will be omitted.

[0065] The lighting device 60 is installed, for example, inside a vehicle. The illumination device 60 has N light-emitting points, where N is an integer equal to or greater than 3. For the sake of convenience, the deviation amount detection device 3 shown in FIG. 17 will be described assuming that N=3. Each of the light-emitting points 60a, 60b, and 60c is a light-emitting portion for outputting light emitted from the light source of the lighting device 60 to the outside. In the deviation amount detecting device 3 shown in FIG. 17, the three light emitting points 60a, 60b, and 60c are not aligned in a straight line, and therefore when the three light emitting points 60a, 60b, and 60c are connected, a triangle is formed.

[0066] The position detection unit 61 is realized by, for example, a position detection circuit 71 shown in FIG. The position detection unit 61 acquires images of the three light emitting points 60a, 60b, and 60c captured by the imaging device 1. The position detection unit 61 detects the light emitting point image positions, which are the positions of the three light emitting points 60a, 60b, and 60c on the image. The position detection unit 61 outputs light emitting point image position information indicating the light emitting point image position to the deviation amount calculation unit 62 .

[0067] The deviation amount calculation section 62 is realized by, for example, a deviation amount calculation circuit 72 shown in FIG. The deviation amount calculation unit 62 acquires light-emitting point image position information from the position detection unit 61. The deviation amount calculation unit 62 also acquires, from the outside, reference image position information indicating reference image positions, which are correct positions on the image of the light-emitting points 60a, 60b, and 60c when the installation position of the imaging device 1 is correct. The deviation amount calculation unit 62 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position based on the reference image position indicated by the reference image position information and the light emitting point image position indicated by the light emitting point image position information. The deviation amount calculation unit 62 outputs the calculated deviation amount to the driver monitoring system 4.

[0068] 17, it is assumed that the position detection unit 61 and the deviation amount calculation unit 62, which are components other than the illumination device 60 of the deviation amount detection device 3, are each realized by dedicated hardware as shown in Fig. 18. In other words, it is assumed that a part of the deviation amount detection device 3 is realized by a position detection circuit 71 and a deviation amount calculation circuit 72. Each of the position detection circuit 71 and the deviation amount calculation circuit 72 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0069] The components of the deviation amount detection device 3 other than the lighting device 60 are not limited to those realized by dedicated hardware, and the components other than the lighting device 60 may be realized by software, firmware, or a combination of software and firmware. When the components of the deviation amount detection device 3 other than the illumination device 60 are realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the position detection unit 61 and the deviation amount calculation unit 62 is stored in the memory 51 shown in Fig. 4. Then, the processor 52 shown in Fig. 4 executes the program stored in the memory 51.

[0070] 18 shows an example in which the components of the misalignment detection device 3 other than the illumination device 60 are realized by dedicated hardware, while Fig. 4 shows an example in which the components of the misalignment detection device 3 other than the illumination device 60 are realized by software, firmware, or the like. However, this is merely an example, and some of the components of the misalignment detection device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, or the like.

[0071] FIG. 20 is a flowchart showing a method for detecting the amount of deviation according to the third embodiment. For example, an installer of the imaging device 1 installs the imaging device 1 in a vehicle (step ST11 in FIG. 20). Here, an installation worker installs the imaging device 1 inside the vehicle. However, in a situation where the imaging device 1 has already been installed inside the vehicle, the installation worker does not need to install the imaging device 1 inside the vehicle. Next, the installer installs the lighting device 60 having three or more light-emitting points inside the vehicle (step ST12 in FIG. 20). Here, an installer installs the lighting device 60 inside the vehicle. However, in a situation where the lighting device 60 has already been installed inside the vehicle, the installer does not need to install the lighting device 60 inside the vehicle. In the deviation amount detection device 3 shown in Fig. 17, an illumination device 60 having three light-emitting points 60a, 60b, and 60c is installed inside a window frame a, as shown in Fig. 19. However, this is merely an example, and an illumination device 60 having three light-emitting points 60a, 60b, and 60c may also be installed outside a vehicle. FIG. 19 is an explanatory diagram showing an example in which light-emitting points 60a, 60b, and 60c of a lighting device 60 are attached to the inside of a window frame a.

[0072] Next, the installer operates the lighting device 60. The installer operates the lighting device 60, causing the lighting device 60 to emit light from the light-emitting points 60a, 60b, and 60c (step ST13 in FIG. 20). The light-emitting points 60a, 60b, and 60c emit light toward the imaging device 1. Next, the installer operates the imaging device 1. The installer operates the imaging device 1, which then captures images of the light-emitting points 60a, 60b, and 60c emitting light (step ST14 in FIG. 20). The deviation amount detection device 3 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the images of the light emitting points 60a, 60b, and 60c captured by the imaging device 1.

[0073] Next, the operations of the deviation amount detecting device 3 and the driver monitoring system 4 shown in FIG. 17 will be described. As described above, when light is irradiated from the three light-emitting points 60a, 60b, and 60c, the imaging device 1 captures the three light-emitting points 60a, 60b, and 60c, and outputs the captured image GP, which is an image of the light-emitting points 60a, 60b, and 60c, to the deviation amount detection device 3. Regardless of whether or not ambient light is incident on the vehicle, the brightness of the light from the light-emitting points 60a, 60b, and 60c is higher than the brightness of the surroundings of the light-emitting points 60a, 60b, and 60c. Therefore, the shapes of the light-emitting points 60a, 60b, and 60c captured in the captured image GP are clear. The captured image GP output from the imaging device 1 to the deviation amount detection device 3 is basically an image captured when the driver is not seated in the driver's seat. However, even if the driver is seated in the driver's seat, the captured image GP may be an image captured when the driver is seated in the driver's seat, as long as the driver is seated in a manner that does not block the structure 2.

[0074] The position detection unit 61 acquires the captured image GP from the imaging device 1 (step ST15 in FIG. 20). The position detection unit 61 detects the light emitting point image positions, which are the positions of the light emitting points 60a, 60b, and 60c on the image, from the captured image GP (step ST16 in FIG. 20). The position detection unit 61 outputs light emitting point image position information indicating the light emitting point image position to the deviation amount calculation unit 62 . The process of detecting the light emitting point image position by the position detection unit 61 is similar to the process of detecting the structure image position by the position detection unit 11 shown in FIG. 1, and therefore a detailed description thereof will be omitted.

[0075] The deviation amount calculation unit 62 receives the light emitting point image position information and the deviation amount Δφ in the rotation direction from the position detection unit 61. roll Obtain each of the following. The deviation amount calculation unit 62 receives from the outside reference image position information indicating the reference image positions, which are the correct positions of the light emitting points 60a, 60b, and 60c on the image when the installation position of the imaging device 1 is correct, and the reference image G REF and get. In the third embodiment, the light emitting point image position information and the reference image G REF are provided from the outside of the deviation amount detection device 3. However, this is only an example, and the reference image position information and the reference image G REF may be stored in the internal memory of the deviation amount calculation unit 62.

[0076] The deviation amount calculation unit 62 calculates the reference image G so that the reference image position indicated by the reference image position information and the light emitting point image position indicated by the light emitting point image position information coincide with each other. REF is moved horizontally, vertically, or rotated. That is, the deviation amount calculation unit 62 calculates the deviation amount Δφ in the rotation direction. roll Only, the reference image G REF Rotate. The deviation amount calculation unit 62 also calculates the minimum SAD (d x ,d y ) at the scanning position (d x ,d y ) and the reference image G REF is moved horizontally or vertically. The deviation amount calculation unit 62 calculates the deviation amount of the reference image G when the reference image position and the light emitting point image position coincide with each other. REF Horizontal displacement Δd x (=xd x ) and reference image G REF Vertical displacement Δd y (=yd y ) are calculated.

[0077] The deviation amount calculation unit 62 calculates the deviation amount of the reference image G REF Horizontal displacement Δd x From this, as shown in the above formula (2), the deviation amount Δφ between the installation position of the imaging device 1 and the correct installation position is x (Step ST17 in FIG. 20). x is the optical axis ΦX in the X-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position. REF and the amount of optical axis deviation between the optical axis of the imaging device 1 in the X-axis direction. The deviation amount calculation unit 62 also calculates the deviation amount of the reference image G REF Vertical displacement Δd y From this, as shown in the above formula (3), the deviation amount Δφ between the installation position of the imaging device 1 and the correct installation position is y (Step ST17 in FIG. 20). y is the optical axis ΦY in the Y-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position. REF and the amount of optical axis deviation between the optical axis of the imaging device 1 in the Y-axis direction.

[0078] The deviation amount calculation unit 62 calculates the deviation amount Δφ in the X-axis direction. x , deviation amount Δφ in the Y-axis direction y and the deviation in the rotational direction Δφ roll Each of the above is output to the driver monitoring system 4.

[0079] In the above-described third embodiment, the deviation amount detection method is configured to include the steps of: an illumination device 60 having three or more light-emitting points emitting light from the light-emitting points; an imaging device 1 capturing an image of the light-emitting points emitting light; and a deviation amount detection device 3 calculating, based on the image of the light-emitting points captured by the imaging device 1, the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1. Therefore, the deviation amount detection method can suppress deterioration in the calculation accuracy of the deviation amount due to changes in the state of ambient light.

[0080] 17, the illumination device 60 has three light-emitting points 60a, 60b, and 60c. However, this is merely an example, and the misalignment amount detection device 3 may be provided with three or more illumination devices 60 each having one light-emitting point.

[0081] It should be noted that the present disclosure allows for free combination of the embodiments, modification of any of the components of the embodiments, or omission of any of the components in the embodiments. [Explanation of symbols]

[0082] 1 imaging device, 2 structure, 3 deviation amount detection device, 4 driver monitoring system, 10 lighting device, 10a light source, 10b light source, 10c reflector, 11 position detection unit, 12 deviation amount calculation unit, 21 position detection circuit, 22 deviation amount calculation circuit, 31 sensing unit, 32 warning output unit, 41 sensing circuit, 42 warning output circuit, 51 memory, 52 processor, 60 lighting device, 60a, 60b, 60c light emitting point, 61 position detection unit, 62 deviation amount calculation unit, 71 position detection circuit, 72 deviation amount calculation circuit.

Claims

1. In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle, a step in which a lighting device irradiates the vehicle structure with light emitted from a surface light source installed on one of both sides of the vehicle structure opposite to the side facing the imaging device; a step of capturing an image of a structure of the vehicle illuminated with the illumination light by the lighting device using the imaging device; a step in which a deviation amount detection device calculates a deviation amount between an installation position of the imaging device and a correct installation position of the imaging device based on an image of a structure of the vehicle captured by the imaging device; A deviation amount detection method comprising:

2. In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle, a step in which the lighting device reflects light emitted from the light source toward the vehicle structure using a reflector installed on one of both sides of the vehicle structure opposite to the side facing the imaging device, thereby irradiating the vehicle structure with the light; a step of capturing an image of a structure of the vehicle illuminated with the illumination light by the lighting device using the imaging device; a step in which a deviation amount detection device calculates a deviation amount between an installation position of the imaging device and a correct installation position of the imaging device based on an image of a structure of the vehicle captured by the imaging device; A deviation amount detection method comprising:

3. An imaging device for imaging a vehicle structure; A lighting device that irradiates light onto a structure of the vehicle, a surface light source that irradiates the illumination light onto a structure of the vehicle; the surface light source is installed on one of both sides of a structure of the vehicle opposite to a side facing the imaging device; and In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle, a position detection unit that acquires an image of the vehicle structure captured by the imaging device when the illumination light from the surface light source of the lighting device is irradiated onto the vehicle structure, and detects a structure image position that is a position on the image of the vehicle structure; a deviation amount calculation unit that calculates a deviation amount between the installation position of the imaging device and the correct installation position based on a reference image position that is a correct position on an image of a structure of the vehicle when the installation position of the imaging device is the correct installation position and the structure image position detected by the position detection unit; A deviation amount detection device comprising:

4. An imaging device for imaging a vehicle structure; A lighting device that irradiates light onto a structure of the vehicle, A light source and a reflector that reflects the light emitted from the light source toward a structure of the vehicle, the lighting device, wherein the reflector is installed on one of both sides of a structure of the vehicle opposite to a side facing the imaging device; In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle, a position detection unit that acquires an image of the vehicle structure captured by the imaging device when the illumination light reflected by the reflector of the illumination device is irradiated onto the vehicle structure, and detects a structure image position, which is the position of the vehicle structure on the image; a deviation amount calculation unit that calculates a deviation amount between the installation position of the imaging device and the correct installation position based on a reference image position that is a correct position on an image of a structure of the vehicle when the installation position of the imaging device is the correct installation position and the structure image position detected by the position detection unit; A deviation amount detection device comprising:

5. 4. The displacement amount detecting device according to claim 3, wherein the brightness of the light emitting surface of said surface light source is uniform.

6. 4. The deviation amount detecting device according to claim 3, wherein the light emitted from the surface light source is light that produces a pattern on the illuminated surface of the vehicle structure.

7. 4. The deviation amount detecting device according to claim 3, wherein the surface light source blinks the light that is irradiated onto the vehicle structure.

8. 5. The deviation amount detecting device according to claim 4, wherein the reflector is installed at a position where it comes into contact with a structure of the vehicle or at a position away from the structure of the vehicle.

9. the light source is a surface light source, 5. The displacement detection device according to claim 4, wherein the brightness of the light emitting surface of said surface light source is uniform.

10. 5. The deviation amount detection device according to claim 4, wherein the light emitted from the light source is light that produces a pattern on the irradiated surface of the vehicle structure, or the reflector has a pattern.

11. 5. The deviation amount detecting device according to claim 4, wherein the light source blinks the irradiated light that is irradiated onto the vehicle structure.

12. 5. The displacement detection device according to claim 4, wherein the reflector is a screen, paper, cloth, a wooden board, a resin board, or a metal board.

13. 5. The deviation amount detecting device according to claim 3, wherein the vehicle structure is a window frame attached to the vehicle.

Images