Mirror position adjustment method

Abstract

This invention provides a mirror position adjustment method that allows for easy and precise adjustment of the rearview mirror to a standard position. [Solution] The mirror position adjustment method includes an installation step of arranging a target mark device that displays a target mark and a vehicle equipped with a rearview mirror with a camera in a predetermined first positional relationship, and a mirror adjustment step of generating a mirror image by capturing the mirror surface of the rearview mirror in which the target mark is reflected with an external camera, and adjusting the posture of the rearview mirror to a reference posture in which the image of the target mark reflected in the rearview mirror, which is included in the mirror image, is in a predetermined position and shape.

Description

Technical Field

[0001] The present invention relates to a mirror position adjustment method.

Background Art

[0002] Driver monitoring systems have been conventionally known. A driver monitoring system captures an image of a driver with a camera, monitors, for example, the driver's line-of-sight direction and the driver's face orientation in real time from the captured driver image, and detects driver inattention, fatigue, drowsiness or the like based on the line-of-sight direction, face orientation and the like to output an alarm at an early stage.

[0003] Furthermore, a mirror-integrated driver monitoring system in which a camera is provided in a rearview mirror is also known. Since the mirror-integrated driver monitoring system is provided separately from the vehicle system, it can reduce the processing load on the vehicle system. Further, the mirror-integrated driver monitoring system allows the driver monitoring system to be externally attached even to a vehicle that is not equipped with a driver monitoring system.

[0004] By the way, a driver monitoring system uses a pre-trained model to detect the driver's line-of-sight direction and the driver's face orientation relative to the optical axis of the camera from a real-time image capturing the driver. However, the posture of the rearview mirror is appropriately adjusted by the driver. In addition, the posture of the rearview mirror may be displaced when, for example, a person or the like comes into contact with it. For this reason, the direction of the optical axis of the camera provided in the rearview mirror changes as the posture of the rearview mirror changes. Therefore, in the mirror-integrated driver monitoring system, when the posture of the rearview mirror changes, the reference of the values representing the detected driver's line-of-sight direction and face orientation becomes different.

[0005] To solve these problems, conventional mirror-integrated driver monitoring systems have a camera mounted on the rearview mirror that captures a reference image while the rearview mirror is adjusted to a reference position. The mirror-integrated driver monitoring system then calculates the difference angle between the optical axis direction of the camera detected from the reference image and the optical axis direction of the camera detected from the real-time image, and corrects the gaze direction and face orientation detected from the real-time image using the calculated difference angle. As a result, the mirror-integrated driver monitoring system can detect the driver's gaze direction or face orientation, etc., based on the reference position of the rearview mirror, regardless of the current position of the rearview mirror.

[0006] For example, Patent Document 1 describes a technique for detecting the direction of the driver's face and gaze based on a straight line connecting the camera and the driver, and the direction of vehicle movement, while the camera is fixed in place. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Patent No. 7063024 [Overview of the project] [Problems that the invention aims to solve]

[0008] Such reference images must be generated by capturing images with a camera mounted on the rearview mirror while the rearview mirror is adjusted to a reference position, for example, during installation, vehicle factory shipment, and maintenance. However, adjusting the rearview mirror to the reference position must be done manually by an operator while visually checking the mirror surface, making it extremely difficult to adjust it accurately.

[0009] The present invention has been made in view of the above, and provides a mirror position adjustment method that allows the rearview mirror to be easily and accurately adjusted to a reference position. [Means for solving the problem]

[0010] The mirror position adjustment method according to the present invention includes an installation step of arranging a target mark device that displays a target mark and a vehicle equipped with a rearview mirror with a camera in a predetermined first positional relationship, and a mirror adjustment step of generating a mirror image by capturing the mirror surface of the rearview mirror in which the target mark is reflected with an external camera, and adjusting the posture of the rearview mirror to a reference posture in which the image of the target mark reflected in the rearview mirror, included in the mirror image, is in a predetermined position and shape. [Effects of the Invention]

[0011] According to the mirror position adjustment method of the present invention, the rearview mirror can be easily and accurately adjusted to a reference position. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a view of the support device and vehicle according to the embodiment, seen from above. [Figure 2] Figure 2 is a side view of the support device and vehicle according to the embodiment. [Figure 3] Figure 3 shows a target marking device. [Figure 4] Figure 4 shows the mirror image when the rearview mirror is not in the standard position. [Figure 5] Figure 5 shows the mirror image when the rearview mirror is in the reference position. [Figure 6] Figure 6 is a flowchart showing the process flow of the mirror position adjustment method according to the embodiment. [Figure 7] Figure 7 shows the configuration of the driver monitoring system. [Figure 8] Figure 8 is a flowchart showing the processing flow of the ECU. [Modes for carrying out the invention]

[0013] The mirror position adjustment method according to the embodiment will be described below with reference to the drawings.

[0014] Figure 1 is a view of the support device 10 and vehicle 20 according to the embodiment, seen from above. Figure 2 is a view of the support device 10 and vehicle 20 according to the embodiment, seen from the side. Figure 3 is a diagram showing the target mark device 32.

[0015] The support device 10 is a device that assists in the mirror adjustment process in the mirror position adjustment method. In the mirror adjustment process, an operator uses the support device 10 to adjust the position of the rearview mirror 22 of a vehicle 20 equipped with a rearview mirror 22 on which a camera 24 is installed to a reference position.

[0016] The rearview mirror 22 has a mirror surface that reflects the image behind the vehicle 20. The rearview mirror 22 is positioned at the front of the interior of the vehicle 20, in a location where a person seated in the driver's seat of the vehicle 20 can see the mirror surface. Furthermore, the rearview mirror 22 is adjustable in position. This allows the driver to see objects behind the vehicle 20 through the mirror surface, regardless of the driver's height or posture.

[0017] Further, the vehicle 20 includes a mirror-integrated driver monitoring system 30. The mirror-integrated driver monitoring system 30 is a monitoring device that monitors the state of the driver. The mirror-integrated driver monitoring system 30 captures an image of the driver seated in the driver's seat with a camera 24 provided in the rearview mirror 22. The mirror-integrated driver monitoring system 30 monitors the driver's line-of-sight direction, the orientation of the driver's face, and the like in real time from the captured driver image, and detects, for example, driver inattention, fatigue, drowsiness, and the like based on the line-of-sight direction and the face orientation. When detecting driver inattention, fatigue, drowsiness, or the like, the mirror-integrated driver monitoring system 30 notifies the driver of the detection information. Such a mirror-integrated driver monitoring system 30 can notify the driver of inattention, fatigue, drowsiness, and the like at an early stage.

[0018] The assistance device 10 includes a target mark device 32 and a display device 34.

[0019] The target mark device 32 has a main surface 42 that is a flat surface. The target mark device 32 displays a target mark 44 of a predetermined shape and size on the main surface 42.

[0020] For example, the target mark 44 is a mark drawn on the main surface 42. The target mark 44 may also be an object formed on the main surface 42. Further, when the main surface 42 is a monitor that displays images, the target mark 44 may be an image displayed on the monitor.

[0021] For example, the target mark 44 is, for example, a rectangular mark of a predetermined color and a predetermined size drawn on the main surface 42.

[0022] In the mirror adjustment step, the target mark device 32 and the vehicle 20 are arranged in a predetermined first positional relationship.

[0023] The first positional relationship is such that the target mark device 32 is positioned at the rear outside the vehicle 20, and the target mark 44 is reflected in the mirror surface of the rearview mirror 22. To ensure that the vehicle 20 and the target mark device 32 are positioned in this first positional relationship, for example, the vehicle 20 is fixed in a predetermined position on the floor of the building. Alternatively, to ensure that the vehicle 20 and the target mark device 32 are positioned in this first positional relationship, for example, the target mark device 32 may be suspended and fixed from the ceiling of the building, or installed on a support base located in a predetermined position on the floor of the building.

[0024] Furthermore, the target mark device 32 is equipped with an external camera 46. The external camera 46 is positioned and positioned to capture images of the mirror surface of the rearview mirror 22 inside the vehicle 20 when the target mark device 32 and the vehicle 20 are arranged in a first positional relationship. For example, the external camera 46 is positioned approximately in the center of the target mark 44, and its objective lens is oriented in a direction that allows it to capture images of the interior of the vehicle 20 through the rear window of the vehicle 20.

[0025] The display device 34 displays the image captured by the external camera 46 during the mirror adjustment process. The display device 34 is positioned in a location visible to the operator during the mirror adjustment process. For example, the display device 34 may be a portable information terminal device that can be carried by the operator. The display device 34 may be connected to the external camera 46 by a wired connection or by a wireless connection.

[0026] Figure 4 shows the mirror image 50 when the rearview mirror 22 is not in the reference position. Figure 5 shows the mirror image 50 when the rearview mirror 22 is in the reference position.

[0027] The standard orientation of the rearview mirror 22 is the orientation in which the image 52 of the target mark 44 reflected on the mirror surface of the rearview mirror 22, as included in the mirror image 50, is in a predetermined position and shape relative to the mirror surface of the rearview mirror 22. Therefore, in the mirror adjustment process, the operator manually adjusts the orientation of the rearview mirror 22 so that the image 52 of the target mark 44 reflected on the rearview mirror 22, as included in the mirror image 50, is in a predetermined position and shape relative to the mirror surface of the rearview mirror 22.

[0028] For example, in this embodiment, the external camera 46 is positioned approximately in the center of the target mark 44. In this embodiment, the reference orientation of the rearview mirror 22 is such that the mirror surface is perpendicular to the optical axis of the external camera 46. In this case, the shape of the image 52 of the target mark 44 reflected in the rearview mirror 22 in the reference orientation matches the shape of the actual target mark 44. Therefore, in this embodiment, the operator adjusts the orientation of the rearview mirror 22 so that the shape of the image 52 of the target mark 44 included in the mirror image 50 matches the shape of the actual target mark 44. This allows the operator to easily adjust the rearview mirror 22 to the reference orientation.

[0029] Furthermore, during the mirror adjustment process, the rearview mirror 22 may have a mirror sheet 56 attached to it that shows a mark area 58 on the mirror surface where the image 52 of the target mark 44 is reflected. For example, the mirror sheet 56 is attached so as to cover the entire mirror surface of the rearview mirror 22. Moreover, the mirror sheet 56 leaves open the mark area 58 on the mirror surface where the target mark 44 is reflected when the rearview mirror 22 is in the reference position. This allows the operator to determine, for example, as shown in Figure 4, that the rearview mirror 22 is not in the reference position if the image 52 of the target mark 44 does not coincide with the mark area 58. Also, the operator can determine that the rearview mirror 22 is in the reference position if the image 52 of the target mark 44 coincides with the mark area 58, for example, as shown in Figure 5.

[0030] Thus, the mark area 58 shown on the mirror sheet 56 coincides with the image 52 of the target mark 44 reflected on the mirror surface of the rearview mirror 22 in the reference position. This allows the operator to easily adjust the rearview mirror 22 to the reference position.

[0031] Furthermore, the rearview mirror 22 may have a protective sheet attached to the mirror sheet 56 at a stage prior to the mirror adjustment process. The protective sheet is attached so as to protect the areas of the mirror surface on the mirror sheet 56 that are exposed, such as the mark area 58. Then, at the stage when the mirror adjustment process begins, the worker removes only the protective sheet. This protects the rearview mirror 22 from scratches on the mirror surface at a stage prior to the mirror adjustment process. The worker may also remove the mirror sheet 56 at the stage when the vehicle 20 is shipped.

[0032] Figure 6 is a flowchart showing the process flow of the mirror position adjustment method according to this embodiment. The mirror position adjustment method according to this embodiment is carried out in the order shown in Figure 6.

[0033] First, in S11, the mirror position adjustment method involves an installation process in which the target mark device 32 and the vehicle 20 are positioned in a first positional relationship. For example, in the installation process, the worker moves the vehicle 20 relative to the target mark device 32, thereby positioning the target mark device 32 and the vehicle 20 in the first positional relationship. By positioning the target mark device 32 and the vehicle 20 in the first positional relationship, the external camera 46 can generate a mirror image 50 that captures the mirror surface of the rearview mirror 22 reflecting the target mark 44.

[0034] Next, in S12, the mirror position adjustment method is performed in the mirror adjustment step. In the mirror adjustment step, the operator starts the imaging operation of the external camera 46 and has the external camera 46 image the mirror surface of the rearview mirror 22 in which the target mark 44 is reflected. As a result, the external camera 46 can generate a mirror image 50 and output it to the display device 34. Furthermore, in the mirror adjustment step, the operator operates the display device 34 to display the mirror image 50 on the display device 34. Then, in the mirror adjustment step, the operator adjusts the posture of the rearview mirror 22 to a reference posture in which the image 52 of the target mark 44 reflected in the rearview mirror 22, which is included in the mirror image 50, is in a predetermined position and shape.

[0035] The worker then terminates this process when the image 52 of the target mark 44 reflected in the rearview mirror 22, which is included in the mirror image 50, is in a reference position where it is in a predetermined position and has a predetermined shape relative to the mirror surface of the rearview mirror 22. For example, if a mirror sheet 56 is attached to the rearview mirror 22, the worker terminates this process when the image 52 of the target mark 44 coincides with the mark area 58.

[0036] The mirror image 50 displayed on the display device 34 is an image with a fixed camera viewpoint. Therefore, even if the operator's viewpoint moves, they can adjust the position of the rearview mirror 22 while checking the mirror surface of the rearview mirror 22 as seen from the fixed camera viewpoint. This allows the operator to easily and accurately adjust the rearview mirror 22 to the reference position.

[0037] Next, in S13, the mirror position adjustment method is registered. In the registration process, the operator causes the rearview mirror 22 to be adjusted to the standard position and to capture an image of the interior of the vehicle 20 using the camera 24 installed on the rearview mirror 22. For example, the operator operates the driver monitoring system 30 to cause the camera 24 installed on the rearview mirror 22 to capture an image of the interior of the vehicle 20.

[0038] Then, in the registration process, the operator operates the driver monitoring system 30 to register information based on the reference image obtained by imaging. For example, the driver monitoring system 30 may store the reference image as information based on the reference image. Alternatively, for example, the driver monitoring system 30 may perform image analysis on the reference image to detect feature points representing the interior of the vehicle 20, such as seats and headrests, and generate feature point information representing these feature points. The driver monitoring system 30 may then store the feature point information as information based on the reference image.

[0039] In this registration process, when the rearview mirror 22 is precisely adjusted to the reference position, information based on the reference image captured by the camera 24 installed on the rearview mirror 22 can be registered in the driver monitoring system 30. As a result, the driver monitoring system 30 can detect the driver's gaze direction or face orientation, etc., based on the reference position of the rearview mirror 22, regardless of the position of the rearview mirror 22.

[0040] The mirror position adjustment process ends when the registration step in S13 is completed.

[0041] Figure 7 shows the configuration of the driver monitoring system 30.

[0042] The driver monitoring system 30 includes a camera 24, a storage device 62, an ECU (Electronic Control Unit) 64, a display device 66, and a speaker 68.

[0043] Camera 24 is mounted on the rearview mirror 22. Camera 24 is fixed to the rearview mirror 22 and captures an image of the driver seated in the driver's seat from the rearview mirror 22. Camera 24 provides the driver image it captures to the ECU 64.

[0044] The storage device 62 is built into the rearview mirror 22. The storage device 62 stores a reference image generated by the mirror position adjustment method according to the embodiment. The storage device 62 may also store feature point information generated based on the reference image.

[0045] The ECU64 is built into the rearview mirror 22. The ECU64 is an information processing device that executes a pre-stored program. The ECU64 acquires a driver image from the camera 24. From the acquired driver image, the ECU64 detects the driver's gaze direction and the orientation of the driver's face. In this case, the ECU64 corrects the gaze direction and face orientation detected from the driver image based on a reference image or feature point information stored in the storage device 62. Based on the corrected gaze direction and face orientation, the ECU64 detects the driver's state, such as driver inattention, fatigue, or drowsiness. If the ECU64 detects driver inattention, fatigue, or drowsiness, it outputs detection information from the display device 66 or speaker 68.

[0046] The display device 66 is located outside the rearview mirror 22. The display device 66 may be, for example, a monitor device provided in a navigation system, or a monitor device provided on the control panel of the vehicle 20. The display device 66 outputs detection information output from the ECU 64.

[0047] The speaker 68 may be located outside the rearview mirror 22 or located on the rearview mirror 22. If the speaker 68 is located outside the rearview mirror 22, it may be, for example, a speaker device provided in the navigation system or a speaker device provided in the control panel of the vehicle 20. The speaker 68 outputs detection information output from the ECU 64.

[0048] Figure 8 is a flowchart showing the processing flow of ECU64.

[0049] For example, the ECU64 executes the processes from S21 to S28 each time the camera 24 outputs a driver image, that is, each frame.

[0050] First, in S21, the ECU64 acquires a driver image. Next, in S22, the ECU64 performs image analysis on the acquired driver image to detect characteristic points of the vehicle's interior. For example, the ECU64 detects characteristic points of the interior, such as seats and headrests, that are included in the driver image.

[0051] Next, in S23, the ECU 64 acquires a reference image from the storage device 62. Then, in S24, the ECU 64 performs image analysis on the acquired reference image to detect feature points in the interior of the vehicle 20. For example, the ECU 64 detects feature points of the interior, such as seats and headrests, that are included in the reference image. If feature point information representing the feature points of the interior, such as seats and headrests, included in the reference image is stored in the storage device 62, the ECU 64 may acquire the feature point information stored in the storage device 62 instead of performing the processes in S23 and S24.

[0052] Next, in S25, the ECU64 calculates the difference angle, which represents the angular difference between the optical axis of the camera 24 when the reference image is captured and the optical axis of the camera 24 when the driver image is captured. For example, the ECU64 calculates the difference angle based on the amount of change between the position of the interior feature points in the reference image and the position of the interior feature points in the driver image.

[0053] Furthermore, the difference angle remains unchanged as long as the optical axis of the camera 24 that captures the driver image does not change. In other words, the difference angle does not change as long as the orientation of the rearview mirror 22 does not change. Therefore, the ECU 64 may perform the processing from S22 to S25 not frame by frame, but at intervals longer than the frame interval, or whenever a predetermined event occurs.

[0054] Next, in S26, the ECU64 calculates the driver's gaze direction and the orientation of the driver's face based on the driver image. The ECU64 may calculate either the gaze direction or the orientation of the face, or both.

[0055] Next, in S27, the ECU64 corrects the driver's gaze direction and the direction of the driver's face based on the difference angle calculated immediately before. For example, the ECU64 corrects the driver's gaze direction and the direction of the driver's face by adding or subtracting the difference angle to the gaze direction and the direction of the face calculated in S26.

[0056] Next, in S28, the ECU 64 performs driver monitoring processing based on the corrected gaze direction and face orientation. For example, the ECU 64 detects driver inattention, fatigue, or drowsiness based on the corrected gaze direction and face orientation. If the ECU 64 detects driver inattention, fatigue, or drowsiness, it outputs detection information from the display device 66 or speaker 68.

[0057] The ECU64 repeats the processes from S21 to S28. This allows the ECU64 to accurately detect driver inattention, fatigue, or drowsiness, and prompt the driver to drive safely.

[0058] As described above, the mirror position adjustment method according to this embodiment involves arranging the target mark device 32 and the vehicle 20 in a first positional relationship, and adjusting the posture of the rearview mirror 22 to a reference posture in which the image 52 of the target mark 44 reflected in the rearview mirror 22, which is included in the mirror image 50, is in a predetermined position and shape. Thus, according to the mirror position adjustment method according to this embodiment, the rearview mirror 22 can be easily and accurately adjusted to the reference posture.

[0059] Furthermore, in the mirror position adjustment method according to this embodiment, when the rearview mirror 22 is accurately adjusted to a reference position, information based on a reference image captured by a camera 24 installed on the rearview mirror 22 is registered in the driver monitoring system 30. As a result, the driver monitoring system 30 can detect the driver's gaze direction or face orientation, etc., based on the reference position of the rearview mirror 22, regardless of the position of the rearview mirror 22. Therefore, even if the position of the rearview mirror 22 is changed by the driver, for example, the driver monitoring system 30 can accurately monitor the driver's condition and encourage safe driving.

[0060] Although embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. Various modifications can be made to the embodiments. [Explanation of Symbols]

[0061] 10...Support device, 20...Vehicle, 22...Rearview mirror, 24...Camera, 30...Driver monitoring system, 32...Target mark device, 34...Display device, 42...Main surface, 44...Target mark, 46...External camera, 50...Mirror image, 52...Image, 56...Mirror sheet, 58...Mark area, 62...Storage device, 64...ECU, 66...Display device, 68...Speaker

Claims

1. An installation process involves arranging a target mark device that displays a target mark and a vehicle equipped with a rearview mirror that has a camera in a predetermined first positional relationship. A mirror adjustment step involves generating a mirror image by capturing the mirror surface of the rearview mirror, in which the target mark is reflected, with an external camera, and adjusting the position of the rearview mirror to a reference position in which the image of the target mark reflected in the rearview mirror, included in the mirror image, is in a predetermined position and shape. A method for adjusting the mirror position, including the mirror position.

2. With the rearview mirror adjusted to the reference position, the camera captures an image of the vehicle's interior, and the information based on the captured reference image is registered in a driver monitoring system that monitors the driver's condition. The mirror position adjustment method according to claim 1, further comprising:

3. The first positional relationship is such that the target mark device is positioned at the rear outside the vehicle, and the target mark is reflected in the mirror surface of the rearview mirror. The external camera is positioned and oriented to capture the rearview mirror when the target mark device and the vehicle are arranged in a first positional relationship. The external camera is located within the target mark. The mirror position adjustment method according to claim 1.

4. In the mirror adjustment process, the rearview mirror is fitted with a mirror sheet indicating the mark area on the mirror surface where the target mark is reflected. The mirror position adjustment method according to claim 3.

Images