Calibration device, calibration method, driving recorder, vehicle, and storage medium

Abstract

The present invention relates to a correction device, a correction method, a driving recorder, a vehicle, and a storage medium, and provides a technology for improving the accuracy of gaze detection. A correction device that corrects gaze detection processing based on an image of a user's face comprises: an indication unit that indicates the user to look at each of a plurality of positions; and an acquisition unit that acquires an image of the user's face when looking at each of the plurality of positions. When the indication unit instructs the user not to move his face and to look at a position that is not in front of the user among a plurality of positions in order to acquire an image of the user's face, the indication unit instructs the user to look at the position to be looked at in order to acquire an image of the user's face, and each time the position to be looked at switches from a position that is not in front of the user to another position that is not in front of the user, the indication unit instructs the user to look at the front of the user.

Description

Technical Field

[0001] The invention relates to a correction device, a correction method, a driving recorder, a vehicle and a storage medium. Background Art

[0002] To provide appropriate services to vehicle occupants or record their work, a gaze detection device is sometimes installed on a vehicle to detect the occupant's gaze. This device detects the person's gaze based on an image of their face. Patent Document 1 describes a technique for correcting (calibrating) gaze detection results as a method for improving the accuracy of gaze detection.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2017-129898 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] Vehicle occupants (particularly the driver) move their gazes with different characteristics depending on the vehicle's speed. Therefore, gaze detection processing is also calibrated based on these characteristics. Proper calibration can improve the accuracy of gaze detection. One aspect of the present invention is to provide technology for improving the accuracy of gaze detection.

[0008] Solutions for solving problems

[0009] In view of the above problems, according to some embodiments, a correction device is provided, which corrects the line of sight detection processing based on the image of the user's face, in which the correction device is provided with: an indication unit, which indicates the user to look at each of a plurality of positions; and an acquisition unit, which acquires the image of the face of the user when looking at each position for each of the plurality of positions. When the indication unit instructs the user not to move the face and to look at a position among the plurality of positions that is not in front of the user, for obtaining the image of the user's face, whenever the position to be looked at is switched from a position that is not in front of the user to another position that is not in front of the user, the indication unit instructs the user to look at the front of the user.

[0010] According to another embodiment, a method for correcting gaze detection processing based on an image of a user's face is provided, the method comprising: an indication step of instructing the user to look at each of a plurality of positions; and an acquisition step of acquiring, for each of the plurality of positions, an image of the user's face when looking at each position. In the indication step, when instructing the user not to move his face and to look at a position among the plurality of positions that is not in front of the user, the position to be looked at is indicated to the user in order to acquire an image of the user's face, and whenever the position to be looked at switches from a position that is not in front of the user to another position that is not in front of the user, the user is instructed to look at the front of the user.

[0011] Effects of the Invention

[0012] Through the above technology, the accuracy of line of sight detection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram illustrating a configuration example of a vehicle according to a part of the embodiment of the present disclosure.

[0014] Figure 2 This is a block diagram illustrating a configuration example of a drive recorder according to one embodiment of the present disclosure.

[0015] Figure 3 These are diagrams illustrating examples of images within a vehicle cabin according to some embodiments of the present disclosure.

[0016] Figure 4 This is a flowchart illustrating an example of operation of a drive recorder according to one embodiment of the present disclosure.

[0017] Figure 5 This is a flowchart illustrating an example of operation of a drive recorder according to one embodiment of the present disclosure.

[0018] Figure 6 Schematic diagram illustrating movement of a driver's line of sight according to a portion of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following embodiments are described in detail with reference to the accompanying drawings. The following embodiments are not intended to limit the inventions to which the claims relate, nor do all the features described in the embodiments need to be combined. Any combination of two or more of the multiple features described in the embodiments is also possible. Identical or similar structures are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0020] Some embodiments of the present invention relate to a line of sight detection device for detecting the line of sight of a user of a vehicle. The line of sight detection device functions as a correction device that corrects the line of sight detection processing based on an image of the user's face. In the following description, an embodiment is described in which the user is a passenger of a vehicle. That is, line of sight detection and correction are performed inside the vehicle. However, the following embodiments can also be applied to line of sight detection and correction in other environments. The vehicle is typically a four-wheeled vehicle. However, the following embodiments can also be applied to other types of vehicles. The passenger of the vehicle is typically the driver of the vehicle. However, the following embodiments can also be applied to passengers other than the driver, such as a passenger sitting in the front passenger seat. In the following description, a case in which line of sight detection is performed on the driver as a passenger of the vehicle is described.

[0021] Reference Figure 1 An example of the installation position of the sight line detection device is described. A driving recorder 101 is installed in a vehicle 100. Figure 1 In the example shown, the drive recorder 101 is mounted on the rearview mirror of the vehicle 100. The mounting position of the drive recorder 101 is not limited thereto, and may be any position that can capture the face of the occupant of the vehicle 100. The drive recorder 101 functions as a sight line detection device.

[0022] The drive recorder 101 can operate independently or in conjunction with a control device (e.g., an ECU (Electronic Control Unit) 102) of the vehicle 100. When the drive recorder 101 operates independently, it can also receive power from the vehicle 100.

[0023] Reference Figure 2 , a configuration example of the drive recorder 101 is described. The drive recorder 101 includes a processor 201, a storage device 202, a front camera 203, an in-vehicle camera 204, an input device 205, an output device 206, and a communication device 207.

[0024] The processor 201 controls the overall operation of the driving recorder 101. The processor 201 may also be implemented by, for example, a CPU (Central Processing Unit). The storage device 202 stores data related to the operation of the driving recorder 101. For example, the storage device 202 may also store a program that specifies the operation of the driving recorder 101, temporary data when executing the program, images captured by the front camera 203 and the in-vehicle camera 204, the results of the driver's line of sight detection, etc. The storage device 202 may also be implemented by a memory such as RAM (Random Access Memory) or ROM (Read Only Memory). In addition, the storage device 202 may also include an auxiliary storage device such as an SSD (Solid State Drive).

[0025] The front camera 203 is a camera used to capture the scene in front of the vehicle 100. For example, the front camera 203 may be a wide-angle camera. The front camera 203 is located in front of the driving recorder 101 (in front of the vehicle 100). The interior camera 204 is a camera used to capture the scene inside the vehicle 100. For example, the interior camera 204 may be a fisheye camera. The interior camera 204 is located behind the driving recorder 101 (behind the vehicle 100).

[0026] The input device 205 is a device for inputting instructions from the occupants of the vehicle 100. The input device 205 can be implemented by, for example, a button, a touch panel, etc. The output device 206 is a device for outputting information to the occupants of the vehicle 100. The output device 206 may be a display device (for example, a liquid crystal display, an indicator) that provides visual information, or an audio device (for example, a speaker) that provides auditory information, or may include both. The driving recorder 101 has the output device 206, or, instead of or in addition to it, the driving recorder 101 may instruct the vehicle 100 to output information. The vehicle 100 that receives the instruction outputs information to the occupants from its own output device (not shown).

[0027] The communication device 207 is a device used by the drive recorder 101 to communicate with other devices (e.g., the control ECU 102). The communication device 207 can communicate with other devices via wired or wireless communication. The wireless communication can also be based on a short-range communication protocol (e.g., Bluetooth (registered trademark)). The communication device 207 can also be omitted.

[0028] Figure 3An example of an image 300 captured by the in-vehicle camera 204 of the drive recorder 101 is shown. Image 300 shows the interior of the vehicle 100, specifically, the driver 301. In this embodiment, the vehicle 100 has a right-hand steering wheel, so the driver 301 is located on the left side of the image 300. If the vehicle 100 has a left-hand steering wheel, the opposite is true.

[0029] Next, the line of sight detection process performed by the drive recorder 101 will be described. The line of sight detection process detects the driver's line of sight (where the driver is looking). The line of sight detection process is performed by analyzing the image of the driver's face captured by the in-vehicle camera 204.

[0030] Generally speaking, the driver of a vehicle has the following tendency: if the vehicle is stopped, the driver moves the face so that the line of sight is directed toward an object; if the vehicle is moving, the driver tries not to move the face (keeps the face facing forward) and only moves the pupil to direct the line of sight toward the object. Therefore, the driving recorder 101 considers the speed of the vehicle 100 to perform line of sight detection processing. For example, when the speed of the vehicle 100 is zero, the driving recorder 101 prioritizes the line of sight detection processing in the direction of the face. Hereinafter, such a line of sight detection processing prioritized in the direction of the face is referred to as face priority processing. When the speed of the vehicle 100 is not zero, the driving recorder 101 prioritizes the line of sight detection processing in the direction of the pupil. Hereinafter, such a line of sight detection processing prioritized in the direction of the pupil is referred to as pupil priority processing. The pupil can be the iris, the pupil, or a combination of the iris and the pupil.

[0031] The dash cam 101 may also perform face priority processing and pupil priority processing using the same method as the existing method. For example, the dash cam 101 may also perform these processes as follows. In some embodiments, in order to perform face priority processing, the dash cam 101 detects the driver's line of sight based on the position of each part of the driver's facial contour (eyes, nose, mouth, etc.) included in the image. Alternatively, in face priority processing, the position of the pupil is not considered, and a low weight is assigned to the line of sight based on the position of the pupil. In order to perform pupil priority processing, the dash cam 101 detects the driver's line of sight based on the position of the pupil in the driver's eyes included in the image. Alternatively, in pupil priority processing, the direction of the face is not considered, and a low weight is assigned to the line of sight based on the direction of the face.

[0032] In other embodiments, the dashcam 101 may perform face priority processing and pupil priority processing based on a model obtained through machine learning. For example, the storage device 202 of the dashcam 101 may store a DNN (Deep Neural Network) model having the following two models: a model that uses images of the driver obtained by moving the driver's face and directing the driver's gaze toward an object as training data (hereinafter referred to as a face priority model); and a model that uses images of the driver obtained by moving the driver's pupils as little as possible and directing the driver's gaze toward an object as training data (hereinafter referred to as a pupil priority model). In order to perform face priority processing, the dashcam 101 applies the acquired image of the driver to the face priority model to detect the driver's gaze. In order to perform pupil priority processing, the dashcam 101 applies the acquired image of the driver to the pupil priority model to detect the driver's gaze.

[0033] Alternatively, the dashcam 101 may select either face priority processing or pupil priority processing based on the speed of the vehicle 100. For example, the dashcam 101 may perform face priority processing when the speed of the vehicle 100 is zero, and perform pupil priority processing when the speed of the vehicle 100 is not zero. Alternatively, the dashcam 101 may use a weight obtained as a function of the speed of the vehicle 100 to weight the results of the face priority processing and the results of the pupil priority processing, thereby detecting the line of sight. The dashcam 101 may determine the speed of the vehicle 100 based on an image captured by the front camera 203 or the in-vehicle camera 204, or may obtain the speed of the vehicle 100 from the vehicle 100.

[0034] In order to improve the accuracy of line of sight detection, the dashcam 101 corrects (calibrates) the line of sight detection process. Calibration is a process of adjusting the parameters of the line of sight detection process in order to reduce the error between the line of sight recognized by the dashcam 101 and the actual line of sight. Specifically, the dashcam 101 instructs the driver to look at a specific correction position (for example, the front, the corner of the front window), and associates the correction position with the image of the driver's face who is looking at the correction position. The correction position is the position to which the driver is instructed to look for the purpose of correction. The dashcam 101 calibrates the face priority processing and the pupil priority processing individually. Alternatively, one of the face priority processing and the pupil priority processing is performed first.

[0035] Reference Figure 4 The correction method of face priority processing is described. The processor 201 of the driving recorder 101 may execute a program read from the storage device 202 (specifically, the memory) to perform Figure 4 Alternatively, it may be performed by a dedicated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Figure 4 Alternatively, the system may start the calibration process based on a correction instruction from the driver. Figure 4 To ensure safety, make corrections while the vehicle is stopped.

[0036] In step S401, the processor 201 instructs the driver to adopt a driving posture. This instruction is output to the occupants, for example, via the output device 206 (e.g., a horn). Instructions to the driver may be output in the same manner below. The driving posture is, for example, the posture of sitting in the seat and holding the steering wheel as in driving. The processor 201 may, for example, issue a notification to the driver such as, "Please hold the steering wheel with both hands and maintain the driving posture." as an instruction for the driver to adopt this posture. This notification may be made by voice or by displaying it on a display.

[0037] In step S402, processor 201 selects an unprocessed calibration position from a plurality of pre-set calibration positions. For example, the pre-set calibration positions may include the driver's front face and the four corners of the front window. The calibration positions may not include some of these positions, but may include other positions. The settings of the plurality of calibration positions are pre-stored in storage device 202. The unprocessed calibration position is a calibration position that has not been processed in steps S403 and S404, described below.

[0038] In step S403, the processor 201 instructs the driver to move his face and look toward a selected correction position (e.g., the upper right side of the front window). The processor 201 then waits for a predetermined time, which takes into account the time required for the driver to move his line of sight. Alternatively, the processor 201 may determine, based on the image from the in-vehicle camera 204, that the direction of the driver's face begins to move and then stops. The instruction to move the face and look toward the correction position may also include an explicit instruction to move the face, such as "Please move your face and look toward the upper right side of the front window." Alternatively, the instruction to move the face and look toward the correction position may include an implicit instruction to move the face, such as "Please look naturally toward the upper right side of the front window." The vehicle is stopped when this step is performed. Therefore, it is considered that if the driver is instructed to look naturally, he will move his face and look toward the correction position.

[0039] In step S404, processor 201 acquires an image of the driver captured by in-vehicle camera 204. The image includes the driver's face looking at the indicated calibration position. Processor 201 then stores the acquired image in association with the selected calibration position in storage device 202.

[0040] In step S405, processor 201 determines whether any unprocessed calibration positions remain among the pre-set plurality of calibration positions. If any unprocessed calibration positions remain ("YES" in step S405), processor 201 returns the process to step S402 and repeats steps S402 to S405. If no unprocessed calibration positions remain ("NO" in step S405), processor 201 proceeds to step S406.

[0041] In step S406, the processor 201 corrects the face priority processing based on the image of the driver's face stored in the storage device 202 in association with a plurality of correction positions. For example, the processor 201 may adjust the ratio of the movement distance of the line of sight to the movement amount in the face direction based on the movement amount (movement angle) in the face direction between the images and the distance between the correction positions. In the case of face priority processing by machine learning, the processor 201 performs correction, for example, in the following manner. First, as a model of the DNN, a model having input parameters that can correct the output of the DNN is selected. During the correction, the processor 201 adjusts the parameters in such a way as to minimize the error between the output of the DNN and the actual correction position. The distance between the correction positions varies depending on the type of vehicle, and therefore may be stored in the storage device 202 in advance.

[0042] Reference Figure 5 The correction method of pupil priority processing is described. The processor 201 of the driving recorder 101 may execute a program read from the storage device 202 (specifically, the memory), thereby performing Figure 5 Alternatively, dedicated circuits such as ASICs and FPGAs can be used to perform the work. Figure 5 Alternatively, the system may start the calibration process based on a correction instruction from the driver. Figure 5 To ensure safety, make corrections while the vehicle is stopped.

[0043] In step S501, the processor 201 instructs the driver to take a driving posture. Step S501 may be similar to step S401.

[0044] In step S502, processor 201 selects one of the previously set multiple correction positions that has not been processed. Step S502 may be similar to step S402. The multiple correction positions used in the pupil-prioritization correction process may be the same as or different from the multiple correction positions used in the face-prioritization correction process.

[0045] In step S503, the processor 201 instructs the driver to look at a selected correction position without moving his face (for example, the upper right side of the front window). After that, the processor 201 stands by for a predetermined time, which is the time required for the driver to move his line of sight. Alternatively, the processor 201 determines, based on the image from the in-vehicle camera 204, that the position of the driver's pupil starts to move and then stops. The instruction to look at the correction position without moving the face may explicitly include an instruction to look at the correction position, such as "Please look at the upper right side of the front window without moving your face." Alternatively, the instruction to look at the correction position without moving the face may implicitly include an instruction to look at the correction position, such as "Please look at the upper right side of the front window with only your eyes."

[0046] In step S504, processor 201 acquires an image of the driver captured by in-vehicle camera 204. This image includes the driver's face (particularly, the pupils) looking at the calibration position. Processor 201 then stores the acquired image in association with the selected calibration position in storage device 202.

[0047] In step S505, processor 201 determines whether any unprocessed calibration positions remain among the pre-set multiple calibration positions. If any unprocessed calibration positions remain ("Yes" in step S505), processor 201 executes step S506 and then returns the process to step S502, repeating steps S502 to S505. If no unprocessed calibration positions remain ("No" in step S505), processor 201 proceeds to step S507. In step S506, processor 201 instructs the driver to look forward. This step will be described in detail later.

[0048] In step S507, processor 201 corrects pupil priority processing based on the image of the driver's face stored in storage device 202 in association with a plurality of correction positions. For example, processor 201 may adjust the ratio of the distance of line of sight movement relative to the amount of pupil movement based on the amount of pupil movement (angle of movement) between images and the distance between correction positions. Alternatively, when pupil priority processing is performed using machine learning, processor 201 may perform correction for pupil priority processing in the same manner as the correction for face priority processing described above.

[0049] Reference Figure 6 ,illustrate Figure 5 The change of the driver's line of sight in the correction method for pupil priority processing. The field of view 600 shows the driver's field of view. The field of view 600 includes components in front of the vehicle cabin 100 such as the front window 601 and the steering wheel. Figure 6 In the example, the multiple correction positions include position 602 in front of the driver and positions 603 to 606 at the four corners of the front window 601. Positions 603 to 606 are not in front of the driver. The order in which the multiple correction positions are selected can be arbitrary. In the following, in step 502, correction positions are selected sequentially from positions 602 to 606. In this example, position 602 in front of the driver is selected first, but position 602 can also be selected midway.

[0050] The processor 201 first instructs the driver to look at position 602 in front of the driver, and obtains an image of the driver's face when looking at position 602. Thereafter, the processor 201 instructs the driver not to move his face and to look at position 603 that is not in front of the driver, and obtains an image of the driver's face when looking at position 603. Then, position 604 is selected as the correction position in step S502. Before the processor 201 instructs the driver not to move his face and to look at position 604 that is not in front of the driver, in step S506, the processor 201 instructs the driver to look at position 602 in front of the driver. The instruction may be an instruction to move the face and look in the front, or an instruction not to move the face and look in the front. Since the photo of the driver's face when looking at position 602 has been obtained, the processor 201 may not obtain a photo of the driver's face when looking at position 602 again. After the driver looks at position 602, the processor 201 instructs the driver not to move his face and to look at position 604 that is not in front of the driver. Similarly, each time the correction position switches to position 605 or 606, as shown in FIG. Figure 6 As shown by the arrow symbol, the processor 201 instructs the driver to face the front.

[0051] Thus, according to some embodiments, when the processor 201 instructs the driver to look at a position (positions 603-606) that is not in front of the driver among a plurality of correction positions (positions 602-606) without moving his face (i.e., during correction with pupil priority processing), the processor 201 instructs the driver to look at a position (positions 602-606) in order to obtain an image of the driver's face. Whenever the position to look at switches from a position (any of positions 603-606) that is not in front of the driver to another position (any of positions 603-606) that is not in front of the driver, the processor 201 instructs the driver to look at the driver's front (position 602). Even when the driver is instructed to look at a correction position (e.g., position 603) that is not in front of the driver without moving his face, the direction of the driver's face may change slightly. Therefore, when the driver is instructed to directly move their sight line from a correction position not in front of the driver (e.g., position 603) to another correction position not in front of the driver (e.g., position 604), the position of the pupil may be different compared to when the driver moves their sight line from the front (i.e., position 602) to another correction position (e.g., position 604). In this embodiment, the instruction is performed so that the sight line is returned to the front of the driver each time the sight line is separated from the front of the driver, so that correction can be performed with good accuracy.

[0052] On the other hand, it can also be, Figure 4 As described in , when the processor 201 instructs the driver to move the face and look at a position (position 603 to 606) that is not in front of the driver among multiple correction positions (positions 602 to 606) (i.e., in the correction of face priority processing), in order to obtain an image of the driver's face, the driver is instructed to look at the position (position 602 to 606). When the position to look at switches from a position (any one of positions 603 to 606) that is not in front of the driver to another position (any one of positions 603 to 606) that is not in front of the driver, the processor 201 does not instruct to look at the front of the driver (position 602). This is because, considering that when the face is moved to move the line of sight, the direction of the face is the same regardless of the position the driver was looking at before. In this way, by not instructing to look at the front for each correction position, the time required for correction can be shortened.

[0053] In the above-described embodiment, the face-priority model is a model learned using images of the driver as training data, where the driver's image is obtained by moving the driver's face so that the driver's line of sight is directed toward an object. Alternatively, or in addition, the face-priority model is a model that adapts while moving the face during calibration. Furthermore, in the above-described embodiment, the pupil-priority model is a model learned using images of the driver as training data, where the driver's image is obtained by moving the driver's pupils while minimizing the movement of the face. Alternatively, or in addition, the pupil-priority model is a model that adapts while moving the pupils without moving the face during calibration.

[0054] In the above embodiment, the driving recorder 101 functions as the sight line detection device. Alternatively, the control ECU 102 built into the vehicle 100 may function as the sight line detection device. In this case, the control ECU 102 executes Figure 4 as well as Figure 5 The control ECU 102 can obtain the image of the driver captured by the in-vehicle camera 204 of the drive recorder 101, or can obtain the image of the driver captured by another in-vehicle camera. In the latter case, the drive recorder 101 can be omitted.

[0055] <Summary of Implementation Methods>

[0056] <Project 1>

[0057] A calibration device 101 is provided for calibrating gaze detection processing based on an image of a user 301's face. The calibration device comprises:

[0058] an instruction unit 201 for instructing the user to look at each of a plurality of positions 602 to 606; and

[0059] The acquiring unit 201 acquires the image 300 of the user's face when looking at each of the plurality of positions 602 to 606.

[0060] When the indication unit instructs the user not to move the face and to look at positions 603 to 606 among the multiple positions that are not in front of the user, for indicating the position to be looked at in order to obtain an image of the user's face, the indication unit instructs the user to look at the front of the user each time the position to be looked at switches from a position that is not in front of the user to another position that is not in front of the user.

[0061] According to this item, the gaze detection process can be appropriately corrected, thereby improving the accuracy of gaze detection.

[0062] <Project 2>

[0063] According to the correction device described in Item 1, the indication unit indicates the position to be looked at to obtain an image of the user's face when instructing the user not to move the face and to look at a position among the multiple positions that is not in front of the user. Whenever the position to be looked at is switched from a position that is not in front of the user to another position that is not in front of the user, the indication unit instructs the user to move the face and look in front of the user.

[0064] According to this item, it is possible to more appropriately perform correction of the line of sight detection processing.

[0065] <Project 3>

[0066] According to the calibration device of item 1 or 2, the plurality of positions include corners of a front window 601 of the vehicle 100 in which the user rides.

[0067] According to this item, it is possible to more appropriately perform correction of the line of sight detection processing.

[0068] <Item 4>

[0069] According to the correction device according to any one of items 1 to 3, the instruction unit instructs the user to hold a steering wheel before instructing the user to look at the plurality of positions.

[0070] According to this item, the possibility that the user will take a driving posture during the correction increases.

[0071] <Item 5>

[0072] According to the correction device described in any one of items 1 to 4, the indication unit indicates the position to be looked at to the user in order to obtain an image of the user's face when instructing to move the face and look at a position among the multiple positions that is not in front of the user, and when the position to be looked at is switched from a position that is not in front of the user to other positions that are not in front of the user, the indication unit does not instruct to look at the front of the user.

[0073] According to this item, the processing time for correction of the visual line detection process is shortened.

[0074] <Item 6>

[0075] A drive recorder 101 functions as the calibration device according to any one of items 1 to 5.

[0076] According to the project, line of sight detection can be performed using an external driving recorder.

[0077] <Item 7>

[0078] A vehicle 100 includes a control device 102 that functions as the calibration device according to any one of items 1 to 5.

[0079] According to this project, line of sight detection can be performed using vehicles.

[0080] <Item 8>

[0081] A program for causing a computer to function as each unit of the calibration device according to any one of items 1 to 5.

[0082] According to this item, a program constituting the above-mentioned calibration device can be provided.

[0083] <Item 9>

[0084] A method for calibrating gaze detection processing based on an image of a user's 301 face, the method comprising:

[0085] an instruction step ( S503 ), instructing the user to look at each of the plurality of positions 602 to 606 ; and

[0086] Acquisition step (S504), for each of the plurality of positions, acquiring an image of the user's face when looking at each position,

[0087] In the instruction process, when the user is instructed not to move his face and to look at a position among the multiple positions that is not in front of the user, the position to be looked at is instructed to the user in order to obtain an image of the user's face, and each time the position to be looked at is switched from a position that is not in front of the user to another position that is not in front of the user, the user is instructed to look at the front of the user (S506).

[0088] According to this item, the line of sight detection processing can be appropriately corrected, thereby improving the accuracy of line of sight detection.

[0089] The present invention is not limited to the above-described embodiment, and various modifications and changes can be made within the scope of the gist of the invention.

Claims

1. A calibration device for calibrating gaze detection processing based on an image of a user's face, the calibration device comprising: an instruction unit for instructing the user to look at each of a plurality of locations; an acquisition unit that acquires, for each of the plurality of positions, an image of the user's face when the user is looking at the respective position; and a calibration unit that corrects a gaze detection process based on the image of the user's face when looking at each position acquired by the acquisition unit and stored in association with the each position, in, In correction of pupil priority processing, when the instruction unit instructs the user not to move the face and to look at a position that is not in front of the user among the multiple positions, for obtaining an image of the user's face, the instruction unit instructs the user to move the face to look in front of the user each time the position to look is switched from a position that is not in front of the user to another position that is not in front of the user. The pupil priority processing is processing for detecting the user's line of sight based on the position of the user's pupil included in the image. In the correction of face priority processing, the indication unit indicates the position to be looked at to the user in order to obtain an image of the user's face when instructing to move the face and look at a position that is not in front of the user among the multiple positions. When the position to be looked at is switched from a position that is not in front of the user to another position that is not in front of the user, the indication unit does not instruct to look at the front of the user. The face priority processing is a process of detecting the user's line of sight based on the position of each part of the user's face included in the image.

2. The calibration device according to claim 1, characterized in that The plurality of locations include corners of a front window of a vehicle in which the user is riding.

3. The calibration device according to claim 1, characterized in that The instructing unit instructs the user to hold a steering wheel before instructing the user to look at the plurality of positions. 4 . A drive recorder functioning as the calibration device according to claim 1 . 5 . A vehicle having a built-in control device functioning as the correction device according to claim 1 . 6 . A storage medium storing a program for causing a computer to function as each unit of the calibration device according to claim 1 .

7. A correction method for correcting gaze detection processing based on an image of a user's face, the method comprising: an instructing step of instructing the user to look at each of a plurality of positions; as well as an acquisition step of acquiring, for each of the plurality of positions, an image of the user's face when the user is looking at the respective position; as well as a calibration step of correcting a gaze detection process based on the image of the user's face when looking at each position acquired by the acquisition step and stored in association with the each position, Among them, in the correction of pupil priority processing, in the instruction step, when instructing not to move the face and to look at a position that is not in front of the user among the multiple positions, in order to obtain an image of the user's face, the user is instructed to look at a position, and each time the position to look is switched from a position that is not in front of the user to another position that is not in front of the user, the user is instructed to move the face to look in front of the user. The pupil priority processing is a process of detecting the user's line of sight based on the position of the user's pupil included in the image. In the correction of face priority processing, in the instruction process, when instructed to move the face and look at a position that is not in front of the user among the multiple positions, the user is instructed to look at a position in order to obtain an image of the user's face, and when the position to look at is switched from a position that is not in front of the user to another position that is not in front of the user, the instruction to look at the front of the user is not given. The face priority processing is a process of detecting the user's line of sight based on the position of each part of the user's face included in the image.

Images