Calibration device and calibration method

Abstract

To provide a technique for improving accuracy of line-of-sight detection.SOLUTION: A calibration device for calibrating line-of-sight detection processing on the basis of, an image of a face of a user includes: an instruction unit configured to instruct the user to look at each of a plurality of positions; and an acquisition unit configured to acquire, for each of the plurality of positions, the image of the face of the user looking at each position. The instruction unit is configured to, when giving an instruction to look at a position that is not in front of the user of the plurality of positions without moving the face, give an instruction to look in front of the user each time a position at which the user is instructed to look in order to obtain the image of the face of the user switches from one position that is not in front of the user to another position that is not in front of the user.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a calibration device and a calibration method. [Background technology]

[0002] In order to provide appropriate services to vehicle occupants and record the actions of the occupants, a gaze detection device for detecting the gaze of the occupants may be installed in a vehicle. The gaze detection device detects the gaze of a person based on an image of the person's face. As a method for improving the accuracy of gaze detection, Patent Document 1 describes a technique for calibrating the gaze detection result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-129898 A Summary of the Invention [Problem to be solved by the invention]

[0004] Vehicle occupants (particularly the driver) move their gaze with different characteristics depending on the speed of the vehicle. Therefore, the calibration of the gaze detection process is also performed according to the respective characteristics. Proper calibration leads to improvement in the accuracy of gaze detection. An object of some aspects of the present invention is to provide a technique for improving the accuracy of gaze detection. [Means for solving the problem]

[0005] In view of the above problem, there is provided a calibration device that calibrates a gaze detection process based on an image of a user's face, the calibration device including: an instruction means for instructing the user to look at each of a plurality of positions; and an acquisition means for acquiring, for each of the plurality of positions, an image of the user's face looking at each position; a calibration means for calibrating the gaze detection process based on the image acquired by the acquisition means;the instruction means instructs the user to look at a position that is not in front of the user among the plurality of positions before instructing the user to look at a position that is not in front of the user without moving the face, and instructs the user to look at a position that is not in front of the user each time the position that the user is instructed to look at in order to obtain an image of the user's face is switched from one position that is not in front of the user to another position that is not in front of the user. and when the instruction means instructs the user to look at a position that is not in front of the user among the plurality of positions without moving the face, the calibration means calibrates the gaze detection process based on the position of the pupil in the image. , a calibration device is provided. Effect of the Invention

[0006] The above means improves the accuracy of gaze detection. [Brief description of the drawings]

[0007] [Figure 1] 1 is a schematic diagram illustrating an example configuration of a vehicle according to some embodiments of the present disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an example of the configuration of a drive recorder according to some embodiments of the present disclosure. [Diagram 3] 1A and 1B are diagrams illustrating example images of a vehicle interior according to some embodiments of the present disclosure. [Figure 4] FIG. 1 is a flow diagram illustrating an example of the operation of a drive recorder according to some embodiments of the present disclosure. [Diagram 5] FIG. 1 is a flow diagram illustrating an example of the operation of a drive recorder according to some embodiments of the present disclosure. [Figure 6] Schematic diagrams illustrating the movement of the driver's line of sight in some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features among the multiple features described in the embodiments may be arbitrarily combined. In addition, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0009] Some embodiments of the present invention relate to a gaze detection device for detecting a gaze of a user. The gaze detection device also functions as a calibration device for calibrating the gaze detection process based on an image of the user's face. In the following description, an embodiment will be described in which the user is an occupant of a vehicle. That is, gaze detection and calibration are performed inside the vehicle. However, the following embodiment is also applicable to gaze detection and calibration in other environments. The vehicle is typically a four-wheeled vehicle. However, the following embodiment may be applied to other types of vehicles. The occupant of the vehicle is typically the driver of the vehicle. However, the following embodiment may be applied to an occupant other than the driver, for example, an occupant riding in a passenger seat. In the following description, a case will be described in which gaze detection of the driver as an occupant of the vehicle is performed.

[0010] An example of the mounting position of the gaze detection device will be described with reference to Fig. 1. A drive recorder 101 is mounted on a vehicle 100. In the example of Fig. 1, the drive recorder 101 is mounted on the rearview mirror of the vehicle 100. The mounting position of the drive recorder 101 is not limited to this, and may be any position that can capture the faces of the occupants of the vehicle 100. The drive recorder 101 functions as the gaze detection device.

[0011] The drive recorder 101 may be capable of operating independently, or may be capable of operating in cooperation with a control device (e.g., a control ECU (Electronic Control Unit) 102) of the vehicle 100. When the drive recorder 101 is capable of operating independently, the drive recorder 101 may receive power supply from the vehicle 100.

[0012] 2, a configuration example of the drive recorder 101 will be described. The drive recorder 101 includes a processor 201, a storage device 202, a front camera 203, an in-vehicle camera 204, an input device 205, an output device 206, and a communication device 207.

[0013] The processor 201 controls the overall operation of the drive recorder 101. The processor 201 may be realized by, for example, a CPU (Central Processing Unit). The storage device 202 stores data related to the operation of the drive recorder 101. For example, the storage device 202 may store a program that defines the operation of the drive recorder 101, temporary data during execution of the program, images captured by the front camera 203 and the in-vehicle camera 204, and results of detection of the driver's line of sight. The storage device 202 may be realized by a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). Furthermore, the storage device 202 may include an auxiliary storage device such as an SSD (Solid State Drive).

[0014] The front camera 203 is a camera for capturing images of the scenery in front of the vehicle 100. The front camera 203 may be, for example, a wide-angle camera. The front camera 203 is located on the front side of the drive recorder 101 (the front side of the vehicle 100). The in-vehicle camera 204 is a camera for capturing images of the scenery in the cabin of the vehicle 100. The in-vehicle camera 204 may be, for example, a fisheye camera. The in-vehicle camera 204 is located on the rear side of the drive recorder 101 (the rear side of the vehicle 100).

[0015] The input device 205 is a device for inputting instructions from an occupant of the vehicle 100. The input device 205 is realized by, for example, a button or a touch panel. The output device 206 is a device for outputting information to an occupant of the vehicle 100. The output device 206 may be a display device (e.g., a liquid crystal display or an indicator) that provides visual information, or may be an audio device (e.g., a speaker) that provides auditory information, or may include both of these. Instead of or in addition to the drive recorder 101 having the output device 206, the drive recorder 101 may instruct the vehicle 100 to output information. The vehicle 100 that has received the instruction outputs information to the occupant from its own output device (not shown).

[0016] The communication device 207 is a device for the drive recorder 101 to communicate with other devices (e.g., the control ECU 102). The communication device 207 may communicate with other devices via a wired connection, or may communicate with other devices wirelessly. The wireless communication may be communication conforming to a protocol such as short-range communication (e.g., Bluetooth (registered trademark)). The communication device 207 may be optional.

[0017] 3 shows an example of an image 300 captured by the in-vehicle camera 204 of the drive recorder 101. The image 300 shows the interior of the vehicle 100, and more specifically, shows the driver 301. In this embodiment, the vehicle 100 has a right-hand drive, so the driver 301 is located on the left side of the image 300. If the vehicle 100 has a left-hand drive, the opposite would be true.

[0018] Next, a description will be given of the gaze detection process executed by the drive recorder 101. The gaze detection process is a process for detecting the driver's gaze (where the driver is looking). The gaze detection process is performed by analyzing an image of the driver's face captured by the in-vehicle camera 204.

[0019] Generally, when the vehicle is stopped, the driver of the vehicle tends to move the face to direct the gaze to an object, and when the vehicle is moving, the driver tends to move only the pupils to direct the gaze to an object without moving the face as much as possible (while keeping the face facing forward). Therefore, the drive recorder 101 performs the gaze detection process taking into consideration the speed of the vehicle 100. For example, when the speed of the vehicle 100 is zero, the drive recorder 101 performs the gaze detection process by prioritizing the direction of the face. In this way, the gaze detection process that prioritizes the direction of the face is referred to as face priority processing. When the speed of the vehicle 100 is non-zero, the drive recorder 101 performs the gaze detection process by prioritizing the direction of the pupil. In this way, the gaze detection process that prioritizes the direction of the pupil is referred to as pupil priority processing. The pupil may be the iris, may be the pupil, or may be a combination of the iris and the pupil.

[0020] The drive recorder 101 may perform face priority processing and pupil priority processing in a manner similar to existing methods. For example, the drive recorder 101 may perform these processes as follows. In some embodiments, to perform face priority processing, the drive recorder 101 detects the driver's gaze based on the position of each feature (eyes, nose, mouth, etc.) in the outline of the driver's face included in the image. In the face priority processing, the pupil position may not be taken into consideration, or a low weight may be assigned to the gaze based on the pupil position. In the pupil priority processing, the drive recorder 101 detects the driver's gaze based on the pupil position of the driver's eyes included in the image. In the pupil priority processing, the face direction may not be taken into consideration, or a low weight may be assigned to the gaze based on the face direction.

[0021] In another embodiment, the drive recorder 101 may perform face priority processing and pupil priority processing based on a model obtained by machine learning. For example, the storage device 202 of the drive recorder 101 may store two DNN (Deep Neural Network) models: a model (hereinafter referred to as a face priority model) that learns as teacher data an image of the driver obtained by the driver moving his / her face and directing his / her gaze to an object, and a model (hereinafter referred to as a pupil priority model) that learns as teacher data an image of the driver obtained by the driver moving only his / her pupils while keeping his / her face as small as possible and directing his / her gaze to an object. In order to perform the face priority processing, the drive recorder 101 detects the driver's gaze by applying the acquired image of the driver to the face priority model. In order to perform the pupil priority processing, the drive recorder 101 detects the driver's gaze by applying the acquired image of the driver to the pupil priority model.

[0022] The drive recorder 101 may alternatively select face priority processing and pupil priority processing based on the speed of the vehicle 100. For example, the drive recorder 101 may execute face priority processing when the speed of the vehicle 100 is zero, and execute pupil priority processing when the speed of the vehicle 100 is non-zero. Alternatively, the drive recorder 101 may detect the line of sight by weighting the results of the face priority processing and the pupil priority processing using a weight obtained as a function of the speed of the vehicle 100. The drive recorder 101 may determine the speed of the vehicle 100 based on an image captured by the front camera 203 or the in-vehicle camera 204, or may acquire the speed from the vehicle 100.

[0023] The drive recorder 101 performs calibration of the gaze detection process to improve the accuracy of gaze detection. Calibration is a process of adjusting parameters of the gaze detection process to reduce an error between the gaze recognized by the drive recorder 101 and the actual gaze. Specifically, the drive recorder 101 instructs the driver to look at a specific calibration position (for example, the front or a corner of the front window) and associates the calibration position with an image of the face of the driver looking at the calibration position. The calibration position is a position that the driver is instructed to look at in order to perform calibration. The drive recorder 101 performs calibration separately for each of the face priority process and the pupil priority process. Either the face priority process or the pupil priority process may be executed first.

[0024] A calibration method for face priority processing will be described with reference to Fig. 4. The operation of Fig. 4 may be performed by the processor 201 of the drive recorder 101 executing a program read into the storage device 202 (specifically, a memory). Alternatively, a part or all of the operation of Fig. 4 may be executed by a dedicated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The operation of Fig. 4 may be started in response to receiving an instruction to start calibration from the driver. The calibration is performed while the vehicle is stopped to ensure safety.

[0025] In step S401, the processor 201 instructs the driver to assume a driving posture. This instruction is output to the passengers by, for example, the output device 206 (for example, a speaker). Hereinafter, instructions to the driver may be output in a similar manner. The driving posture is, for example, a posture in which the driver sits in the seat and grips the steering wheel as when driving. As an instruction to have the driver assume such a posture, the processor 201 may issue a notification to the driver, such as, for example, "Please grip the steering wheel with both hands and assume a driving posture." This notification may be issued by voice or by display on a display.

[0026] In step S402, the processor 201 selects one of the multiple calibration positions that have not been processed yet among the multiple calibration positions that have been set in advance. For example, the multiple calibration positions that have been set in advance include a position in front of the driver and the four corners of the windshield. The multiple calibration positions may not include some of these positions, or may include other positions. The settings of the multiple calibration positions are stored in the storage device 202 in advance. The unprocessed calibration position is a calibration position that has not been subjected to the processing in steps S403 and S404 described later.

[0027] In step S403, the processor 201 instructs the driver to move his / her face to look at one selected calibration position (e.g., the upper right of the windshield). The processor 201 then waits for a predetermined time that is considered to be required for the driver to move his / her line of sight. Alternatively, the processor 201 may determine that the direction of the driver's face has started to move and then stopped based on an image from the in-vehicle camera 204. The instruction to move the face to look at the calibration position may explicitly include an instruction to move the face, for example, "Move your face and look at the upper right of the windshield." Alternatively, the instruction to move the face to look at the calibration position may implicitly include an instruction to move the face, for example, "Look naturally at the upper right of the windshield." The vehicle is stopped when this step is executed. Therefore, it is considered that the driver will move his / her face to look at the calibration position if instructed to look naturally.

[0028] In step S404, the processor 201 acquires an image of the driver captured by the in-vehicle camera 204. The image includes the face of the driver looking at the indicated calibration position. The processor 201 then stores the acquired image in the storage device 202 in association with the selected calibration position.

[0029] In step S405, the processor 201 determines whether or not an unprocessed calibration position remains among the multiple calibration positions set in advance. If an unprocessed calibration position remains ("YES" in step S405), the processor 201 returns the process to step S402 and repeats steps S402 to S405. If an unprocessed calibration position does not remain ("NO" in step S405), the processor 201 transitions the process to step S406.

[0030] In step S406, the processor 201 calibrates the face priority processing based on the image of the driver's face stored in the storage device 202 in association with a plurality of calibration positions. For example, the processor 201 may adjust the ratio of the gaze movement distance to the amount of movement of the face direction based on the amount of movement (movement angle) of the face direction between the images and the distance between the calibration positions. When the face priority processing is performed by machine learning, the processor 201 performs the calibration, for example, as follows. First, as a model of the DNN, a model having an input parameter that can correct the output of the DNN is selected. During the calibration, the processor 201 adjusts this parameter so that the error between the output of the DNN and the actual calibration position is minimized. The distance between the calibration positions may differ depending on the vehicle model, and may be stored in the storage device 202 in advance.

[0031] A method for calibrating the pupil priority processing will be described with reference to Fig. 5. The operation of Fig. 5 may be performed by the processor 201 of the drive recorder 101 executing a program read into the storage device 202 (specifically, a memory). Alternatively, a part or all of the operation of Fig. 5 may be executed by a dedicated circuit such as an ASIC or an FPGA. The operation of Fig. 5 may be started in response to receiving an instruction to start calibration from the driver. The calibration is performed while the vehicle is stopped to ensure safety.

[0032] In step S501, the processor 201 instructs the driver to assume a driving posture. Step S501 may be similar to step S401.

[0033] In step S502, the processor 201 selects one of a plurality of calibration positions that have not been processed yet among a plurality of calibration positions that have been set in advance. Step S502 may be the same as step S402. The plurality of calibration positions in the calibration of the pupil priority processing and the plurality of calibration positions in the calibration of the face priority processing may be the same or different.

[0034] In step S503, the processor 201 instructs the driver to look at one selected calibration position (e.g., the upper right of the windshield) without moving the face. The processor 201 then waits for a predetermined time that is considered to be required for the driver to move his / her line of sight. Alternatively, the processor 201 may determine that the position of the driver's eyes has started to move and then stopped moving based on an image from the in-vehicle camera 204. The instruction to look at the calibration position without moving the face may explicitly include an instruction to not move the face, for example, "Look at the upper right of the windshield without moving the face." Alternatively, the instruction to look at the calibration position without moving the face may implicitly include an instruction to not move the face, for example, "Look at the upper right of the windshield with only your eyes."

[0035] In step S504, the processor 201 acquires an image of the driver captured by the in-vehicle camera 204. This image includes the face (particularly the pupils) of the driver looking at the calibration position. The processor 201 then stores the acquired image in the storage device 202 in association with the selected calibration position.

[0036] In step S505, processor 201 determines whether or not an unprocessed calibration position remains among the multiple calibration positions set in advance. If an unprocessed calibration position remains ("YES" in step S505), processor 201 returns the process to step S502 after executing step S506, and repeats steps S502 to S505. If an unprocessed calibration position does not remain ("NO" in step S505), processor 201 transitions the process to step S507. In step S506, processor 201 instructs the driver to look ahead. Details of this step will be described later.

[0037] In step S507, processor 201 calibrates pupil priority processing based on the image of the driver's face stored in storage device 202 in association with a plurality of calibration positions. For example, processor 201 may adjust the ratio of the gaze movement distance to the pupil movement amount based on the pupil movement amount (movement angle) between images and the distance between calibration positions. When pupil priority processing is performed by machine learning, processor 201 may calibrate pupil priority processing in the same manner as the calibration of face priority processing described above.

[0038] With reference to FIG. 6, a change in the driver's line of sight in the calibration method of the pupil priority processing in FIG. 5 will be described. A field of view 600 indicates the field of view of the driver. The field of view 600 includes components in the front of the cabin of the vehicle 100, such as a windshield 601 and a steering wheel. In the example of FIG. 6, the multiple calibration positions include a position 602 in front of the driver and positions 603 to 606 at the four corners of the windshield 601. None of the positions 603 to 606 are in front of the driver. The order in which the multiple calibration positions are selected may be arbitrary. In the following, it is assumed that the calibration positions are selected in step 502 in the order of positions 602 to 606. In this example, the position 602 in front of the driver is selected first, but the position 602 may be selected in the middle.

[0039] The processor 201 first instructs the driver to look at a position 602 in front of the driver, and obtains an image of the driver's face looking at the position 602. The processor 201 then instructs the driver to look at a position 603 that is not in front of the driver without moving the face, and obtains an image of the driver's face looking at the position 603. Then, the position 604 is selected as the calibration position in step S502. Before instructing the driver to look at the position 603 that is not in front of the driver without moving the face, the processor 201 instructs the driver to look at the position 602 in front of the driver in step S506. This instruction may be an instruction to move the face and look at the front, or an instruction to look at the front without moving the face. Since a picture of the driver's face at the position 602 has already been obtained, the processor 201 does not need to obtain a picture of the driver's face looking at the position 602 again. After the driver looks at the position 602, the processor 201 instructs the driver to look at the position 603 that is not in front of the driver without moving the face. Similarly, each time the calibration position switches to positions 605 and 606, the processor 201 instructs the driver to face forward as indicated by the arrow in FIG.

[0040] Thus, according to some embodiments, when processor 201 instructs the driver to look at a position (position 603-606) that is not in front of the driver among multiple calibration positions (positions 602-606) without moving the face (i.e., in calibration of pupil priority processing), the processor instructs the driver to look at a position in front of the driver (position 602) every time the position (position 602-606) that the driver is instructed to look at to obtain an image of the driver's face switches from one position (any of positions 603-606) that is not in front of the driver to another position (any of positions 603-606) that is not in front of the driver. Even if the driver is instructed to look at a calibration position (e.g., position 603) that is not in front of the driver without moving the face, the direction of the driver's face will change slightly. Therefore, if the driver is instructed to move his / her gaze directly from one calibration position (e.g., position 603) that is not in front of the driver to another calibration position (e.g., position 604) that is not in front of the driver, the pupil may be different from the gaze that is moved from the front of the driver (i.e., position 602) to the other calibration position (e.g., position 604). In this embodiment, the driver is instructed to return his / her gaze to the front of the driver every time the gaze deviates from the front of the driver, so that calibration can be performed with high accuracy.

[0041] On the other hand, as described in FIG. 4, when instructing the driver to move his / her face to look at a position (position 603-606) that is not in front of the driver among a plurality of calibration positions (positions 602-606) (i.e., in calibration of face priority processing), when the position (position 602-606) to be instructed to the driver to look at in order to obtain an image of the driver's face is switched from one position (any of positions 603-606) that is not in front of the driver to another position (any of positions 603-606) that is not in front of the driver, the processor 201 may not instruct the driver to look at the front of the driver (position 602). This is because when the driver moves his / her line of sight by moving his / her face, the direction of the face is considered to be the same regardless of the position the driver was looking at before. By not instructing the driver to look at the front for each calibration position in this way, the time required for calibration can be shortened.

[0042] In the above-mentioned embodiment, the face priority model is a model that is trained using as training data an image of the driver obtained by the driver moving his / her face and directing his / her gaze to an object. Alternatively or in addition, the face priority model may be a model that is adapted while moving the face during calibration. Also, in the above-mentioned embodiment, the pupil priority model is a model that is trained using as training data an image of the driver obtained by the driver moving only the pupils while keeping the face as still as possible and directing the gaze to an object. Alternatively or in addition, the pupil priority model may be a model that is adapted while moving the pupils without moving the face during calibration.

[0043] In the above embodiment, the drive recorder 101 functions as the gaze detection device. Alternatively, the control ECU 102 built into the vehicle 100 may function as the gaze detection device. In this case, the control ECU 102 executes the operation of the drive recorder 101 described in FIG. 4 and FIG. 5. The control ECU 102 may acquire an image of the driver captured by the in-vehicle camera 204 of the drive recorder 101, or may acquire an image of the driver captured by another in-vehicle camera. In the latter case, the drive recorder 101 can be omitted.

[0044] <Summary of the embodiment> <Item 1> A calibration device (101) for calibrating a gaze detection process based on an image of a face of a user (301), comprising: An instruction means (201) for instructing the user to look at each of a plurality of positions (602 to 606); and acquiring means (201) for acquiring, for each of the plurality of positions (602 to 606), an image (300) of the face of the user looking at each position; a calibration device, wherein when instructing the user to look at a position (603 to 606) among the multiple positions that is not in front of the user without moving the face, the instruction means instructs the user to look in front of the user each time the position the user is instructed to look at in order to obtain an image of the user's face switches from one position that is not in front of the user to another position that is not in front of the user. According to this item, the gaze detection process can be appropriately calibrated, thereby improving the accuracy of gaze detection. <Item 2> 2. The calibration device according to claim 1, wherein, when instructing the user to look at a position among the plurality of positions that is not in front of the user without moving the face, the instruction means instructs the user to move the face and look in front of the user each time the position that the user is instructed to look at in order to obtain an image of the user's face switches from one position that is not in front of the user to another position that is not in front of the user. According to this item, the calibration of the gaze detection process can be performed more appropriately. <Item 3> The calibration device of item 1 or 2, wherein the multiple positions include corners of a front window (601) of a vehicle (100) in which the user is riding. According to this item, the calibration of the gaze detection process can be performed more appropriately. <Item 4> 4. The calibration device of any one of claims 1 to 3, wherein the instruction means instructs the user to grip a handle before instructing the user to look at the plurality of positions. This item increases the likelihood that the user will be in a driving position during calibration. <Item 5> 5. The calibration device according to any one of claims 1 to 4, wherein, when instructing the user to move his / her face to look at a position among the plurality of positions that is not in front of the user, the instruction means does not instruct the user to look directly in front of the user when the position the user is instructed to look at in order to obtain an image of the user's face switches from one position that is not in front of the user to another position that is not in front of the user. According to this item, the processing time for calibrating the gaze detection process is reduced. <Item 6> A drive recorder (101) that functions as a calibration device according to any one of items 1 to 5. According to this item, gaze detection can be performed using an external drive recorder. <Item 7> A vehicle (100) incorporating a control device (102) that functions as the calibration device described in any one of items 1 to 5. According to this item, gaze detection can be performed by the vehicle. <Item 8> A program for causing a computer to function as each of the means of the calibration device described in any one of items 1 to 5. According to this item, a program for configuring the above-mentioned calibration device is provided. <Item 9> A method for calibrating a gaze detection process based on an image of a face of a user (301), comprising: an instruction step (S503) of instructing the user to look at each of a plurality of positions (602 to 606); and an acquisition step (S504) of acquiring, for each of the plurality of positions, an image of the face of the user looking at each position, When the instruction step instructs the user to look at one of the multiple positions that is not in front of the user without moving the face, the method instructs the user to look at a position in front of the user each time the position the user is instructed to look at in order to obtain an image of the user's face switches from one position that is not in front of the user to another position that is not in front of the user (S506). According to this item, the gaze detection process can be appropriately calibrated, thereby improving the accuracy of gaze detection.

[0045] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0046] 100 Vehicle, 101 Drive recorder, 601 Front window

Claims

1. A calibration device for calibrating a gaze detection process based on an image of a user's face, comprising: an instruction means for instructing the user to look at each of a plurality of positions; an acquisition means for acquiring, for each of the plurality of positions, an image of the face of the user looking at each position; a calibration unit that calibrates the gaze detection process based on the image acquired by the acquisition unit, The instruction means is instructing the user to look directly in front of the user before instructing the user to look, without moving his / her face, at a position among the plurality of positions that is not directly in front of the user; instructing the user to look in front of the user each time a position to which the user is instructed to look in order to obtain an image of the user's face is switched from one position that is not in front of the user to another position that is not in front of the user; A calibration device in which the calibration means calibrates the gaze detection process based on the position of the pupil in the image when the instruction means instructs the user to look at a position among the multiple positions that is not in front of the user without moving the face.

2. 2. The calibration device of claim 1, wherein when instructing the user to look at a position among the multiple positions that is not in front of the user without moving the face, the instruction means instructs the user to move the face and look directly in front of the user each time the position that the user is instructed to look at in order to obtain an image of the user's face switches from one position that is not in front of the user to another position that is not in front of the user.

3. The calibration device of claim 1 or 2, wherein the plurality of positions include corners of a windshield of a vehicle in which the user is riding.

4. The calibration device according to claim 1 , wherein the instruction means instructs the user to grip a steering wheel used by the user to steer a vehicle in which the user is riding before instructing the user to look at the plurality of positions.

5. when instructing the user to move his / her face to look at a position among the plurality of positions that is not in front of the user, when the position to be instructed to the user to look at in order to obtain an image of the user's face is switched from one position that is not in front of the user to another position that is not in front of the user, the instructing means does not instruct the user to look at a position in front of the user, The calibration device according to any one of claims 1 to 4, further comprising a calibration means for calibrating the gaze detection process based on the positions of facial features in the image when the acquisition means instructs the user to move his or her face to look at a position among the multiple positions that is not in front of the user.

6. A drive recorder that functions as the calibration device according to any one of claims 1 to 5.

7. A vehicle incorporating a control device that functions as the calibration device according to any one of claims 1 to 5.

8. A program for causing a computer to function as each of the means of the calibration device according to any one of claims 1 to 5.

9. 1. A method for calibrating a gaze detection process based on an image of a user's face, comprising: instructing the user to view each of a plurality of locations; acquiring, for each of the plurality of positions, an image of the face of the user looking at each position; a calibration step of calibrating the gaze detection process based on the image acquired in the acquisition step, In the instruction step, instructing the user to look directly in front of the user before instructing the user to look, without moving his / her face, at a position among the plurality of positions that is not directly in front of the user; instructing the user to look in front of the user each time a position to which the user is instructed to look in order to obtain an image of the user's face is switched from one position that is not in front of the user to another position that is not in front of the user; A method in which, in the calibration process, when the instruction process instructs the user to look at a position among the multiple positions that is not in front of the user without moving the face, the gaze detection process is calibrated based on the position of the pupil in the image.

Images