Gaze direction estimation program, information processing device, and gaze direction estimation method

Abstract

This invention provides a gaze direction estimation system, gaze direction estimation program, information processing device, and gaze direction estimation method that make it easier to estimate the gaze direction of a subject. [Solution] First measurement data indicating the movement state of the head is acquired from a first sensor attached to the head of a person. A first tilt estimation model is used to estimate the tilt angle of the head from the measurement data indicating the movement state of the head, thereby acquiring a first tilt angle of the head estimated from the acquired first measurement data. A first gaze estimation model is used to estimate the gaze direction of the person from the tilt angle of the head, thereby acquiring a first gaze direction of the person estimated from the acquired first tilt angle, and information regarding the acquired first gaze direction is output.

Description

Technical Field

[0001] The present disclosure relates to a line-of-sight direction estimation program, an information processing apparatus, and a line-of-sight direction estimation method.

Background Art

[0002] For example, a technique for estimating the line-of-sight direction of a subject has emerged by using measurement data measured by various sensors (hereinafter also simply referred to as sensors) such as an acceleration sensor and an angular velocity sensor attached to the head of the subject (hereinafter also referred to as a person) (see Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] The gaze direction estimation program, information processing device, and gaze direction estimation method described herein make it easier to estimate the gaze direction of a subject. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the configuration of the information processing system 10 in the first embodiment. [Figure 2] Figure 2 shows an example of the configuration of the information processing device 1 in the first embodiment. [Figure 3] Figure 3 is a diagram illustrating the outline of the gaze estimation process and other related processes in the first embodiment. [Figure 4] Figure 4 is a diagram illustrating the outline of the gaze estimation process and other related processes in the first embodiment. [Figure 5] Figure 5 is a flowchart illustrating the details of the first model generation process in the first embodiment. [Figure 6] Figure 6 is a flowchart illustrating the details of the first model generation process in the first embodiment. [Figure 7] Figure 7 is a flowchart illustrating the details of the first model generation process in the first embodiment. [Figure 8] Figure 8 is a flowchart illustrating the details of the first model generation process in the first embodiment. [Figure 9] Figure 9 is a diagram illustrating the details of the first model generation process in the first embodiment. [Figure 10] Figure 10 is a diagram illustrating the details of the first model generation process in the first embodiment. [Figure 11] Figure 11 is a diagram illustrating the details of the first model generation process in the first embodiment. [Figure 12] Figure 12 is a flowchart illustrating the details of the second model generation process in the first embodiment. [Figure 13] Figure 13 is a flowchart illustrating the details of the second model generation process in the first embodiment. [Figure 14] Figure 14 is a diagram illustrating the details of the second model generation process in the first embodiment. [Figure 15] Figure 15 is a diagram illustrating the details of the second model generation process in the first embodiment. [Figure 16] Figure 16 is a flowchart illustrating the details of the gaze estimation process in the first embodiment. [Figure 17] Figure 17 is a diagram illustrating the details of the gaze estimation process in the first embodiment. [Figure 18] Figure 18 is a diagram illustrating the details of the gaze estimation process in the first embodiment. [Figure 19] Figure 19 is a diagram illustrating the details of the gaze estimation process in the first embodiment. [Figure 20] Figure 20 is a diagram illustrating the gaze estimation process and other aspects in the first modified example. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, such descriptions should not be construed in a limiting sense and do not limit the subject matter recited in the claims. Also, various changes, substitutions, and modifications can be made without departing from the spirit and scope of the present disclosure. Further, different embodiments can be appropriately combined.

[0010] [Configuration Example of Information Processing System 10 in the First Embodiment] First, a configuration example of the information processing system 10 in the first embodiment will be described. FIG. 1 is a diagram showing a configuration example of the information processing system 10 in the first embodiment.

[0011] The information processing system 10 includes, for example, an information processing apparatus 1 (hereinafter also referred to as a line-of-sight direction estimation apparatus 1), a sensor 2a (hereinafter also referred to as a first sensor 2a), and a sensor 2b (hereinafter also referred to as a second sensor 2b). The information processing apparatus 1 is connected to the sensor 2a and the sensor 2b, respectively, by wire or wirelessly, and can communicate with each of the sensor 2a and the sensor 2b. [[ID=十二]]

[0012] The sensor 2a is, for example, a sensor group including at least one of an acceleration sensor and an angular velocity sensor. The sensor 2a is attached to the head of the subject OB, for example. That is, the sensor 2a is a sensor that measures acceleration data and angular velocity data (hereinafter collectively also simply referred to as first measurement data) at the head of the subject OB. [[ID=十六]]

[0013] Specifically, the sensor 2a continuously measures acceleration data generated along with the movement of the head of the subject OB, for example. In other words, the sensor 2a measures time-series data of acceleration data generated along with the movement of the head of the subject OB, for example. Then, the sensor 2a continuously transmits the measured acceleration data to the information processing apparatus 1, for example.

[0014] Furthermore, sensor 2a continuously measures angular velocity data generated in conjunction with the movement of the subject OB's head, for example. In other words, sensor 2a measures time-series data of angular velocity data generated in conjunction with the movement of the subject OB's head, for example. Then, sensor 2a continuously transmits the measured angular velocity data to the information processing device 1, for example.

[0015] The sensor 2a may be built into a terminal device (not shown), such as headphones. The terminal device may be attached to the head of the subject OB. Furthermore, in this case, the sensor 2a may communicate with the information processing device 1 via a communication device (not shown) provided by the terminal device. In addition, the sensor 2a may include, for example, an acceleration sensor or an angular velocity sensor, or, together with the acceleration sensor or angular velocity sensor, another sensor different from the acceleration sensor or angular velocity sensor (for example, at least one of a position sensor, a velocity sensor, and an angle sensor).

[0016] Sensor 2b is a group of sensors that includes, for example, at least one of an acceleration sensor and an angular velocity sensor. Sensor 2b is attached to, for example, a part of the subject OB other than the head (hereinafter also referred to as a specific part). The specific part is, for example, the torso of the subject OB. In other words, sensor 2b is a sensor that is attached to the subject OB at a different part from the part to which sensor 2a is attached. Sensor 2b is also a sensor that measures, for example, acceleration data and angular velocity data (hereinafter collectively referred to as second measurement data) at a specific part of the subject OB.

[0017] Specifically, sensor 2b continuously measures acceleration data generated in response to movement in a specific part of the subject OB. In other words, sensor 2b measures time-series data of acceleration data generated in response to movement in a specific part of the subject OB. Sensor 2b then continuously transmits the measured acceleration data to information processing device 1.

[0018] Furthermore, sensor 2b continuously measures angular velocity data generated in conjunction with the movement of a specific part of the subject OB, for example. In other words, sensor 2b measures time-series data of angular velocity data generated in conjunction with the movement of a specific part of the subject OB, for example. Then, sensor 2b continuously transmits the measured angular velocity data to the information processing device 1, for example.

[0019] Furthermore, sensor 2b may be built into a terminal device (not shown), such as a smartphone. The terminal device may be attached to a specific part of the subject OB. In this case, sensor 2b may communicate with the information processing device 1 via a communication device (not shown) provided by the terminal device. Also, sensor 2b may include, for example, an acceleration sensor or an angular velocity sensor, or, together with an acceleration sensor or an angular velocity sensor, another sensor different from the acceleration sensor or angular velocity sensor (for example, at least one of a position sensor, a velocity sensor, and an angle sensor).

[0020] The information processing device 1 performs, for example, the following processes: generating an estimation model (hereinafter simply referred to as the estimation model) for estimating the gaze direction of subject OB (hereinafter also referred to as the first gaze direction), and estimating the first gaze direction of subject OB using the estimation model (hereinafter also referred to as the gaze direction estimation process). The estimation model is, for example, a model that estimates the tilt angle of subject OB's head (hereinafter also referred to as the first tilt angle) from first measurement data indicating the movement state of subject OB's head, and then further estimates the first gaze direction of subject OB from the estimated first tilt angle.

[0021] Specifically, in this embodiment, when the information processing device 1 performs model generation processing, it acquires, for example, first measurement data (hereinafter also referred to as first learning measurement data) from sensor 2a that indicates the operating state of the subject OB's head. The information processing device 1 also acquires, for example, second measurement data (hereinafter also referred to as second learning measurement data) from sensor 2b that indicates the operating state of a specific part of the subject OB. The second learning measurement data is, for example, measurement data measured at a timing where the time difference with the timing at which the first learning measurement data was measured is within a predetermined time (for example, at the same timing as the timing at which the first learning measurement data was measured). The information processing device 1 then calculates, for example, the first tilt angle of the subject OB's head (neck) (hereinafter also referred to as the first learning tilt angle) by using the acquired first learning measurement data and second learning measurement data. Subsequently, the information processing device 1 generates, for example, training data including the acquired first learning measurement data and first learning tilt angle. Subsequently, the information processing device 1 generates an estimation model (hereinafter also referred to as the slope estimation model or first slope estimation model) by, for example, learning from the generated training data. That is, in this case, the information processing device 1 generates a learned model as the slope estimation model. Note that the information processing device 1 may also generate an estimation model other than the learned model (for example, a mathematical formula) as the slope estimation model.

[0022] Furthermore, in this embodiment, when the information processing device 1 performs model generation processing, it acquires, for example, first learning measurement data from sensor 2a that shows the movement state of the subject OB's head when the subject OB's gaze direction is a predetermined gaze direction. The predetermined gaze direction is, for example, a gaze direction specified by the administrator of the information processing system 10 (hereinafter also referred to simply as the administrator) (i.e., a known gaze direction). The information processing device 1 then generates, for example, an estimation model (hereinafter also referred to as a gaze estimation model or first gaze estimation model) that shows the relationship between the acquired first learning measurement data and the predetermined gaze direction. That is, in this case, the information processing device 1 generates, for example, a mathematical formula as the gaze estimation model. Note that the information processing device 1 may also generate an estimation model other than a mathematical formula (for example, a learning model) as the gaze estimation model.

[0023] Furthermore, in this embodiment, when the information processing device 1 performs gaze estimation processing, it acquires, for example, first measurement data (hereinafter also referred to as first estimation measurement data) indicating the movement state of the subject OB's head from sensor 2a. The information processing device 1 then acquires a first tilt angle (hereinafter also referred to as first estimation tilt angle) estimated from the acquired first estimation measurement data by, for example, using a tilt estimation model. In addition, the information processing device 1 acquires a first gaze direction estimated from the acquired first estimation tilt angle by, for example, using a gaze estimation model. Subsequently, the information processing device 1 outputs, for example, information regarding the acquired first gaze direction.

[0024] In other words, the first tilt angle of the subject OB's head can be determined to have a high correlation with, for example, the subject OB's first line of sight direction. Therefore, the information processing device 1 in this embodiment indirectly estimates the subject OB's first line of sight direction from first measurement data indicating the movement state of the subject OB's head by utilizing, for example, the correlation between the subject OB's first tilt angle of the head and the subject OB's first line of sight direction.

[0025] Specifically, in this embodiment, the information processing device 1 generates, for example, a tilt estimation model capable of estimating a first tilt angle of the subject OB's head from first measurement data indicating the movement state of the subject OB's head, and a gaze estimation model capable of estimating the subject OB's first gaze direction from the first tilt angle of the subject OB's head, during the model learning process. Then, in the gaze estimation process, for example, the information processing device 1 estimates the first tilt angle by inputting the first measurement data acquired from sensor 2a into the tilt estimation model, and then further estimates the first gaze direction by inputting the estimated first tilt angle into the gaze estimation model. In other words, in the gaze estimation process, for example, the information processing device 1 first estimates the first tilt angle using the first measurement data, and then further estimates the first gaze direction, instead of directly estimating the first gaze direction from the first measurement data.

[0026] Here, for example, if an estimation model (hereinafter simply referred to as "other estimation models") is generated that directly estimates the subject OB's first gaze direction from first measurement data showing the subject OB's head movement state, the administrator will need to include the subject OB's first gaze direction in each of the multiple training data used to train the other estimation models. Therefore, in this case, the administrator will need to measure tracking data about the subject OB's first gaze direction while, for example, the first measurement data is being measured by sensor 2a.

[0027] In contrast, when generating the gaze estimation model in this embodiment, the information processing device 1 only needs to identify, for example, the correlation between the first tilt angle of the subject OB's head and the subject OB's first gaze direction. Therefore, the information processing device 1 in this embodiment does not need to perform a large number of measurements regarding the subject OB's first gaze direction, and does not need to measure tracking data regarding the subject OB's first gaze direction.

[0028] Therefore, the information processing device 1 in this embodiment makes it possible to easily estimate the first gaze direction of the subject OB. Specifically, the administrator can estimate the first gaze direction of the subject OB without using a measuring device (not shown) capable of measuring tracking data about the subject OB's first gaze direction. As a result, the administrator can reduce the cost required to estimate the subject OB's first gaze direction.

[0029] Furthermore, if there are multiple target individuals (OBs), the information processing device 1 in this embodiment may, for example, generate at least a slope estimation model for each target individual during the model learning process. The information processing device 1 in this embodiment may, for example, estimate the first gaze direction by using a different slope estimation model for each target individual during the gaze estimation process.

[0030] Furthermore, the following description will cover cases where the information processing device 1, sensor 2a, and sensor 2b are different devices, but is not limited to these cases. Specifically, the information processing device 1 and sensor 2a may be, for example, a single device. That is, each of the information processing device 1 and sensor 2a may be built into a terminal device such as headphones. In this case, each of the information processing device 1 and sensor 2a may communicate with each other within the terminal device such as headphones. Also, the information processing device 1 and sensor 2b may be, for example, a single device. That is, each of the information processing device 1 and sensor 2b may be built into a terminal device such as a smartphone. In this case, each of the information processing device 1 and sensor 2b may communicate with each other within the terminal device such as a smartphone.

[0031] [Example of the configuration of the information processing device 1 in the first embodiment] Next, we will describe an example of the configuration of the information processing device 1. Figure 2 is a diagram showing an example of the configuration of the information processing device 1 in the first embodiment.

[0032] As shown in Figure 2, the information processing device 1 includes, for example, a CPU 101 (which is a processor), a memory 102, a communication device 103, a storage medium 104, and an output device 105. Each part is connected to the others via a bus 106.

[0033] The storage medium 104 has, for example, a program storage area (not shown) for storing a program 110 for performing model generation processing and gaze estimation processing (hereinafter collectively referred to as gaze estimation processing, etc.).

[0034] Furthermore, the storage medium 104 has a storage unit 130 (hereinafter also referred to as the information storage area 130) that stores information used when performing, for example, gaze estimation processing. The storage medium 104 may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0035] The CPU 101, for example, executes a program 110 loaded from the storage medium 104 into memory 102 to perform gaze estimation processing and the like.

[0036] The communication device 103, for example, connects to sensors 2a and 2b via a wired or wireless connection and communicates with sensors 2a and 2b.

[0037] The output device 105 is, for example, a display or similar device that outputs (displays) information such as the results of gaze estimation processing.

[0038] The following explanation assumes that the information processing system 10 has one information processing device 1, but the information processing system 10 may have, for example, multiple information processing devices 1. Furthermore, the gaze estimation process, etc., may be performed in a distributed manner across multiple information processing devices 1. Specifically, the model generation process and the gaze estimation process may each be performed on different information processing devices 1.

[0039] [Outline of the gaze estimation process, etc., in the first embodiment] Next, we will describe the outline of the gaze estimation process and the like in the first embodiment. Figures 3 and 4 are diagrams illustrating the outline of the gaze estimation process and the like in the first embodiment.

[0040] As shown in Figure 3, the information processing device 1 implements various functions, including, for example, a data acquisition unit 111, an angle calculation unit 112, a data generation unit 113, a model generation unit 114, a formula calculation unit 115, an angle estimation unit 116, a gaze estimation unit 117, and a gaze output unit 118.

[0041] Furthermore, as shown in Figure 4, the information storage area 130 stores, for example, first measurement data 131, second measurement data 132, first inclination angle 133 (hereinafter also simply referred to as inclination angle 133), first line of sight direction 134 (hereinafter also simply referred to as line of sight direction 134), training data DT, learning model MD1, and mathematical formula MD2.

[0042] The following explanation assumes that the slope estimation model is the learned model MD1, but is not limited to this case. Specifically, the slope estimation model may be other models, such as mathematical formulas. Furthermore, the following explanation assumes that the gaze estimation model is the mathematical formula MD2, but is not limited to this case. Specifically, the gaze estimation model may be other models, such as learned models.

[0043] First, we will explain the functions implemented in the process of generating the learning model MD1 (hereinafter also referred to as the first model generation process) within the model generation process.

[0044] The data acquisition unit 111 acquires, for example, first measurement data 131 from the sensor 2a that indicates the movement state of the subject OB's head. The data acquisition unit 111 then stores the acquired first measurement data 131 in the information storage area 130.

[0045] Furthermore, the data acquisition unit 111 acquires, for example, second measurement data 132 from the sensor 2b that indicates the operating state of a specific part of the subject OB (for example, the torso of the subject OB). The data acquisition unit 111 then stores the acquired second measurement data 132 in the information storage area 130.

[0046] The angle calculation unit 112 calculates the tilt angle 133 of the subject OB's head by using, for example, the first measurement data 131 and the second measurement data 132 acquired by the data acquisition unit 111. The angle calculation unit 112 then stores the calculated tilt angle 133 in the information storage area 130. The tilt angle 133 may be, for example, the angle when the subject OB's head is tilted to the side, or the angle when the subject OB's head is tilted diagonally upward or diagonally downward. The following explanation will focus on the case where the subject OB's head is tilted to the side.

[0047] The data generation unit 113 generates training data DT that includes, for example, first measurement data 131 (first measurement data 131 acquired by the data acquisition unit 111) stored in the information storage area 130 and inclination angle 133 (inclination angle 133 calculated by the angle calculation unit 112) also stored in the information storage area 130. Specifically, for example, the data generation unit 113 generates training data DT for each first measurement data 131 acquired by the data acquisition unit 111, including each first measurement data 131 and the inclination angle 133 calculated using each first measurement data 131. Then, for example, the data generation unit 113 stores the generated training data DT in the information storage area 130.

[0048] The model generation unit 114 generates a learning model MD1 by, for example, learning from the training data DT generated by the data generation unit 113. Then, the model generation unit 114 stores the generated learning model MD1 in the information storage area 130.

[0049] Next, we will explain the functions implemented in the process of calculating the mathematical formula MD2 (hereinafter also referred to as the second model generation process) within the model generation process.

[0050] The data acquisition unit 111 acquires, for example, first measurement data 131 from the sensor 2a that shows the movement state of the subject OB's head when the subject OB's gaze direction is in a predetermined direction.

[0051] The formula calculation unit 115 generates a formula MD2 that shows the relationship between the acquired first measurement data 131 and a predetermined line of sight direction. The formula calculation unit 115 then stores the calculated formula MD2 in the information storage area 130.

[0052] Next, we will explain the functions implemented in the gaze estimation process.

[0053] The data acquisition unit 111 acquires, for example, first measurement data 131 indicating the movement state of the subject OB's head from the sensor 2a, similar to the case in the first model generation process. The data acquisition unit 111 then stores the acquired first measurement data 131 in the information storage area 130.

[0054] The angle estimation unit 116 obtains the inclination angle 133 estimated from the first measurement data 131 acquired by the data acquisition unit 111, for example, by using the learning model MD1 generated in the first model generation process.

[0055] The gaze estimation unit 117 obtains the gaze direction 134 estimated from the inclination angle 133 acquired by the angle estimation unit 116, for example, by using the mathematical formula MD2 calculated in the second model generation process. The gaze estimation unit 117 then stores the acquired gaze direction 134 in the information storage area 130.

[0056] The gaze output unit 118 outputs, for example, the gaze direction 134 acquired by the gaze estimation unit 117. Specifically, the gaze output unit 118 outputs, for example, the gaze direction 134 acquired by the gaze estimation unit 117 to an output device 105 or an operation terminal (not shown) that can be viewed by an administrator.

[0057] [Details of the first model generation process in the first embodiment] Next, we will describe the details of the first model generation process in the first embodiment. Figures 5 to 8 are flowcharts illustrating the details of the first model generation process in the first embodiment. Figures 9 to 11 are diagrams illustrating the details of the first model generation process in the first embodiment.

[0058] As shown in Figure 5, the data acquisition unit 111 waits, for example, until the first acquisition timing occurs (NO in S11). The first acquisition timing may be a periodic timing, such as once every second.

[0059] Then, when the first acquisition timing is reached (YES in S11), the data acquisition unit 111 acquires, for example, first measurement data 131 from sensor 2a (S12). After that, the data acquisition unit 111 stores, for example, the acquired first measurement data 131 in the information storage area 130.

[0060] In other words, as shown in Figure 9, the data acquisition unit 111 stores, for example, the time-series data of the first measurement data 131 continuously acquired from the sensor 2a in the information storage area 130.

[0061] Specifically, for example, if sensor 2a includes an acceleration sensor attached to the head of subject OB, the data acquisition unit 111 stores time-series data of acceleration data indicating the movement state of subject OB's head in the information storage area 130. Also, for example, if sensor 2a includes an angular velocity sensor attached to the head of subject OB, the data acquisition unit 111 stores time-series data of angular velocity data indicating the movement state of subject OB's head in the information storage area 130.

[0062] The data acquisition unit 111 may, for example, acquire the first measurement data 131 that is spontaneously transmitted from the sensor 2a.

[0063] Furthermore, as shown in Figure 6, the data acquisition unit 111 waits, for example, until the second acquisition timing occurs (NO in S21). The second acquisition timing may be a periodic timing, such as once every second. Alternatively, the second acquisition timing may be the same timing as the first acquisition timing.

[0064] Then, when the second acquisition timing is reached (YES in S21), the data acquisition unit 111 acquires, for example, second measurement data 132 from sensor 2b (S22). After that, the data acquisition unit 111 stores, for example, the acquired second measurement data 132 in the information storage area 130.

[0065] In other words, as shown in Figure 9, the data acquisition unit 111 stores the time-series data of the second measurement data 132 continuously acquired from the sensor 2b in the information storage area 130.

[0066] Specifically, for example, if sensor 2b includes an acceleration sensor attached to the torso of subject OB, the data acquisition unit 111 stores time-series data of acceleration data indicating the operating state of subject OB's torso in the information storage area 130. Also, for example, if sensor 2b includes an angular velocity sensor attached to the torso of subject OB, the data acquisition unit 111 stores time-series data of angular velocity data indicating the operating state of subject OB's torso in the information storage area 130.

[0067] The data acquisition unit 111 may, for example, acquire the second measurement data 132 that is spontaneously transmitted from the sensor 2b.

[0068] Furthermore, when the data acquisition unit 111 stores the first measurement data 131 received in S12 and the second measurement data 132 received in S22 in the information storage area 130, it may store the first measurement data 131 and the second measurement data 132, which were measured at the same timing, in the information storage area 130 in a manner that associates them.

[0069] Then, as shown in Figure 7, the angle calculation unit 112 waits until, for example, the timing for calculating the inclination angle 133 (hereinafter also referred to as the calculation timing) (NO in S31). The calculation timing may be, for example, a timing specified by the administrator.

[0070] Then, when it is time to calculate (YES in S31), the angle calculation unit 112 calculates the inclination angle 133 (S32) by using, for example, the first measurement data 131 acquired from sensor 2a in S12 and the second measurement data 132 acquired from sensor 2b in S22, as shown in Figure 10. After that, the angle calculation unit 112 stores the calculated inclination angle 133 in the information storage area 130.

[0071] Specifically, the angle calculation unit 112 calculates the inclination angle 133 by using, for example, the first measurement data 131 and the second measurement data 132, which were measured at the same time.

[0072] More specifically, the angle calculation unit 112 sets the posture angle of the mounting position of sensor 2a (mounting position on the head) as an initial value (e.g., 0°) when the posture of the subject OB is in a predetermined state. The predetermined state is, for example, when the subject OB is standing facing a predetermined direction. The angle calculation unit 112 also sets the posture angle of the mounting position of sensor 2b (mounting position on a specific body part) as an initial value (e.g., 0°) when the posture of the subject OB is in a predetermined state. Then, the angle calculation unit 112 calculates the posture angle corresponding to the first measurement data 131 (hereinafter also referred to as the first posture angle) when the initial value of the mounting position of sensor 2a is used as a reference. The angle calculation unit 112 also calculates the posture angle corresponding to the second measurement data 132 (hereinafter also referred to as the second posture angle) when the initial value of the mounting position of sensor 2b is used as a reference. After that, the angle calculation unit 112 calculates the relative value of the calculated first posture angle and the second posture angle as the inclination angle 133.

[0073] The angle calculation unit 112 may calculate the inclination angle 133 by using, for example, data from the first measurement data 131 and the second measurement data 132 stored in the information storage area 130 at the time of calculation that has not yet been used to calculate the inclination angle 133.

[0074] Furthermore, when the angle calculation unit 112 stores the inclination angles 133 calculated in S32 in the information storage area 130, it may store each inclination angle 133 in the information storage area 130 in a manner that associates each inclination angle 133 with the first measurement data 131 and the second measurement data 132 used to calculate each inclination angle 133.

[0075] Subsequently, the data generation unit 113 waits until the timing for generating the learning model MD1 (hereinafter also referred to as the first generation timing) arrives, as shown in Figure 8 (NO in S41). The first generation timing may be, for example, a timing specified by the administrator.

[0076] Then, when the first generation timing is reached (YES in S41), the data generation unit 113 generates training data DT, which includes, for example, the first measurement data 131 acquired in S11 and the inclination angle 133 calculated in S32, as shown in Figure 11 (S42). After that, the data generation unit 113 stores the generated training data DT in the information storage area 130, for example.

[0077] The data generation unit 113 may generate the training data DT by using, for example, the first measurement data 131 and the inclination angle 133 stored in the information storage area 130 at the time of the first generation timing, data that has not yet been used to generate the training data DT.

[0078] Next, the model generation unit 114 generates a learning model MD1 by learning the training data DT generated in S42, for example, as shown in Figure 11 (S43). After that, the model generation unit 114 stores the generated learning model MD1 in the information storage area 130, for example.

[0079] [Details of the second model generation process in the first embodiment] Next, we will describe the details of the second model generation process in the first embodiment. Figures 12 and 13 are flowcharts illustrating the details of the second model generation process in the first embodiment. Figures 14 and 15 are diagrams illustrating the details of the second model generation process in the first embodiment. Below, we will describe the case where a specific part of the subject OB is the torso of the subject OB.

[0080] As shown in Figure 12, the data acquisition unit 111 waits, for example, until the third acquisition timing occurs (NO in S51). The third acquisition timing may be, for example, a timing specified by the administrator.

[0081] Then, when the third acquisition timing is reached (YES in S51), the data acquisition unit 111 presents, for example, information indicating a predetermined gaze direction SD to the subject OB (S52).

[0082] Specifically, the data acquisition unit 111 transmits information to a terminal device (not shown), such as a smartphone, held by the subject OB, instructing the subject OB to look in a predetermined gaze direction SD.

[0083] The data acquisition unit 111 may, for example, instruct the subject OB to look in a predetermined line of sight direction SD (direction of the marker) by displaying a marker at a predetermined position in the room (on the wall surface of the room). Alternatively, the data acquisition unit 111 may instruct the subject OB to look in a predetermined line of sight direction SD (direction of the marker) by displaying a pseudo-marker at a predetermined position in the AR (Augmented Reality) space.

[0084] Subsequently, the data acquisition unit 111 acquires, for example, first measurement data 131 when the subject OB looks in a predetermined line of sight direction SD (S53). Then, the data acquisition unit 111 stores, for example, the acquired first measurement data 131 in the information storage area 130.

[0085] Specifically, as shown in Figure 14, the data acquisition unit 111 acquires measurement data 131 measured during the time when the orientation of the head OB1 changes from direction D0 to direction D1, for example, when the subject OB looks in a predetermined line of sight direction SD.

[0086] The data acquisition unit 111 may, for example, repeatedly perform steps S52 and S53 for each of a plurality of predetermined line-of-sight directions SD.

[0087] Then, as shown in Figure 13, the formula calculation unit 115 waits until, for example, the timing for generating formula MD2 (hereinafter also referred to as the second generation timing) arrives (NO in S61). The second generation timing may be, for example, a timing specified by the administrator.

[0088] Subsequently, if the second generation timing occurs (YES in S61), the formula calculation unit 115 calculates a formula MD2 that shows the relationship between the predetermined gaze direction SD presented to the subject OB in S52 and the first measurement data 131 acquired in S53, as shown in Figure 15 (S62). Then, the formula calculation unit 115 stores the calculated formula MD2 in the information storage area 130.

[0089] Specifically, the formula calculation unit 115 may, for example, plot points on a two-dimensional plane that show the relationship between each line of sight SD and the corresponding measurement data 131 for each of a plurality of predetermined line of sight SDs. The formula calculation unit 115 may then calculate the formula MD2 by, for example, fitting each point plotted on the two-dimensional plane to a predetermined formula such as an approximation line.

[0090] [Details of the gaze estimation process in the first embodiment] Next, we will describe the details of the gaze estimation process in the first embodiment. Figure 16 is a flowchart illustrating the details of the model generation process in the first embodiment. Figures 17 to 19 are diagrams illustrating the details of the gaze estimation process in the first embodiment.

[0091] As shown in Figure 16, the data acquisition unit 111 waits, for example, until the estimated timing is reached (NO in S71). The estimated timing may be, for example, a timing specified by the administrator.

[0092] Then, when the estimated timing is reached (YES in S71), the data acquisition unit 111 acquires, for example, first measurement data 131 from the sensor 2a that indicates the operating state of the subject OB's head OB1 (S72).

[0093] Specifically, as shown in Figure 17, the data acquisition unit 111 acquires measurement data 131 measured during the period when the orientation of the head OB1 changes from direction D0a to direction D1a, for example, when the direction of the head OB1 changes from direction D0a to direction D1a. That is, direction D0a is, for example, the direction of the head OB1 before performing the action corresponding to the acquired first measurement data 131. Direction D1a (hereinafter also referred to as first direction D1a) is, for example, the direction of the head OB1 after performing the action corresponding to the acquired first measurement data 131.

[0094] Next, as shown in Figure 18, the angle estimation unit 116 obtains the inclination angle 133 estimated from the first measurement data 131 acquired in S72 by using, for example, the learning model MD1 stored in the information storage area 130 (S73).

[0095] Specifically, the angle estimation unit 116 inputs, for example, the first measurement data 131 acquired in S72 to the learning model MD1. Then, the angle estimation unit 116 acquires, for example, the inclination angle 133 output from the learning model MD1 in conjunction with the input of the first measurement data 131. Hereinafter, as shown in Figure 17, the inclination angle 133 acquired in S73 (the inclination angle 133 between direction D0b and direction D1a) is also referred to as angle a. That is, direction D0b (hereinafter also referred to as second direction D0b) is, for example, the direction of the fuselage OB2. Specifically, direction D0b is, for example, the direction of the fuselage OB2 before performing the action corresponding to the first measurement data 131 acquired in S72, the direction of the fuselage OB2 during the action corresponding to the first measurement data 131 acquired in S72, or the direction of the fuselage OB2 after performing the action corresponding to the first measurement data 131 acquired in S72. In other words, direction D0b is the direction of the fuselage OB2 before and after performing the operation corresponding to the first measurement data 131 acquired in S72, for example. Note that direction D0b may be the same as direction D0a, for example.

[0096] Subsequently, as shown in Figure 19, the gaze estimation unit 117 obtains the gaze direction 134 estimated from the tilt angle 133 obtained in S73 by, for example, using the mathematical formula MD2 stored in the information storage area 130 (S74).

[0097] Specifically, the gaze estimation unit 117 obtains the gaze direction 134, calculated by substituting the inclination angle 133 obtained in S73 into the formula MD2. Hereinafter, as shown in Figure 17, the angle between direction D1a and the gaze direction 134 obtained in S74 is also referred to as angle b.

[0098] Subsequently, the gaze output unit 118 outputs information indicating the gaze direction 134 acquired in S74, for example (S75).

[0099] Specifically, the gaze output unit 118 outputs, for example, the gaze direction 134 that was acquired in S74. The gaze output unit 118 also outputs, for example, the angle b corresponding to the gaze direction 134 acquired in S74.

[0100] In this case, the line-of-sight output unit 118 may output, for example, an angle c (see Figure 17) calculated by adding angle a and angle b as information indicating the line-of-sight direction 134 acquired in S74.

[0101] As described above, the information processing device 1 in this embodiment acquires, for example, first measurement data 131 from sensor 2a that indicates the movement state of the subject OB's head as part of gaze estimation processing. The information processing device 1 in this embodiment then acquires the tilt angle 133 estimated from the acquired first measurement data 131 by, for example, using a learning model MD1. Furthermore, the information processing device 1 in this embodiment acquires the gaze direction 134 estimated from the acquired tilt angle 133 by, for example, using the mathematical formula MD2. Subsequently, the information processing device 1 outputs, for example, the acquired gaze direction 134.

[0102] As a result, the information processing device 1 in this embodiment can easily estimate the gaze direction 134 of the subject OB. Specifically, the administrator can estimate the gaze direction 134 of the subject OB without using a measuring device (not shown) capable of measuring tracking data about the subject OB's gaze direction 134. Furthermore, the information processing device 1 in this embodiment can estimate the gaze direction 134 of the subject OB by using, for example, the first measurement data 131 acquired by the sensor 2a, eliminating the need to use a camera or other imaging device (not shown). Therefore, the information processing device 1 in this embodiment can reduce the power consumption required for estimating the gaze direction 134 of the subject OB. Furthermore, the information processing device 1 in this embodiment can reduce the cost required for estimating the gaze direction 134 of the subject OB.

[0103] Furthermore, in this embodiment, the information processing device 1 can estimate the gaze direction 134 of the subject OB without using the second measurement data 132 from the sensor 2b in the gaze estimation process. Therefore, in this embodiment, the information processing device 1 does not need to attach the sensor 2b to the subject OB when performing the gaze estimation process.

[0104] [Eyewitness estimation processing in the first modified example] Next, we will explain the gaze estimation process in the first modified example. Figure 20 is a diagram illustrating the gaze estimation process in the first modified example.

[0105] The angle estimation unit 116 may, for example, determine whether the first measurement data 131 acquired by the data acquisition unit 111 is equal to or greater than a predetermined threshold (hereinafter also referred to as the first threshold) in the line-of-sight estimation process.

[0106] Furthermore, the angle estimation unit 116 may, for example, determine that the first measurement data 131 acquired by the data acquisition unit 111 is equal to or greater than a first threshold, and then use the learning model MD1 to acquire the inclination angle 133 estimated from the acquired first measurement data 131.

[0107] Specifically, as shown in Figure 20, the angle estimation unit 116 may estimate the tilt angle 133 by inputting data from the time series data of the first measurement data 131 acquired by the data acquisition unit 111, corresponding to the time periods (time periods t1 and t2) in which the first measurement data 131 is equal to or greater than the first threshold, to the learning model MD1. In this case, the gaze estimation unit 117 may input the tilt angle 133 output from the learning model MD1 into the formula MD2, for example, as the tilt angle 133 corresponding to the time period in which the first measurement data 131 is equal to or greater than the first threshold.

[0108] On the other hand, as shown in Figure 20, the angle estimation unit 116 may, for example, not input to the learning model MD1 data that corresponds to time periods (time periods other than time periods t1 and t2) in which the first measurement data 131 is less than the first threshold, from the time series data of the first measurement data 131 acquired by the data acquisition unit 111. In this case, the angle estimation unit 116 may, for example, interpolate the inclination angle 133 corresponding to the time period in which the first measurement data 131 is less than the first threshold by using the inclination angle 133 estimated in other time periods (for example, time periods t1 and t2) in which the first measurement data 131 was above the first threshold. Subsequently, the gaze estimation unit 117 may, for example, input the interpolated inclination angle 133 as the inclination angle 133 corresponding to the time period in which the first measurement data 131 is less than the first threshold to the formula MD2.

[0109] In other words, it is possible to determine that the first measurement data 131 corresponding to the time period in which the subject OB's head movement is small may be the first measurement data 131 corresponding to the time period in which estimation is difficult in the learning model MD1. Therefore, in this modified example, the information processing device 1 performs estimation using the learning model MD1 by using only the data from the time series data of the first measurement data 131 acquired by the data acquisition unit 111 that corresponds to the time period in which the first measurement data 131 is above the first threshold.

[0110] As a result, the information processing device 1 in this modified example can, for example, further improve the estimation accuracy of the gaze direction 134 of the subject OB.

[0111] In this modified example, even when generating the learning model MD1 in the first model generation process, the information processing device 1 may use only the first measurement data 131 acquired by the data acquisition unit 111 that is equal to or greater than the first threshold.

[0112] Furthermore, the angle estimation unit 116 may, for example, determine whether the first measurement data 131 acquired by the data acquisition unit 111 is below a predetermined threshold (hereinafter also referred to as the second threshold) in the gaze estimation process. If the angle estimation unit 116 determines that the first measurement data 131 acquired by the data acquisition unit 111 is below the second threshold, it may use the learning model MD1 to acquire the inclination angle 133 estimated from the acquired first measurement data 131.

[0113] [Eyewitness estimation processing in the second modified example] Next, we will explain the gaze estimation process and other aspects in the second modified example.

[0114] The angle calculation unit 112 may, for example, perform a process (hereinafter also called a cut process) in the first model generation process to cut out portions of the first measurement data 131 (time-series data of the first measurement data 131) acquired by the data acquisition unit 111 that correspond to predetermined frequencies (hereinafter also called specific frequencies). The angle calculation unit 112 may then calculate the inclination angle 133 using the first measurement data 131 that has undergone the cut process.

[0115] Specifically, the angle calculation unit 112 may, for example, in the first model generation process, perform a process (hereinafter also called a low-pass filter process) on the first measurement data 131 (time-series data of the first measurement data 131) acquired by the data acquisition unit 111 to cut out the portion corresponding to frequencies above a predetermined threshold (hereinafter also called a third threshold). Then, the angle calculation unit 112 may, for example, calculate the inclination angle 133 by using the first measurement data 131 that has undergone the low-pass filter process.

[0116] In other words, it can be determined that the portion of the time-series data of the first measurement data 131 corresponding to frequencies above the third threshold may contain a large amount of data that does not contribute to the estimation of the slope angle 133. Therefore, in this modified example, the information processing device 1 performs the training of the learning model MD1 by using only the portion of the first measurement data 131 acquired by the data acquisition unit 111 that corresponds to frequencies below the third threshold.

[0117] As a result, the information processing device 1 in this modified example can, for example, suppress the occurrence of overfitting in the learning model MD1, and further improve the estimation accuracy of the gaze direction 134 of the subject OB.

[0118] [Eyewitness estimation processing in the third modified example] Next, we will explain the gaze estimation process and other aspects in the third modified example.

[0119] The data generation unit 113 may, for example, classify each of the first measurement data 131 (first measurement data 131 stored in the information storage area 130) acquired by the data acquisition unit 111 into multiple groups (hereinafter also simply referred to as multiple groups) corresponding to each type of operating state that the subject OB was performing at the measurement timing of each first measurement data 131. Similarly, the data generation unit 113 may, for example, classify each of the second measurement data 132 (second measurement data 132 stored in the information storage area 130) acquired by the data acquisition unit 111 into multiple groups.

[0120] Specifically, the data generation unit 113 may classify each of the first measurement data 131 (first measurement data 131 stored in the information storage area 130) acquired by the data acquisition unit 111 into multiple groups, including a group consisting of first measurement data 131 measured during periods when the subject OB was stationary and a group consisting of first measurement data 131 measured during periods when the subject OB was walking. Similarly, the data generation unit 113 may classify each of the second measurement data 132 (second measurement data 132 stored in the information storage area 130) acquired by the data acquisition unit 111 into multiple groups, including a group consisting of second measurement data 132 measured during periods when the subject OB was stationary and a group consisting of second measurement data 132 measured during periods when the subject OB was walking.

[0121] In this case, the data generation unit 113 may generate training data DT for each of the multiple groups, which includes the first measurement data 131 included in each group and the inclination angle 133 calculated from the first measurement data 131 and the second measurement data 132 included in each group. Furthermore, in this case, the model generation unit 114 may generate a learning model MD1 for each of the multiple groups by using the training data DT corresponding to each group.

[0122] Subsequently, the angle estimation unit 116 may, for example, in the gaze estimation process, when the data acquisition unit 111 acquires the first measurement data 131, identify a learning model MD1 corresponding to the movement state of the subject OB at the measurement timing of the acquired first measurement data 131, and input the first measurement data 131 to the identified learning model MD1.

[0123] As a result, the information processing device 1 in this modified example can, for example, use different learning models MD1 depending on the state of the subject OB, thereby improving the estimation accuracy of the subject OB's gaze direction 134.

[0124] [Eyewitness estimation processing in the fourth modified example] Next, we will explain the gaze estimation process and other aspects in the fourth modified example.

[0125] The data generation unit 113 may, for example, in the first model generation process, generate training data DT that includes, in addition to the first measurement data 131 acquired by the data acquisition unit 111 (the first measurement data 131 stored in the information storage area 130) and the inclination angle 133 calculated by the angle calculation unit 112 (the inclination angle 133 stored in the information storage area 130), also information indicating the type of action performed by the subject OB at the measurement timing of each first measurement data 131 (hereinafter also referred to as state information or learning state information). The model generation unit 114 may, for example, generate a learning model MD1 by using the training data DT that includes state information.

[0126] Subsequently, the angle estimation unit 116 may, for example, in the gaze estimation process, input the acquired first measurement data 131 and state information indicating the movement state of the subject OB at the measurement timing of the acquired first measurement data 131 to the learning model MD1 when the data acquisition unit 111 acquires the first measurement data 131.

[0127] As a result, the information processing device 1 in this modified example can, for example, estimate the gaze direction 134 of the subject OB according to the state of the subject OB, and can further improve the estimation accuracy of the gaze direction 134 of the subject OB. [Explanation of Symbols]

[0128] 1: Information Processing Device 2a: Accelerometer 2b: Accelerometer 10: Eye-line direction estimation system 101:CPU 102: Memory 103: Communication device 104:Storage medium 105: Output device 106: Bus 110: Program 111: Data Acquisition Unit 112: Angle calculation unit 113: Data Generation Unit 114: Model generation unit 115: Formula Calculation Unit 116:Angle estimation part 117: Gaze estimation part 118: Eye-tracking output unit 130: Information storage area 131: First measurement data 132: Second measurement data 133: Inclination angle 134: Gaze direction D0: Direction D0a: Direction D0b: Direction D1a: Direction D1: Direction MD1: Learning Model MD2: Formula OB: Target audience OB1: Head OB2: Torso SD: Designated line of sight direction

Claims

1. First measurement data indicating the operating state of the head is acquired from a first sensor attached to the person's head. By using a first tilt estimation model that estimates the tilt angle of the head from measurement data indicating the operating state of the head, a first tilt angle of the head estimated from the acquired first measurement data is obtained. By using a first gaze estimation model that estimates the gaze direction of the person from the tilt angle of the head, the first gaze direction of the person estimated from the acquired first tilt angle is obtained. Output the acquired information regarding the first line of sight direction. A gaze direction estimation program characterized by having a computer perform the processing.

2. In claim 1, The first sensor includes at least one of a plurality of sensors, including an acceleration sensor and an angular velocity sensor. The first measurement data includes at least one of a plurality of data, which includes acceleration data measured by the acceleration sensor and angular velocity data measured by the angular velocity sensor. A gaze direction estimation program characterized by the following features.

3. In claim 1, The first tilt estimation model is a learning model that outputs a tilt angle for the head in response to the input of measurement data indicating the movement state of the head. A gaze direction estimation program characterized by the following features.

4. In claim 1, The first gaze estimation model is a mathematical formula that calculates the first gaze direction from the tilt of the head. A gaze direction estimation program characterized by the following features.

5. In claim 3, Furthermore, prior to the process of outputting information regarding the first line of sight, first learning measurement data indicating the movement state of the head is acquired from the first sensor. Second learning measurement data indicating the operating state of the specific part is obtained from a second sensor attached to a specific part of the person other than the head. By using the acquired first learning measurement data and the second learning measurement data, the first learning tilt angle for the head is calculated. Training data including the first training measurement data and the first training tilt angle is generated. The learning model is generated by learning from the generated training data. A gaze direction estimation program characterized by having a computer perform the processing.

6. In claim 5, The aforementioned specific part is the torso of the person. A gaze direction estimation program characterized by the following features.

7. In claim 4, Furthermore, prior to the process of outputting information regarding the first gaze direction, first learning measurement data indicating the movement state of the head when the person's gaze direction becomes a predetermined gaze direction is acquired from the first sensor. The formula that shows the relationship between the acquired first learning measurement data and the predetermined line of sight direction is calculated. A gaze direction estimation program characterized by having a computer perform the processing.

8. In claim 5, In the process of generating the aforementioned training data, In the process of acquiring the first learning measurement data, the acquired first learning measurement data is classified into a plurality of groups, including a first group containing the first learning measurement data measured when the person is in a first state, and a second group containing the first learning measurement data measured when the person is in a second state. For each of the aforementioned multiple groups, the training data including the first learning measurement data included in each group is generated. In the process of generating the learning model, the learning model is generated by learning the training data corresponding to each of the multiple groups, In the process of obtaining the first tilt angle, In the process of acquiring the first measurement data, a specific group corresponding to the acquired first measurement data is identified from the plurality of groups. By using the learning model corresponding to the identified specific group, the first inclination angle estimated from the acquired first measurement data is obtained. A gaze direction estimation program characterized by the following features.

9. In claim 5, In the process of generating the aforementioned training data, In the process of acquiring the first learning measurement data, learning state information indicating the state of the person at the measurement timing of the acquired first learning measurement data is identified, The training data including the identified learning state information is generated, In the process of generating the learning model, the learning model is generated by learning the training data which includes the learning state information. In the process of obtaining the first tilt angle, In the process of acquiring the first measurement data, state information indicating the state of the person at the measurement timing of the acquired first measurement data is identified, By using the learning model, the first inclination angle estimated from the acquired first measurement data and the identified state information is obtained. A gaze direction estimation program characterized by the following features.

10. In claim 1, In the process of obtaining the first tilt angle, In the process of acquiring the first measurement data, it is determined whether the acquired first measurement data is equal to or greater than a first threshold. If it is determined that the acquired first measurement data is equal to or greater than the first threshold, the first tilt angle for the head estimated from the acquired first measurement data is obtained by using the first tilt estimation model. A gaze direction estimation program characterized by the following features.

11. In claim 1, In the process of obtaining the first tilt angle, In the process of acquiring the first measurement data, it is determined whether the acquired first measurement data is below the second threshold. If it is determined that the acquired first measurement data is below the second threshold, the first tilt angle for the head estimated from the acquired first measurement data is obtained by using the first tilt estimation model. A gaze direction estimation program characterized by the following features.

12. In claim 1, In the process of obtaining the first tilt angle, In the process of acquiring the first measurement data, a cut process is performed on the acquired first measurement data to cut out the portion corresponding to a specific frequency. By using the first tilt estimation model, the first tilt angle for the head is obtained from the first measurement data after the cutting process. A gaze direction estimation program characterized by the following features.

13. In claim 12, In the process of obtaining the first tilt angle, In the process of acquiring the first measurement data, a low-pass filter is applied to the acquired first measurement data to cut out the portion corresponding to frequencies above the third threshold. By using the first tilt estimation model, a first tilt angle for the head is obtained, which is estimated from the first measurement data that has undergone low-pass filtering. A gaze direction estimation program characterized by the following features.

14. In claim 1, In the process for outputting information regarding the first line of sight, the information output regarding the first line of sight includes an angle between the first direction of the head after the head has performed an action corresponding to the first measurement data and the first line of sight. A gaze direction estimation program characterized by the following features.

15. In claim 5, In the process of outputting information regarding the first line of sight, the information regarding the first line of sight includes information indicating the angle between the second direction of the specific part of the head before and after it performs an action corresponding to the first measurement data, and the first line of sight. A gaze direction estimation program characterized by the following features.

16. A data acquisition unit that acquires first measurement data indicating the operating state of the head from a first sensor attached to the head of a person, An angle estimation unit that obtains a first tilt angle for the head estimated from the acquired first measurement data by using a first tilt estimation model that estimates the tilt angle for the head from measurement data indicating the operating state of the head, A gaze estimation unit that obtains the first gaze direction of the person estimated from the acquired first tilt angle by using a first gaze estimation model that estimates the gaze direction of the person from the tilt angle of the head, It includes a gaze output unit that outputs information about the acquired first gaze direction, An information processing device characterized by the following:

17. First measurement data indicating the operating state of the head is acquired from a first sensor attached to the person's head. By using a first tilt estimation model that estimates the tilt angle of the head from measurement data indicating the operating state of the head, a first tilt angle of the head estimated from the acquired first measurement data is obtained. By using a first gaze estimation model that estimates the gaze direction of the person from the tilt angle of the head, the first gaze direction of the person estimated from the acquired first tilt angle is obtained. Output the acquired information regarding the first line of sight direction. A method for estimating line of sight direction, characterized in that the processing is performed by a computer.

Images