Eye-tracking evaluation device and eye-tracking guidance device, as well as eye-tracking evaluation program and eye-tracking guidance program

Abstract

Compared to conventional technologies, this system can accurately evaluate the user's gaze position and guide the user's gaze to the correct position. [Solution] The system acquires important regions within the image presented to the user, acquires data on the user's gaze position within the image, obtains the degree of agreement between the acquired gaze position within the image and the acquired important regions within the image, and evaluates the user's proficiency based on the acquired degree of agreement.

Description

Technical Field

[0001] The present invention relates to a gaze evaluation device, a gaze guidance device, a gaze evaluation program, and a gaze guidance program.

Background Art

[0002] Patent Document 1 (Japanese Unexamined Patent Application Publication No. 2023-022877) describes an invention in which, during treatment, in order to grasp where to focus to proceed with the treatment, a site to be gazed at within the VR screen at the medical site is displayed with a pointer.

[0003] Patent Document 2 (Japanese Patent No. 6649912) describes an invention in which profile information such as the age and gender of each patient, medical record information, three-dimensional CT imaging information obtained by X-ray CT imaging, pre-designed drilling information for implanting fixtures in implant treatment, and three-dimensional shape data of implants and the like are selectively superimposed and displayed on the three-dimensional visual field video of a simulated patient body for simulated practice.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of the present invention is to accurately evaluate the gaze position of a user and to be able to indicate and guide the gaze position of the user to a correct position as compared with the conventional technology.

Means for Solving the Problems

[0006] The first embodiment is a gaze evaluation device comprising: an important region acquisition unit that acquires important regions within a presented image presented to a user; a gaze position data acquisition unit that acquires data on the user's gaze position within the presented image; a degree of agreement acquisition unit that acquires the degree of agreement between the gaze position within the presented image acquired by the gaze position data acquisition unit and the important regions within the presented image acquired by the important region acquisition unit; and a presentation unit that presents the degree of agreement acquired by the degree of agreement acquisition unit to the user.

[0007] A second embodiment is a gaze evaluation device in which, in the first embodiment, the important region is set according to the target gaze position in the presented image or the subjective survey results.

[0008] A third aspect is a gaze guidance device comprising: an important area acquisition unit that acquires important areas within a presented image presented to a user; a gaze position data acquisition unit that acquires data on the user's gaze position within the presented image; a degree of agreement acquisition unit that acquires the degree of agreement between the gaze position within the presented image acquired by the gaze position data acquisition unit and the important areas within the presented image acquired by the important area acquisition unit; and an instruction unit that displays instructions on the presented image to guide the user's gaze position to the important areas according to the degree of agreement acquired by the degree of agreement acquisition unit.

[0009] A fourth aspect is a gaze guidance device in which, in the third aspect, the important area is set according to the target gaze position in the presented image or the subjective survey results.

[0010] A fifth aspect is a gaze evaluation program that causes a computer to perform the following processes, the gaze evaluation program including: an important region acquisition process that acquires important regions within a presented image presented to a user; a gaze position data acquisition process that acquires data on the user's gaze position within the presented image; a degree of agreement acquisition process that acquires the degree of agreement between the acquired gaze position within the presented image and the acquired important regions within the presented image; and a presentation process that presents the acquired degree of agreement to the user.

[0011] A sixth aspect is a gaze evaluation program in which, in the fifth aspect, the important region is set according to the target gaze position in the presented image or the subjective survey results.

[0012] A seventh aspect is a gaze guidance program that causes a computer to perform the following processes, the program comprising: an important area acquisition step of acquiring important areas within a presentation image presented to a user; a gaze position data acquisition process of acquiring data on the user's gaze position within the presentation image; a degree of agreement acquisition process of acquiring the degree of agreement between the acquired gaze position within the presentation image and the acquired important areas within the presentation image; and an instruction process of displaying instructions on the presentation image to guide the user's gaze position to the important areas according to the degree of agreement acquired by the degree of agreement acquisition unit.

[0013] The eighth aspect is a gaze guidance program in the seventh aspect, in which the important region is set according to the target gaze position in the presented image or the subjective survey results. [Effects of the Invention]

[0014] According to the first to eighth embodiments, compared to the prior art, it is possible to accurately evaluate the user's gaze position and to guide the user's gaze to the correct position. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a block diagram showing the functional configuration of the gaze evaluation device according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing the functional configuration of the eye-tracking device according to the second embodiment. [Figure 3] Figure 3 illustrates an example of a gaze evaluation and guidance system in which the gaze evaluation device and gaze guidance device of the first embodiment are configured as a network. [Figure 4]FIG. 4 is a diagram illustrating the hardware configurations of the learner-side computer terminal, the expert-side computer terminal, the instructor-side computer terminal, the administrator-side computer terminal, and the server in the embodiment, and is a hardware configuration diagram for realizing the functional configurations of the apparatuses in FIGS. 1 and 2. [Figure 5] FIG. 5 is a functional block diagram showing the configuration of the head-mounted display. [Figure 6] FIG. 6 is a diagram exemplifying a presentation image presented to a user such as a learner. [Figure 7A] FIG. 7A is a flowchart showing the gaze evaluation program of the first embodiment. [Figure 7B] FIG. 7B is a flowchart showing the gaze guidance program of the second embodiment. [Figure 8] FIG. 8 is a diagram corresponding to FIG. 6, and is a diagram exemplifying a presentation image as an HMD display area presented to a user (expert) wearing an HMD. [Figure 9] FIG. 9 is a diagram corresponding to FIG. 6, and is a diagram exemplifying a presentation image as an HMD display area presented to a user (learner) wearing an HMD. [Figure 10] FIGS. 10(A) and (B) are diagrams respectively showing the transition of the gaze positions of a learner and an expert in a time series, for example, in a predetermined time period. [Figure 11] FIG. 11 is a diagram showing a state in which a pointer indicating gaze position guidance is displayed on a presentation screen [Figure 12] FIG. 12 is a block diagram showing the functional configuration of the gaze evaluation and guidance device of the embodiment. [Figure 13A] FIG. 13A is a diagram used to explain the embodiment. [Figure 13B] FIG. 13B is a diagram used to explain the embodiment. [Figure 13C] FIG. 13C is a diagram used to explain the embodiment. [Figure 13D] FIG. 13D is a diagram used to explain the embodiment. [Figure 13E]Figure 13E is a diagram used to illustrate an example. [Figure 13F] Figure 13E is a diagram used to illustrate an example. [Figure 14] Figures 14(A) and (B) respectively show the time-series transitions in the gaze positions of learners and proficient learners over a given period of time. [Modes for carrying out the invention]

[0016] Hereinafter, embodiments of the gaze evaluation device, gaze guidance device, gaze evaluation program, and gaze guidance program according to the present invention will be described with reference to the drawings.

[0017] (Device configuration)

[0018] Figure 1 is a block diagram showing the functional configuration of the gaze evaluation device 100 according to the first embodiment.

[0019] The gaze evaluation device 100 of the first embodiment is configured to include an important area acquisition unit 110, a gaze position data acquisition unit 120, a degree of agreement acquisition unit 130, and a presentation unit 140.

[0020] Figure 2 is a block diagram showing the functional configuration of the eye-tracking device 101 according to the second embodiment.

[0021] The eye-tracking device 101 of the second embodiment is configured to include an important area acquisition unit 110, an eye-tracking position data acquisition unit 120, a degree of agreement acquisition unit 130, and an instruction unit 150.

[0022] Figure 3 illustrates an example of a gaze evaluation and guidance system in which the gaze evaluation device 100 and gaze guidance device 101 of the first embodiment are configured as a network.

[0023] The functions of the devices shown in Figures 1 and 2 can be realized by a combination of a learner-side computer terminal 10, a proficient-side computer terminal 20, an instructor-side computer terminal 30, an administrator-side computer terminal 40, a server 50, and a network 60 that connects these learner-side computer terminals 10, proficient-side computer terminals 20, instructor-side computer terminals 30, administrator-side computer terminals 40 and server 50 in a communicative manner, as shown in Figure 3.

[0024] Server 50 stores the gaze evaluation program PB1 and the gaze guidance program PB2, making them accessible from the outside.

[0025] The administrator's computer terminal 40 is located on the system administrator's side. The system administrator has the authority to access the server 50 via the administrator's computer terminal 40 to build, edit, update, etc., the eye-tracking evaluation program PB1 and the eye-tracking guidance program PB2.

[0026] The learner-side computer terminal 10 is located on the learner's side and is used to view learning content. The learner accesses the server 50 via the learner-side computer terminal 10 to run the eye-tracking evaluation program PB1 and the eye-tracking guidance program PB2, receiving an evaluation of their eye movements and instructions for guiding their gaze.

[0027] Furthermore, the device functions shown in Figures 1 and 2 can also be realized on a single computer terminal, that is, for example, the learner's computer terminal 10 alone. By installing the eye-tracking evaluation program PB1 and the eye-tracking guidance program PB2 on a single computer terminal, the functions of the eye-tracking evaluation device 100 and the eye-tracking guidance device 101 can be realized on a single computer terminal, that is, for example, the learner's computer terminal 10 alone.

[0028] Network 60 consists of the internet, intranet, etc. Server 50 consists of server equipment as specific computers or virtual servers built on cloud computing services, etc.

[0029] Figure 4 is a diagram illustrating the hardware configuration of the learner-side computer terminal 10, the proficiency-side computer terminal 20, the instructor-side computer terminal 30, the administrator-side computer terminal 40, and the server 50 in the embodiment, and is a hardware configuration diagram for realizing the functional configuration of the devices in Figures 1 and 2.

[0030] As shown in Figure 4, the learner's computer terminal 10, the proficiency-student's computer terminal 20, the instructor's computer terminal 30, the administrator's computer terminal 40, and the server 50 are each configured to communicate with one another via the system bus 15, with their respective CPUs (Central Processing Unit) 11, ROMs (Read Only Memory) 12, RAMs (Random Access Memory) 13, storage 14, input devices 16, display devices 17, communication interfaces 18, and external storage devices 19 being interconnected.

[0031] The CPU 11 is a central processing unit that executes various programs and controls devices connected to the system bus 15. Specifically, the CPU 11 reads programs from the ROM 12 or storage 14 and executes them using the RAM 13 as a workspace. The CPU 11 controls devices connected to the system bus 15 and performs various calculations according to the programs recorded in the ROM 12 or storage 14.

[0032] ROM 12 or storage 14 holds various programs, including the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by the CPU 11, as well as programs that can be read and executed by a computer to realize this embodiment, and various necessary data.

[0033] RAM13 functions as the main memory and work area of ​​the CPU11, temporarily storing programs or data as a working area. Storage14 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs and data, including the BIOS and OS.

[0034] The input device 16 includes a pointing device such as a mouse, and a reading device such as a keyboard or scanner, and is used for various types of input.

[0035] The display device 17 is, for example, an organic EL display or a liquid crystal display, and displays various types of information. The display device 17 may also function as an input device 16 by employing a touch panel method.

[0036] The communication interface 18 is an interface for communicating with other devices, and standards such as Ethernet®, FDDI, and Wi-Fi® are used. The communication interface 18 connects to the network 40 and controls the transmission and reception of data.

[0037] The external storage device 19 consists of various external storage media that can be attached and detached, such as USB memory sticks, memory cards, HDDs, SSDs, etc.

[0038] (First Embodiment)

[0039] (Image shown)

[0040] In the first embodiment, the images presented to users such as learners are assumed to be VR (virtual reality) images of a medical setting. The VR images of the medical setting are preferably videos, but may also be still images. Audio may be added to the VR images (videos) of the medical setting. The data of the VR images of the medical setting is stored in the database 51 of the server 50 as learning content.

[0041] Figure 5 is a functional block diagram showing the configuration of the head-mounted display (hereinafter referred to as "HMD") 70.

[0042] Figure 6 illustrates the 360-degree spherical image 300 and the presentation image 310 extracted from the 360-degree spherical image 300 and presented to the user, such as a learner.

[0043] VR images used in medical settings are captured as 360-degree spherical images. A 360-degree spherical image is an image captured using an omnidirectional camera, covering 360 degrees in all directions from the user's viewpoint.

[0044] By wearing a head-mounted display (hereinafter referred to as "HMD") 70, users can view 300 360-degree images.

[0045] From the 360-degree image 300, an HMD display area 310, extracted according to the user's facial movements, is displayed as a presented image on the display screen 79 of the HMD 70. As a result, the HMD display area 310 is presented within the field of view of the user wearing the HMD 70.

[0046] The HMD70 and the learner's computer terminal 10 are interconnected via wired or wireless connections to enable data transmission and reception. VR image data of the medical setting is received by the HMD70 via the server 50's database 51, the network 60, and the learner's computer terminal 10, and the HMD display area 310 is presented to the learner wearing the HMD70.

[0047] Similarly, the HMD70 and the trainee's computer terminal 20 are interconnected via wired or wireless connections to enable data transmission and reception. VR image data from the medical setting is received by the HMD70 via the server 50's database 51, the network 60, and the trainee's computer terminal 20, and the HMD display area 310 is presented to the trainee wearing the HMD70.

[0048] Furthermore, the present image is not limited to a full-spherical image, but may be a hemispherical image, a panoramic image, or a planar image. Also, the screen on which the present image is displayed is not limited to the display screen 79 of the HMD 70. The present image may be displayed on the display screen 17 of the learning computer terminal 10 or the proficiency-user computer terminal 20.

[0049] The HMD 70 includes an HMD display area detection unit 71 that detects an HMD display area 310 to be displayed from a 360-degree image 300 according to the orientation of the user's face, a gaze position detection unit 72 that detects the user's gaze position VS, a display screen 79 on which the HMD display area 310 is displayed, and a display control unit 73 that controls the display of the display screen 79.

[0050] The HMD display area detection unit 71 is configured to include, for example, a gyro sensor (a 3-axis accelerometer). Based on the detection signal from the gyro sensor, the orientation of the user's face is measured, and the HMD display area 310 is detected according to the measured orientation of the user's face.

[0051] The HMD display area detection unit 71 detects the HMD display area 310 as a frame image at time t, for example, at a frame rate of 30fps.

[0052] The gaze position detection unit 72 includes a camera that captures images with both of the user's eyes within the field of view. This camera is, for example, a camera built into the HMD 70, and the user's gaze position VS is detected by image processing of the image captured by the built-in camera. The gaze position VS is expressed as VS(t, x, y, z), where time t and orthogonal coordinate positions x, y, and z are variables. The gaze position VS(t, x, y, z) can be displayed on the xyz coordinate axis of the 360-degree image 300 for each time step t.

[0053] The display control unit 73 controls the display of the display screen 79. The display control unit 73 sequentially displays the HMD display area 310 on the display screen 79 of the HMD 70. The display screen 79 is positioned so that it is directly in front of the user's eyes when the HMD 70 is worn.

[0054] Figure 7A is a flowchart of the gaze evaluation program PB1 of the first embodiment. Each process of the gaze evaluation program PB1 corresponds to each step performed by the gaze evaluation device of the first embodiment.

[0055] The gaze evaluation program PB1 is stored in the ROM 12, storage 14, or external storage device 19 of the server 50 or the learner's computer terminal 10, so that it can be processed on the learner's computer terminal 10 and the proficient user's computer terminal 20. These ROM 12, storage 14, or external storage device 19 are examples of computer-readable storage media on which the gaze evaluation program PB1 is recorded. Similarly, the gaze guidance program PB2 shown in Figure 7B is also recorded in a computer-readable storage medium such as the ROM 12, storage 14, or external storage device 19.

[0056] The following describes each process performed by the gaze evaluation device 100 of the first embodiment, following the flowchart shown in Figure 7A.

[0057] (Important area acquisition process S11)

[0058] The important region acquisition unit 110 acquires the important region Gi within the presented image 310 presented to the user (S11).

[0059] The critical region Gi is determined based on data from the target gaze position Vr within the presented image or subjective survey results.

[0060] The trainee's computer terminal 20 is located on the user's (trainee's) side, where the target gaze position Vr is set. The user accesses the database 51 on the server 50 via the trainee's computer terminal 20 and the network 60 to retrieve VR image data of the medical field from the database 51 on the server 50, and imports it into the HMD 70 via the network 60 and the trainee's computer terminal 20.

[0061] The user accesses the server 50 via the trainee computer terminal 20 and network 60 to execute the eye-tracking evaluation program PB1 and set the target eye-tracking position VSR.

[0062] Figure 8 corresponds to Figure 6 and illustrates the presented image 310 as the HMD display area 310 presented to a user (expert) wearing the HMD 70.

[0063] The HMD display area detection unit 71 detects the HMD display area 310 corresponding to the user's face orientation at each time t, and the HMD display area 310 is sequentially displayed as a frame image on the display screen 79 of the HMD 70 at each time t.

[0064] Furthermore, the gaze position detection unit 72 detects the user's gaze position VSR(t, x, y, z) at each time t, and displays it sequentially on the xyz coordinate axes of the 360-degree image 300 at each time t.

[0065] Figure 8 illustrates the transition of the gaze position VSr of a proficient practitioner during a given time period Δtbe (the time period from time tb to time te) using a dashed line. In medical settings, there are many situations where practitioners alternately check multiple locations such as medical procedures and measuring instruments, and the importance of each location in a given situation differs. Locations where the gaze position VS dwells for a longer period, and locations where the gaze position VS changes more frequently, are considered more important.

[0066] Therefore, the expert's line of sight position VSR is set as the target line of sight position, and the areas that overlap with the target line of sight position VSR are set as important regions G1 and G2 in a predetermined time period Δtbe. A process is performed to associate weights W1 and W2, which indicate higher importance, i.e., larger weighting, with important regions G1 and G2 in the order of G1 and G2. Similarly, important regions G1, G2, G3, G4, G5, and G6 are set for the 300-degree spherical image shown in Figure 6, and weights W1, W2, W3, W4, W5, and W6 are associated with these important regions G1, G2, G3, G4, G5, and G6, respectively. The important regions G1, G2, G3, G4, G5, and G6 are represented by Gi(t, x, y, z), where the variables are the number i (i=1, 2, ...), time t, and coordinate positions x, y, and z. The coordinate positions x, y, and z are, for example, the center coordinates of important regions G1, G2, G3, G4, G5, and G6. The important region Gi(t, x, y, z) can be displayed on the xyz coordinate axes of the 300-sphere image at each time point t.

[0067] Data indicating the trainee's gaze position Vr, as well as data indicating important regions G1, G2, G3, G4, G5, G6 and their weights W1, W2, W3, W4, W5, W6, are stored in the database 51 of the server 50 via the trainee's computer terminal 20 and network 60.

[0068] The data indicating the important regions G1, G2, G3, G4, G5, G6 and their weights W1, W2, W3, W4, W5, W6 may be manually set by the user (expert). That is, the important regions G1, G2, G3, G4, G5, G6 and their associated weights W1, W2, W3, W4, W5, W6 are manually set based on the user's (expert's) subjective judgment. For example, the important regions G1, G2, G3, G4, G5, G6 may be set by displaying a 360-degree image 300 on the display screen of the display device 17 of the expert's computer terminal 20 and performing an operation to indicate and select the area to be focused on on the display screen of the display device 17.

[0069] (Eye-gaze position data acquisition process S12)

[0070] The gaze position data acquisition unit 120 acquires data on the gaze position VS within the image 310 presented by the user (learner) (S12).

[0071] The learner's computer terminal 10 is located on the user's (learner's) side. The user accesses the database 51 on the server 50 via the learner's computer terminal 10 and the network 60, retrieves VR image data of the medical setting from the database 51 on the server 50, and imports it into the HMD 70 via the network 60 and the learner's computer terminal 10.

[0072] The user accesses the server 50 via the network 60 from the learner's computer terminal 10 to execute the eye-tracking evaluation program PB1 and obtain the eye-tracking position VS.

[0073] Figure 9 corresponds to Figure 6 and illustrates the presented image 310 as the HMD display area 310 presented to the user (learner) wearing the HMD 70.

[0074] The HMD display area detection unit 71 detects the HMD display area 310 corresponding to the user's face orientation at each time t, and the HMD display area 310 is sequentially displayed as a frame image on the display screen 79 of the HMD 70 at each time t.

[0075] Furthermore, the gaze position detection unit 72 detects the user's gaze position VS(t, x, y, z) at each time t, and displays it sequentially on the xyz coordinate axes of the 360-degree image 300 at each time t.

[0076] Figure 9 shows the transition of the learner's gaze position VS during a predetermined time period Δtbe, indicated by a dashed line.

[0077] Data indicating the user's (learner's) gaze position VS is stored in the database 51 of the server 50 via the learner's computer terminal 10 and the network 60.

[0078] (Compatibility acquisition process S13)

[0079] The degree of agreement acquisition unit 130 acquires the degree of agreement DS between the gaze position VS in the acquired presented image 310 and the important regions G1, G2, G3, G4, G5, and G6 in the presented image 310 (S13).

[0080] The degree of agreement DS can be obtained as a combination of at least one or more of the following: the length of gaze dwell time L within important regions G1, G2, G3, G4, G5, and G6; the number of gaze dwell times N; and the number of gaze transitions M between important regions G1, G2, G3, G4, G5, and G6.

[0081] Figure 10(A) shows, for example, the transition of the learner's gaze position VS in a time series over a predetermined time period Δtbe. The horizontal axis is the time axis, and the time durations shown in hatching in the figure represent the gaze position dwell time L1 in important region G1, and the time durations shown in double hatching represent the gaze position dwell time L2 in important region G2. The gaze position dwell time L1 in important region G1 and the gaze position dwell time L2 in important region G2 shown in Figure 10(A) can be obtained as the degree of agreement DS.

[0082] Figure 10(B) shows the time-series transition of the expert's gaze position VSR during the same predetermined time period Δtbe. Figure 10(B) also shows the time-series ideal gaze orientation during the predetermined time period Δtbe.

[0083] For example, the degree of agreement DS can be determined by the difference between this degree of agreement DS and the degree of agreement DSr obtained from the time-series data showing the transition of the learner's gaze position VS as shown in Figure 10(A), which is obtained from the length of time of gaze position dwell time L, the number of times of gaze position dwell time N, and the number of times of gaze transitions M obtained from the time-series data showing the transition of the expert's gaze position VSr as shown in Figure 10(B).

[0084] Alternatively, the degree of agreement DS may be expressed as a numerical representation of the timing of the user's (learner's) head rotation and eye movement in the important domains G1, G2, G3, G4, G5, and G6, as an indicator of the user's (learner's) response and predictive behavior in the important domains G1, G2, G3, G4, G5, and G6.

[0085] Alternatively, the degree of agreement DS may be obtained as an indicator of the efficiency of the user's (learner's) eye-gaze behavior, based on the total angle of the user's (learner's) head rotation, the total distance of eye movement, and the time and speed of movement between stopping points.

[0086] Furthermore, in order to determine whether the gaze position VS is a gaze position that the user (learner) intentionally focused on or a gaze position that was unconsciously directed, a subjective survey of the user (learner) may be conducted, and based on the subjective survey, processing such as removing gaze positions that were unconsciously directed may be performed.

[0087] Data indicating the degree of agreement DS associated with each learner is stored in the database 51 of the server 50.

[0088] (Presentation process S14)

[0089] The presentation unit 140 presents the acquired degree of agreement DS to the user (learner) (S14).

[0090] The instructor's computer terminal 30 is located on the user (instructor)'s side, where the user instructs the learners. The user accesses the database 51 on the server 50 via the instructor's computer terminal 30 and the network 60, retrieves data indicating the degree of agreement DS associated with each learner from the database 51 on the server 50, and imports it into the instructor's computer terminal 30 via the network 60. The user (instructor) can then confirm the data indicating the degree of agreement DS associated with each learner on the display screen 17 of the instructor's computer terminal 30.

[0091] The matching degree DS data is retrieved from the database 51 on the server 50 and transmitted via the network 60 to the learner's computer terminal 10 of the corresponding user (learner). The user (learner) can check their own matching degree DS on the display screen 17 of the learner's computer terminal 10. For example, the contents shown in Figures 10(A) and (B) are displayed on the display screen 17 of the learner's computer terminal 10.

[0092] (Second Embodiment)

[0093] In the following description of the second embodiment, we will omit explanations of processes that overlap with the first embodiment as appropriate, and only describe the different processes.

[0094] In the second embodiment, the gaze guidance program PB2 shown in Figure 7B is executed, and the important area acquisition process S11, gaze position data acquisition process S12, and degree of agreement acquisition process S13 are executed, similar to the first embodiment. Next, the following instruction process S15 is executed.

[0095] (Instruction processing S15)

[0096] The instruction unit 150 displays instructions on the presentation image 310 to guide the user's (learner's) gaze position VS to the important regions G1, G2, G3, G4, G5, and G6, according to the acquired degree of agreement DS (S15).

[0097] For example, as shown in Figures 10(A) and (B), if the learner's gaze position VS is outside of important regions G1 and G2 at a predetermined time τ within a predetermined time period Δtbe, a pointer PT indicating the ideal target gaze position VSr (important region G1) to be focused on at that time will be displayed on the presented image 310, as shown in Figure 11. Similarly, for the other important regions G3, G4, G5, and G6, a pointer PT indicating the gaze position at a predetermined time will be displayed.

[0098] The data indicating the pointer PT that guides the user's gaze is generated on the server 50, received by the HMD 70 via the network 60 and the corresponding user's (learner's) learner-side computer terminal 10, and the pointer PT indicating the gaze guidance is superimposed on the HMD display area 310.

[0099] As a result, if the user (learner) deviates from the important regions G1, G2, G3, G4, G5, or G6 at a predetermined time, they can correct the gaze position VS to guide it to the important regions G1, G2, G3, G4, G5, or G6, which is the ideal target gaze position VSr at that time.

[0100] (Examples)

[0101] Next, we will describe an example of the learning content for "Medical situations during life-saving using an AED (Automated External Defibrillator)." In the following description, the same reference numerals will be used for components identical to those in the first and second embodiments, and explanations will be omitted as appropriate.

[0102] Figure 12 is a block diagram showing the functional configuration of the eye-line evaluation and line guidance device 102 of the embodiment.

[0103] The gaze evaluation and line guidance device 102 consists of an omnidirectional camera (360-degree camera) 80, an HMD 70, a personal computer unit 90, and a display device (monitor) 17. The personal computer unit 90 implements the functions of a learner-side computer terminal 10, a proficient-side computer terminal 20, an instructor-side computer terminal 30, and a server 50.

[0104] The following explanation will be given with reference to Figures 13A through 13E.

[0105] As shown in Figure 13A, a 360-degree camera (all-around camera) 80 captures a 360-degree spherical image (video) 300 of the medical scene during a life-saving procedure using an AED. The data representing the 360-degree spherical image 300 is imported into the personal computer 90.

[0106] When a user with experience wears the HMD70, the HMD display area 310 of the 360-degree image 300 is displayed as a VR image on the display screen 79, as shown in Figure 13B.

[0107] As shown in Figure 13C, the gaze position detection unit 72 sequentially acquires data indicating the trainee's target gaze position VSr, and as shown in Figure 13D, important regions G1, G2, G3, and G4 are set within the 360-degree image 300. The gaze regions C1, C2, C3, and C4 where the trainee's target gaze position VSr has a long dwell time L and a high number of gaze position dwell times N and gaze transition times M can be set as important regions G1, G2, G3, and G4. Alternatively, important regions G1, G2, G3, and G4 may be set by the trainee's instruction and selection operation.

[0108] Data indicating important areas G1, G2, G3, and G4 is stored in the storage unit 91 of the personal computer main unit 90. Data indicating the trainee's target gaze position Vr is stored in the database 51.

[0109] When a learner puts on the HMD70, the HMD display area 310 of the 360-degree image 300 is displayed as a VR image on the display screen 79, as shown in Figure 13B.

[0110] As shown in Figure 13E, the gaze position detection unit 72 sequentially acquires data indicating the learner's gaze position VS. This data indicating the learner's gaze position VS is taken into the personal computer main unit 90.

[0111] The matching determination module 92 of the personal computer main unit 90 determines whether the learner's gaze position VS matches the important regions G1, G2, G3, and G4.

[0112] The determination result from the matching determination module 92 is sent to the gaze data visualization module 93.

[0113] In the gaze data visualization module 93, the degree of agreement DS is obtained based on the determination of agreement or disagreement between the learner's gaze position VS and important regions G1, G2, G3, and G4.

[0114] Furthermore, the gaze data visualization module 93 acquires data indicating the learner's target gaze position VSR. Based on the determination of agreement or disagreement between the learner's target gaze position VSR and the important regions G1, G2, G3, and G4, the degree of agreement DSr is acquired. As shown in Figures 14(A) and (B), the learner's degree of agreement DS and the learner's degree of agreement DSr are acquired, similar to Figures 10(A) and (B).

[0115] As shown in Figures 14(A) and (B), the learner's degree of agreement DS and the proficiency level DSr are displayed on the display screen of the display device 17 by the results display module 94.

[0116] Therefore, learners can easily identify problems with their gaze direction. As a result, learners can efficiently repeat the learning process by putting the HMD70 back on, as shown in Figure 13(B).

[0117] Furthermore, the instruction display module 95 compares the learner's gaze position VS with the expert's gaze position VSr (important regions G1, G2). If the learner's gaze position VS is outside of important regions G1, G2, a pointer PT is superimposed on the HMD display area 310, as shown in Figure 13F, to guide the learner's gaze to the ideal target gaze position VSr (important region G1) that they should be focusing on at that time.

[0118] Therefore, during learning, if the gaze position VS deviates from the important regions G1, G2, G3, and G4 at a predetermined timing, learners can correct the gaze position VS to guide it to the important regions G1, G2, G3, and G4, which are the ideal target gaze position VSr at that timing.

[0119] The first and second embodiments and examples are not limited to the medical field, but can be applied to video content in other fields, such as nursing care settings, fire drills, evacuation drills, disaster rescue training, and hazard prediction driving for automobiles. [Explanation of Symbols]

[0120] 100 Eye-tracking evaluation device 110 Important area acquisition department 120 Eye-line position data acquisition unit 130 Matching degree acquisition unit 140 Presentation section 150 Presentation section

Claims

1. A key region acquisition unit that acquires important regions within the image presented to the user, A gaze position data acquisition unit that acquires data on the gaze position of the user within the presented image, A degree of agreement acquisition unit acquires the degree of agreement between the gaze position in the presented image acquired by the gaze position data acquisition unit and the important region in the presented image acquired by the important region acquisition unit, A presentation unit that presents the degree of agreement obtained by the degree of agreement acquisition unit to the user. A gaze evaluation device equipped with the following features.

2. The aforementioned important region is set according to the target gaze position in the presented image or the subjective survey results. The gaze evaluation device according to claim 1.

3. A key region acquisition unit that acquires important regions within the image presented to the user, A gaze position data acquisition unit that acquires data on the gaze position of the user within the presented image, A degree of agreement acquisition unit acquires the degree of agreement between the gaze position in the presented image acquired by the gaze position data acquisition unit and the important region in the presented image acquired by the important region acquisition unit, An instruction unit displays instructions on the presented image to guide the user's gaze to the important area, according to the degree of match obtained by the degree of match acquisition unit. A gaze guidance device equipped with the following features.

4. The aforementioned important region is set according to the target gaze position in the presented image or the subjective survey results. The gaze guidance device according to claim 1.

5. A gaze evaluation program that causes a computer to perform the following processes: The process involves obtaining important regions within the image presented to the user, and A gaze position data acquisition process that acquires data on the gaze position of the user within the presented image, A matching degree acquisition process that acquires the degree of matching between the gaze position in the acquired presented image and the important region in the acquired presented image, A presentation process that shows the acquired degree of match to the user. A gaze-tracking evaluation program that includes this feature.

6. The aforementioned important region is set according to the target gaze position in the presented image or the subjective survey results. The gaze evaluation program according to claim 5.

7. A gaze guidance program that causes a computer to perform the following processes: A key region acquisition step that acquires key regions within the image presented to the user, A gaze position data acquisition process that acquires data on the gaze position of the user within the presented image, A matching degree acquisition process that acquires the degree of matching between the gaze position in the acquired presented image and the important region in the acquired presented image, Instruction processing to display instructions on the presented image to guide the user's gaze to the important area, according to the degree of match obtained by the degree of match acquisition unit. A program that includes eye-tracking guidance.

8. The aforementioned important region is set according to the target gaze position in the presented image or the subjective survey results. The eye-tracking program according to claim 7.

Images