Information processing device and program

Abstract

This invention provides an information processing device that can generate data that facilitates the analysis of eye-tracking information. [Solution] The information processing device comprises a transmission unit, an acquisition unit, a calculation unit, and a generation unit. The transmission unit transmits video data to a terminal worn by the user. The acquisition unit acquires first gaze data relating to the user's right eye gaze and second gaze data relating to the user's left eye gaze from the terminal. The calculation unit calculates third gaze data relating to the user's gaze with both eyes based on the first and second gaze data. The generation unit generates recorded data based on the third gaze data calculated by the calculation unit and information regarding the video data.

Description

Technical Field

[0006] , ,

[0005] ,

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] Conventionally, as a technology for realizing virtual reality (VR: Virtual Reality), a head-mounted terminal worn by a user is known. Further, a technology for acquiring the gaze information of a user wearing the head-mounted terminal from the head-mounted terminal is known.

[0003] Regarding this, Patent Document 1 discloses a configuration for detecting the gaze related to the left image of the left eye of a user and the gaze related to the right image of the right eye of the user in a terminal having a left-eye display unit and a right-eye display unit that are arranged side by side in the left-right direction and can display images.

Prior Art Documents

Patent Documents

[0007] To solve the above problems, the information processing apparatus of the present invention comprises: a transmission unit that transmits video data to a terminal worn by a user; an acquisition unit that acquires first gaze data relating to the gaze of the user's right eye and second gaze data relating to the gaze of the user's left eye from the terminal; a calculation unit that calculates third gaze data relating to the gaze of both of the user's eyes based on the first gaze data and the second gaze data; and a generation unit that generates recorded data based on the third gaze data calculated by the calculation unit and information relating to the video data.

[0008] Furthermore, the acquisition unit acquires posture data relating to the user's body posture from the terminal, and the generation unit generates the recorded data based on the third gaze data, the video data information, and the posture data.

[0009] Furthermore, the aforementioned video data is a video captured by a virtual camera located within the virtual space, whose angle and position change in conjunction with the user's body movements.

[0010] Furthermore, the recorded data is a video re-imaged by the virtual camera based on the third gaze data, the video data information, and the posture data.

[0011] Furthermore, the recorded data is a video re-recorded by the virtual camera so that the angle and position change in conjunction with the user's body movements.

[0012] Furthermore, the recorded data is a video of a predetermined area within the virtual space captured by the virtual camera.

[0013] Furthermore, the program of the present invention causes the computer to function as a transmission unit that transmits video data to a terminal worn by the user, an acquisition unit that acquires first gaze data relating to the user's right eye gaze and second gaze data relating to the user's left eye gaze from the terminal, a calculation unit that calculates third gaze data relating to the gaze of both of the user's eyes based on the first and second gaze data, and a generation unit that generates recorded data based on the third gaze data calculated by the calculation unit and the video data. [Effects of the Invention]

[0014] According to the present invention, the information processing device and program can generate data that facilitates the analysis of gaze information. [Brief explanation of the drawing]

[0015] [Figure 1] This diagram schematically shows an example of the overall configuration of the eye-tracking analysis system according to the present invention. [Figure 2] This figure shows an example of a specific configuration of the terminal shown in Figure 1. [Figure 3] Figure 1 shows an example of the functional configuration of the information processing device. [Figure 4] Figure 3 shows an example of virtual space data. [Figure 5A] This figure shows an example of the video and first gaze data displayed on the right-hand display of the terminal shown in Figure 1. [Figure 5B] This figure shows an example of the video and second eye-tracking data displayed on the left-hand display of the terminal shown in Figure 1. [Figure 5C] This figure shows an example of third-line gaze data generated by the information processing device shown in Figure 1. [Figure 6A] This figure shows an example of the movement of a virtual camera when the information processing device shown in Figure 1 generates recorded data. [Figure 6B] This figure shows another example of the movement of the virtual camera when the information processing device shown in Figure 1 generates recorded data. [Figure 7]It is a flowchart showing an example of the processing flow performed by the information processing apparatus shown in FIG. 1.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present invention (hereinafter referred to as "the present embodiments") will be described with reference to the accompanying drawings. For ease of understanding of the description, the same reference numerals are given to the same components and steps in each drawing as much as possible, and duplicate descriptions are omitted.

[0017] <Overall Configuration> FIG. 1 is a diagram schematically showing an example of the overall configuration of the gaze analysis system 1 according to the present invention. As shown in FIG. 1, the gaze analysis system 1 mainly includes, for example, an information processing apparatus 10, a terminal 20, and a terminal 30.

[0018] The information processing apparatus 10 is, for example, an information processing apparatus such as a desktop personal computer, a notebook personal computer, a smartphone, or a tablet. The information processing apparatus 10 transmits video data to the terminal 20 of the user 2. Further, the information processing apparatus 10 acquires data regarding the right eye gaze of the user 2 and data regarding the left eye gaze from the terminal 20. Further, the information processing apparatus 10 acquires data regarding the posture of the user 2 from the terminal 20 and the terminal 30. Further, the information processing apparatus 10 calculates data regarding both eyes of the user 2 from the acquired respective information and stores it in association with the video data. The information processing apparatus 10 is mainly configured to include, for example, a CPU (Central Processing Unit) 11, a memory 12, a storage device 13, an input / output device 14, and a communication device 15.

[0019] The CPU 11 functions as various functional configurations by executing a predetermined program stored in the memory 12 or the storage device 13 or the like. Details of this functional configuration will be described later. The storage device 13 is configured by a hard disk or the like. This storage device 13 stores various programs, various information, and information on processing results necessary for the execution of processing in the CPU 11.

[0020] The input / output device 14 is a device that includes, for example, an input device such as a keyboard or mouse, and an output device such as a display or speakers. The input / output device 14 receives input from the operator of the information processing device 10 for performing operations on the information processing device 10, and displays information provided by the information processing device 10 to the operator. The communication device 15 consists of a communication interface for communicating with external devices. The communication device 15 sends and receives various types of information with the user 2's terminals 20 and 30.

[0021] The information processing device 10 can be implemented using a dedicated or general-purpose server computer or other information processing device. Furthermore, the information processing device 10 may consist of a single information processing device or multiple information processing devices distributed across a communication network. Also, Figure 1 only shows a portion of the main hardware configuration of the information processing device 10, and the information processing device 10 may have other configurations.

[0022] The user 2's terminal 20 is, for example, a VR goggle, which is attached to the user 2's head at a mounting location b1 by a belt or the like. The terminal 20 receives video data from the information processing device 10 and generates video data for the right eye and video data for the left eye from the received video data. The terminal 20 displays the video data for the right eye on the right-side display unit 21, which is positioned over the user 2's right eye, and displays the video data for the left eye on the left-side display unit 22, which is positioned over the user 2's left eye. The terminal 20 also acquires first gaze data related to the gaze of the user 2's right eye using the right-side imaging unit 23, and second gaze data related to the gaze of the user 2's left eye using the left-side imaging unit 24. The terminal 20 also acquires posture data, including the position and angle of the mounting location b1 on the user 2's head, using the posture sensor 25. The terminal 20 transmits the acquired first gaze data, second gaze data, and posture data to the information processing device 10.

[0023] Here, with reference to Figure 2, the specific configuration of terminal 20 will be described. Figure 2 is a diagram showing an example of the configuration of terminal 20 shown in Figure 1. As shown in Figure 2, terminal 20 is composed of, for example, a receiving unit 210, a video splitting unit 220, a right-side display unit 21, a left-side display unit 22, a right-side imaging unit 23, a left-side imaging unit 24, a posture sensor 25, and a transmitting unit 230.

[0024] The receiving unit 210 receives video data transmitted from the information processing device 10. The receiving unit 210 transmits the received video data to the video splitting unit 220.

[0025] The video splitting unit 220 splits the video data received by the receiving unit 210 into video data for display on the right-side display unit 21 and video data for display on the right-side display unit 21. For example, the video splitting unit 220 provides video data for display on the right-side display unit 21 obtained by trimming the video data transmitted from the receiving unit 210 by a predetermined percentage and then shifting it to the left by a predetermined number of pixels. Alternatively, the video splitting unit 220 provides video data for display on the left-side display unit 22 obtained by trimming the video data transmitted from the receiving unit 210 by a predetermined percentage and then shifting it to the right by a predetermined number of pixels. The video splitting unit 220 transmits the video data for display on the right-side display unit 21 to the right-side display unit 21 and transmits the video data for display on the left-side display unit 22 to the left-side display unit 22.

[0026] The right-side display unit 21 and the left-side display unit 22 are displays such as liquid crystal panels or organic electro-luminescence (EL) panels. The right-side display unit 21 is provided on the terminal 20 to display an image toward the user 2's right eye and displays the image data transmitted from the image splitting unit 220 for display on the right-side display unit 21. The left-side display unit 22 is provided on the terminal 20 to display an image toward the user 2's left eye and displays the image data transmitted from the image splitting unit 220 for display on the left-side display unit 22.

[0027] The right-side imaging unit 23 and the left-side imaging unit 24 are, for example, cameras that capture the movement of the user 2's eyeballs. The right-side imaging unit 23 is located near or inside the right-side display unit 21 and captures the movement of the user 2's right eyeball. The left-side imaging unit 24 is located near or inside the left-side display unit 22 and captures the movement of the user 2's left eyeball. The right-side imaging unit 23 and the left-side imaging unit 24 transmit the captured images or videos to the transmission unit 230.

[0028] The posture sensor 25 is installed in a suitable location on the terminal 20 and detects the position and angle of the mounting location b1 on the user 2's head. The posture sensor 25 transmits the detected information to the transmission unit 230. Alternatively, the posture sensor 25 may be an imaging device such as a camera that captures images in front of the terminal 20. The posture sensor 25 transmits an image or video captured in front of the terminal 20 to the transmission unit 230.

[0029] The transmitting unit 230 transmits images or videos captured by the right-side imaging unit 23 and the left-side imaging unit 24 to the information processing device 10. The transmitting unit 230 also transmits to the information processing device 10 the position and angle of the mounting location b1 on the user 2's head, as detected by the posture sensor 25.

[0030] Returning to Figure 1, terminal 30 is, for example, a controller for a VR device that accepts operations for user 2 to view or play VR content. Terminal 30 is connected to the information processing device 10 in a communicative manner by a wired or wireless connection. Terminal 30 is equipped with attitude sensors such as an accelerometer and a gyroscope to detect its position and angle, and transmits the detected information to the information processing device 10 as attitude data. Note that terminal 30 is not limited to a controller; for example, it may be a simple attitude sensor configured as a wristband and attached to a mounting location b2 such as the user 2's wrist or ankle.

[0031] <Functional configuration> Next, the functional configuration of the information processing device 10 will be described. Figure 3 is a diagram showing an example of the functional configuration of the information processing device 10 shown in Figure 1. As shown in Figure 3, the information processing device 10 includes, for example, a storage unit 100, a playback unit 110, a transmission unit 120, an acquisition unit 130, a calculation unit 140, and a generation unit 150 as its functional configuration. The functional configurations other than the storage unit 100 are realized by the CPU 11 executing programs stored in the storage device 13, etc.

[0032] The memory unit 100 has a functional configuration that stores virtual space data 101, video data 102, first gaze data 103, second gaze data 104, third gaze data 105, posture data 106, and recording data 107. The memory unit 100 also stores programs necessary for controlling the operation of the information processing device 10 and the terminal 20.

[0033] Virtual space data 101 is data relating to a virtual space in which a 3D model and a virtual camera are provided, such as VR content, AR (Augmented Reality) content, and MR (Mixed Reality) content. Here, virtual space data 101 will be explained with reference to Figure 4. Figure 4 is a diagram showing an example of virtual space data 101 shown in Figure 3. As shown in Figure 4, virtual space data 101 is composed of a 3D model OB1 and a virtual camera CAM provided within a virtual space AREA. 3D model OB1 is a building object formed in the shape of a rectangular prism. 3D model OB1 has a 3D model OB10 representing a window on one of its sides, and a 3D model OB12 representing a door on the side adjacent to the side where 3D model OB10 is provided. Note that while virtual space data 101 includes 3D models OB1, OB10, and OB12, it is not limited to these, and is merely an example; it may include many more 3D models or different 3D models.

[0034] The virtual camera CAM is a virtual camera that captures images within the virtual space AREA, capturing images within the field of view a visible from the virtual camera CAM according to its angle and position (position and height). The virtual camera CAM changes its imaging angle and position (position and height) within the virtual space AREA according to the posture data 106 transmitted from the user 2's posture sensor 25 and terminal 30. In addition to the posture data 106, the virtual camera CAM may also change its angle and position based on operations performed by the user 2 on the terminal 20 (such as pressing buttons not shown, operating levers or pads not shown, or voice input to an audio input device not shown). The virtual camera CAM extracts the captured image as video data 102.

[0035] Returning to Figure 3, the video data 102 is, for example, video data. The video data 102 is generated by being captured by the virtual camera CAM in the virtual space data 101. Note that the video data 102 may also be image data.

[0036] The first gaze data 103 is data relating to the gaze of user 2's right eye. The first gaze data 103 indicates where user 2's right eye is looking on the video data displayed on the right-side display unit 21. The first gaze data 103 will be explained with reference to Figure 5A. Figure 5A is a diagram showing an example of the video data 112 and the first gaze data 103 displayed on the right-side display unit 21 of the terminal 20 shown in Figure 1. In Figure 5A, point P1 indicates the position of user 2's right eye's gaze on the video data 112. The first gaze data 103 is data in which the coordinate values ​​(X coordinate indicating the horizontal position and Y coordinate indicating the vertical position) of point P1 on the video data 112 are associated with the playback time of the video data 112.

[0037] Returning to Figure 3, the second gaze data 104 is data relating to the gaze of user 2's left eye. The second gaze data 104 indicates where user 2's left eye is looking on the video data displayed on the left display unit 22. The second gaze data 104 will be explained with reference to Figure 5B. Figure 5B is a diagram showing an example of the video data 113 and the second gaze data 104 displayed on the left display unit 22 of the terminal 20 shown in Figure 1. In Figure 5B, point P2 indicates the position of user 2's left eye's gaze on the video data 113. The second gaze data 104 is, for example, data in which the coordinate values ​​of point P2 on the video data 113 are associated with the playback time of the video data 113.

[0038] Returning to Figure 3, the third gaze data 105 is data relating to the gaze of both of user 2's eyes. The third gaze data 105 indicates where user 2's eyes are looking on the video data 102. The third gaze data 105 will be explained with reference to Figure 5C. Figure 5C is a diagram showing an example of the third gaze data 105 generated by the information processing device 10 shown in Figure 1. In Figure 5C, point P3 indicates the position of the gaze of both of user 2's eyes on the video data 102. The third gaze data 105 is, for example, data in which the coordinate values ​​of point P3 on the video data 102 are associated with the playback time of the video data 102.

[0039] Returning to Figure 3, the posture data 106 is data relating to the posture of user 2's body and is transmitted from the posture sensor 25 and the terminal 30. The posture data 106 includes the position and angle of attachment points b1 and b2 on user 2. Note that the posture data 106 is transmitted from the posture sensor 25 and the terminal 30 with the position and angle of attachment points b1 and b2 included, but is not limited to this. The posture data 106 may also be generated when measurement values ​​from the gyro sensor and accelerometer mounted on the posture sensor 25 and the terminal 30 are transmitted to the information processing device 10, and predetermined calculation processing is performed on the measurement values. Furthermore, if the posture sensor 25 is an imaging device, the posture data 106 may be generated to include the position, angle, and movement of user 2's hands and fingers by performing hand tracking processing on the image or video captured by the posture sensor 25.

[0040] Recorded data 107 is a video re-imaged by the virtual camera CAM based on the third gaze data 105, video data 102, and posture data 106. The re-imaged recording data 107 will be explained later, so the explanation is omitted here.

[0041] The playback unit 110 plays back the virtual space data 101 stored in the storage unit 100 and uses the virtual camera CAM to capture images within the virtual space AREA. The playback unit 110 also changes the angle and position of the virtual camera CAM in the virtual space data 101 according to the data on the user 2's posture acquired by the acquisition unit 130 from the posture sensor 25 and the terminal 30. The playback unit 110 also stores the captured image as video data 102 in the storage unit 100.

[0042] The transmitting unit 120 transmits the video data 102 stored in the storage unit 100 to the terminal 20. The video data 102 transmitted to the terminal 20 is split into video data 112 for the right display unit 21 and video data 113 for the left display unit 22. The video data 102 transmitted to the terminal 20 is displayed on the right display unit 21 as video data 112 for the right display unit 21, and on the left display unit 22 as video data 113 for the left display unit 22.

[0043] The acquisition unit 130 acquires first gaze data 103 and second gaze data 104 from terminal 20. The acquisition unit 130 also acquires posture data 106 related to user 2's posture from the posture sensor 25 of terminal 20 and terminal 30. The acquisition unit 130 stores the acquired first gaze data 103, second gaze data 104, and posture data 106 in the storage unit 100.

[0044] The calculation unit 140 calculates third gaze data 105 relating to the gaze of both eyes of user 2 based on the first gaze data 103 and the second gaze data 104. Specifically, for example, the calculation unit 140 calculates an intermediate coordinate between the coordinate indicated by the first gaze data 103 and the coordinate indicated by the second gaze data 104 for each playback time, and uses the calculated coordinate as the coordinate value of the third gaze data 105 for that playback time. Alternatively, the calculation unit 140 may simply associate the coordinate indicated by the first gaze data 103 and the coordinate indicated by the second gaze data 104 for each playback time, and use the data where the coordinate of the right eye's gaze, the coordinate of the left eye's gaze, and the playback time are associated as the third gaze data 105. Alternatively, the calculation unit 140 may alternately select the coordinate indicated by the first gaze data 103 and the coordinate indicated by the second gaze data 104 for each playback time. Furthermore, the calculation unit 140 may calculate the coordinates of the third gaze data 105 by approximating the change in coordinates as playback time progresses with a polynomial using an approximation method such as the least squares method. The calculation unit 140 stores the calculated third gaze data 105 in the storage unit 100.

[0045] The generation unit 150 generates recording data 107 based on the third gaze data 105 and video data 102 calculated by the calculation unit 140. An example of the generation of recording data 107 by the generation unit 150 will be described with reference to Figure 6A. Figure 6A is a diagram showing an example of the movement of the virtual camera CAM when the information processing device 10 shown in Figure 1 generates recording data 107. In this example, the generation unit 150 generates recording data 107 based on the third gaze data 105, video data 102 and posture data 106. Specifically, the generation unit 150 plays back the virtual space data 101 and re-images the virtual space AREA with the virtual camera CAM so that the angle and position change in conjunction with the movement of the user 2's body indicated by the posture data 106 acquired by the acquisition unit 130. As shown in Figure 6A, the virtual camera CAM images an area W1 centered on point C1 when the user 2 directs their gaze along axis A1 which is slightly upward relative to the ground. Furthermore, when user 2 directs their gaze along axis A2, which is further upward than axis A1, the virtual camera CAM captures an area W2 centered on point C2. Although not shown in the diagram, the virtual camera CAM itself moves up and down in accordance with the height of user 2's gaze. In addition, the virtual camera CAM itself moves and changes its position and direction in accordance with the movement and direction changes of user 2's gaze in the forward, backward, left, and right directions. In this example, areas W1 and W2 are both rectangles, but are not limited to this and may be circles or other shapes. The generation unit 150 generates recording data 107 by adding visually recognizable effects (such as circles or trajectories) to the playback time and coordinates indicated by the third gaze data 105 on the re-captured image.

[0046] Referring to Figure 6B, another example of the generation of recording data 107 by the generation unit 150 will be described. Figure 6B is a diagram showing another example of the movement of the virtual camera CAM when the information processing device 10 shown in Figure 1 generates recording data 107. In this example, the generation unit 150 reproduces the virtual space data 101 and re-images a predetermined area W3 within the virtual space AREA with the virtual camera CAM. Area W3 is, for example, a rectangular area centered on point C3, but is not limited to this, and may be circular or other shapes. As shown in Figure 6B, the virtual camera CAM moves up and down to match the height of the user 2's line of sight. In addition, the direction of the virtual camera CAM is set so that it always faces point C3, regardless of its position within the virtual space AREA. Specifically, when the virtual camera CAM is at the same height as point C3, it faces point C3 along axis A3 which is parallel to the ground within the virtual space AREA and passes through point C3. Furthermore, if the virtual camera CAM is located at a lower height than point C3, it moves towards point C3 along axis A4, which is slightly upward from the virtual camera CAM. In addition, the virtual camera CAM itself moves and changes its position and orientation in accordance with the movement and direction changes of user 2's gaze in the forward, backward, left, and right directions, so that point C3 is within the imaging range. The generation unit 150 generates recording data 107 by adding visually recognizable effects (such as circles or trajectories) to the playback time and coordinates indicated by the third gaze data 105 in the re-captured video.

[0047] The functional configuration of the information processing device 10 has been described above. Next, the sequence of processing steps of the information processing device 10 will be described in detail. Figure 7 is a flowchart showing an example of the processing steps performed by the information processing device 10 shown in Figure 1.

[0048] (Step SP10) The information processing device 10 uses the playback unit 110 to play back the virtual space data 101 stored in the storage unit 100 and starts capturing video data 102. The information processing device 10 uses the transmission unit 120 to transmit the captured video data 102 to the terminal 20. Then, the process moves on to step SP12.

[0049] (Step SP12) The information processing device 10 determines, based on the playback unit 110, whether or not user 2 has finished viewing the video data 102. Specifically, the information processing device 10 determines whether or not user 2 has finished viewing the video data 102 when the playback unit 110 finishes playing the virtual space data 101 and the transmission of the video data 102 to the terminal 20 is completed. If the determination is negative, the process repeats the process of step SP12. On the other hand, if the determination is positive, the process proceeds to the process of step SP14.

[0050] (Step SP14) The information processing device 10 acquires the first gaze data 103 from the terminal 20 using the acquisition unit 130. The information processing device 10 stores the acquired first gaze data 103 in the storage unit 100. Then, the process proceeds to step SP16.

[0051] (Step SP16) The information processing device 10 acquires the second gaze data 104 from the terminal 20 using the acquisition unit 130. The information processing device 10 stores the acquired second gaze data 104 in the storage unit 100. Then, the process proceeds to step SP18.

[0052] (Step SP18) The information processing device 10 acquires posture data 106 from the posture sensor 25 of terminal 20 and terminal 30 using the acquisition unit 130. Specifically, the information processing device 10 acquires measured values ​​from the posture sensor 25 of terminal 20 and terminal 30 using the acquisition unit 130. The information processing device 10 stores the acquired measured values ​​as posture data 106 in the storage unit 100. Then, the process moves on to the process of step SP20.

[0053] (Step SP20) The information processing device 10 calculates the third gaze data 105 from the first gaze data 103 and the second gaze data 104 using the calculation unit 140. The method for calculating the third gaze data 105 is as described above, so its explanation is omitted. The information processing device 10 stores the calculated third gaze data 105 in the storage unit 100. Then, the process proceeds to step SP22.

[0054] (Step SP22) The information processing device 10 calculates recorded data 107 from the third gaze data 105 and video data 102 using the generation unit 150. The method for calculating the recorded data 107 is as described above, so its explanation is omitted. Then, the process moves on to step SP24.

[0055] (Step SP24) The information processing device 10 stores the recorded data 107 in the storage unit 100 using the generation unit 150. Then, the series of processes shown in Figure 7 is completed.

[0056] <Effects> In this embodiment, the information processing device 10 comprises a transmission unit 120, an acquisition unit 130, a calculation unit 140, and a generation unit 150. The transmission unit 120 transmits video data 102 to a terminal 20 worn by user 2. The acquisition unit 130 acquires first gaze data 103 relating to the gaze of user 2's right eye and second gaze data 104 relating to the gaze of user 2's left eye from the terminal 20. The calculation unit 140 calculates third gaze data 105 relating to the gaze of both eyes of user 2 based on the first gaze data 103 and the second gaze data 104. The generation unit 150 generates recorded data 107 based on the third gaze data 105 calculated by the calculation unit 140 and information regarding the video data 102. Therefore, since the information processing device 10 calculates third gaze data 105 relating to the gaze of both eyes from the first gaze data 103 and the second gaze data 104, it can generate data that facilitates the analysis of gaze information.

[0057] In this embodiment, the acquisition unit 130 acquires posture data 106 related to the posture of the user 2's body from the terminal 20. The generation unit 150 generates recorded data 107 based on the third gaze data 105, information related to the video data 102 (i.e., virtual space data 101), and the posture data 106. Therefore, the information processing device 10 can reflect information related to the posture of the user 2's body in the recorded data 107, and thus can generate data that facilitates the analysis of gaze information, including more detailed information.

[0058] Furthermore, in this embodiment, the video data 102 is a video captured by a virtual camera CAM located within the virtual space AREA, with its angle and position changing in conjunction with the movements of the user 2's body. Therefore, the information processing device 10 can generate data that facilitates the analysis of gaze information with respect to the video captured within the virtual space AREA.

[0059] Furthermore, in this embodiment, the recorded data 107 is a video re-captured by a virtual camera CAM based on information regarding the third gaze data 105 and video data 102 (i.e., virtual space data 101) and posture data 106. Therefore, the information processing device 10 can generate data that facilitates the analysis of gaze information with respect to a video captured within the virtual space AREA, while reflecting information regarding the posture of the user 2's body.

[0060] Furthermore, in this embodiment, the recorded data 107 is a video re-captured by a virtual camera CAM so that the angle and position change in conjunction with the movements of the user 2's body. Therefore, the information processing device 10 can generate the recorded data 107 as a first-person perspective video based on the user 2 within the virtual space AREA.

[0061] Furthermore, in this embodiment, the recorded data 107 is a video of a predetermined area W3 within the virtual space AREA captured by a virtual camera CAM. Therefore, the information processing device 10 can generate the recorded data 107 as a video with an overhead perspective, since it is a video of a predetermined area W3 within the virtual space AREA.

[0062] <Variation> It should be noted that the present invention is not limited to the embodiments described above. That is, any design modifications made to the above embodiments by those skilled in the art are also included within the scope of the present invention, as long as they retain the features of the present invention. Furthermore, the elements of the above embodiments and the modifications described later can be combined to the extent that it is technically possible, and any combination thereof is also included within the scope of the present invention, as long as it retains the features of the present invention.

[0063] For example, in this embodiment, the generation unit 150 generates the recorded data 107 as a video, but is not limited to this. The generation unit 150 may also generate the recorded data 107 as a file in a format other than video, such as a text file or a CSV (comma-separated values) file. With this configuration, the information processing device 10 can generate recorded data 107 in a format other than video, so it can generate data more suitable for analyzing gaze information.

[0064] Furthermore, in this embodiment, the information processing device 10 plays back the virtual space data 101 and transmits the video data 102 captured within the virtual space AREA to the terminal 20, but is not limited to this. The information processing device 10 may transmit pre-prepared video data 102 to the terminal 20 without capturing the video data 102 using the virtual space data 101. In addition, when generating the recorded data 107, the information processing device 10 may generate the recorded data 107 by adding visually recognizable effects (such as circles or trajectories) to the video data 102 at the playback time and coordinates indicated by the third gaze data 105. Therefore, even when the virtual space data 101 is not used, the information processing device 10 can generate data that facilitates the analysis of gaze information.

[0065] Furthermore, after the information processing device 10 generates video data 102 by imaging within the virtual space AREA, it may generate the recorded data 107 using the video data 102 and the third gaze data 105 without re-imaging. Specifically, the information processing device 10 generates the recorded data 107 by applying a visually recognizable effect to the video data 102, indicating the playback time and coordinates shown by the third gaze data 105. Therefore, by transmitting video data 102 using virtual space data 101 while not re-imaging when generating the recorded data 107, the information processing device 10 can reduce the burden of the recording data 107 generation process and generate data that facilitates the analysis of gaze information. [Explanation of symbols]

[0066] 2...User, 10...Information Processing Device (Computer), 20...Terminal, 120...Transmission Unit, 130...Acquisition Unit, 140...Calculation Unit, 150...Generation Unit

Claims

1. A transmission unit that sends video data to the device worn by the user, An acquisition unit that acquires first gaze data relating to the user's right eye gaze and second gaze data relating to the user's left eye gaze from the terminal, A calculation unit calculates a third line of sight data relating to the gaze of both eyes of the user based on the first line of sight data and the second line of sight data. A generation unit generates recording data based on the third gaze data calculated by the calculation unit and the video data information, An information processing device equipped with the following features.

2. The acquisition unit acquires posture data relating to the user's body posture from the terminal. The generation unit generates the recording data based on the third gaze data, the video data information, and the posture data. The information processing apparatus according to claim 1.

3. The aforementioned video data is a video captured by a virtual camera located in a virtual space, the angle and position of which change in conjunction with the user's body movements. The information processing apparatus according to claim 2.

4. The recorded data is a video re-captured by the virtual camera based on the third gaze data, the video data information, and the posture data. The information processing apparatus according to claim 3.

5. The recorded data is a video re-recorded by the virtual camera so that the angle and position change in conjunction with the user's body movements. The information processing apparatus according to claim 4.

6. The recorded data is a video of a predetermined area within the virtual space captured by the virtual camera. The information processing apparatus according to claim 4.

7. Computers, A transmission unit that sends video data to the device worn by the user. An acquisition unit that acquires first gaze data relating to the user's right eye gaze and second gaze data relating to the user's left eye gaze from the terminal. A calculation unit calculates a third line of sight data relating to the gaze of both of the user's eyes based on the first line of sight data and the second line of sight data. A generation unit generates recording data based on the third gaze data calculated by the calculation unit and the video data information. A program designed to function as such.

Images