Information processing device, information processing system, information processing program, and information processing method
The information processing device addresses the inefficiency in confirming content sharing by analyzing user gaze ranges to determine if shared content has been successfully viewed, enhancing communication efficiency.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2024-11-27
- Publication Date
- 2026-06-08
Smart Images
Figure 2026093253000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an information processing apparatus.
Background Art
[0002] In recent years, due to the spread of remote video conferencing systems and social networking services (SNS), communication between users such as content sharing among users has been actively carried out. However, in such communication between users, there is a problem that it is difficult to know where the other party's line of sight is compared to the case of actual face-to-face communication. To address such a problem, Patent Document 1 discloses a method and apparatus for receiving the line-of-sight data of users of other communication terminals, specifying the position of the line of sight of the user with respect to the display image, and displaying the viewpoint information of the user at the display position.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] Here, users who share content have a desire to confirm with the sharing partner whether the content they want to share has been successfully shared. However, in the prior art disclosed in Patent Document 1 mentioned above, although it is possible to confirm where the line of sight of the partner who shared the content is, it is troublesome to follow the time-series change of the line-of-sight position to confirm whether the content that the user wants to share has been successfully shared.
[0005] This invention has been made in view of these problems, and aims to provide an information processing device that reduces the effort required for a user sharing content to confirm with the recipient whether the content they wished to share has been successfully shared. [Means for solving the problem]
[0006] To achieve the above objective, the information processing apparatus of the present invention is characterized by comprising: receiving means for receiving an image and a first gaze range in the image that includes the trajectory of the gaze of a first user using the external device over a predetermined period of time; display control means for controlling the image received by the receiving means to be displayed on a display unit; detection means for detecting a second gaze range that includes the trajectory of the gaze of a second user using the information processing apparatus over a predetermined period of time; acquisition means for acquiring first information based on the difference between the first gaze range and the second gaze range; and transmission means for transmitting the first information to the external device. [Effects of the Invention]
[0007] According to the present invention, it is possible to provide an electronic device that makes it possible to confirm whether or not another user has viewed a specific area of focus within a content image that the user wishes to share. [Brief explanation of the drawing]
[0008] [Figure 1] This is an external view of wearable terminal 10 and wearable terminal 510 according to the first embodiment of the present invention. [Figure 2] This is a block diagram showing examples of the configurations of wearable terminals 10 and 510 according to the first embodiment of the present invention. [Figure 3] This is a flowchart of an information processing device according to the first embodiment of the present invention. [Figure 4] This figure illustrates an example of gaze range detection according to the first embodiment of the present invention. [Figure 5]This figure illustrates a specific example of the content sharing degree according to the first embodiment of the present invention, and a specific example of an image showing the determination result of the content sharing degree. [Modes for carrying out the invention]
[0009] The embodiments will be described below with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant explanations will be omitted as appropriate. Furthermore, some components, members, and processes will be omitted in each drawing. The following embodiments are not limiting to the present invention, and not all combinations of features described in these embodiments are essential to the solutions of the present invention. The configuration of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (operating conditions, operating environment, etc.). In the following embodiments, the same components will be denoted by the same reference numerals.
[0010] (First embodiment) <Internal configuration of information processing device> Figures 1(a) and 1(b) show external views of wearable terminal 10 and wearable terminal 510, which are information processing devices to which the present invention can be applied. In this embodiment of the present invention, two users using wearable terminals are assumed, and the explanation will be given using wearable terminal 510 as the content image sender and wearable terminal 10 as the content image receiver. However, since the components of wearable terminal 10 and wearable terminal 510 are the same, the details of wearable terminal 10 as the content image receiver will be explained, and the explanation of wearable terminal 510 as the content image sender will be omitted. In this embodiment, the information processing device is described as AR glasses, but is not limited to this. It may also be an information processing device such as an HMD, a PC (personal computer) connected to an HMD, a tablet terminal, or a smartphone. Furthermore, it may be an information processing system equipped with each of the components of the information processing device of this embodiment.
[0011] In the wearable device shown in Figure 1(a), the right-eye display unit 160R and the left-eye display unit 160L are display units that include an eyepiece 16, a display unit 29, and an eyeball detection unit 161, respectively, as described later in Figure 2(a). The user can view the image displayed on the internal display unit 29 through the eyepiece 16. The display unit 29 is a transparent display unit and is positioned to cover the user's field of view. The user perceives the image displayed on the display unit 29 as superimposed on their real field of view.
[0012] The right-eye imaging unit 150R and the left-eye imaging unit 150L are imaging units that include the lens 103 and imaging unit 22 described later, respectively, and are arranged to capture an area corresponding to the user's field of view. The captured images for the right and left eyes can also be displayed on the right-eye display unit 160R and the left-eye display unit 160L, respectively. The right-eye imaging unit 150R and the left-eye imaging unit 150L are each equipped with a zoom mechanism, and the zoom magnification can be changed using the operation unit 70. The zoom operation can be a combination of optical zoom using the lens zoom mechanism and electronic zoom by cropping the captured image.
[0013] The power switch 72 is an operating component that switches the power of the wearable terminal 10 ON and OFF.
[0014] Speaker 90 is a speaker designed to notify the user with various sounds when the user is wearing it.
[0015] In Figure 1(b), the right eye imaging unit 650R, the left eye imaging unit 650L, the right eye display unit 660R, and the left eye display unit 660L correspond to the right eye imaging unit 150R, the left eye imaging unit 150L, the right eye display unit 160R, and the left eye display unit 160L in Figure 1(a). Also, the power switch 572 and speaker 590 in Figure 1(b) correspond to the power switch 72 and speaker 90 in Figure 1(a).
[0016] <Block diagram of a wearable device> FIG. 2(a) and FIG. 2(b) show block diagrams illustrating configuration examples of the wearable terminal 10 and the wearable terminal 510 according to the present embodiment. Here too, since the components of the wearable terminal 10 and the wearable terminal 510 are the same, the details of the wearable terminal 10 of the recipient of the content image will be described, and the description of the wearable terminal 510 of the sender of the content image will be omitted.
[0017] The system control unit 50 is a control unit composed of at least one processor or circuit, and controls the entire wearable terminal 10. By executing the program recorded in the non-volatile memory 56 described later, each process of the present embodiment described later is realized. For example, RAM is used for the system memory 52, and constants, variables, programs read from the non-volatile memory 56, etc., for the operation of the system control unit 50 are expanded. Further, the system control unit 50 performs display control by controlling the memory 32, the display unit 29, etc. For example, the display of the display unit 29 is changed based on the determination result of the user state determination unit 165 described later. Also, the system control unit 50 notifies the user, based on the determination result of the user state determination unit 165 described later, that there has been a request to share an image from another person, by display or sound. Note that the means of notification is not limited to display and sound. Note that instead of the system control unit 201 controlling the entire apparatus, the entire apparatus may be controlled by a plurality of hardware sharing the processing.
[0018] The non-volatile memory 56 is an electrically erasable and recordable memory, and for example, Flash-ROM or the like is used. In the non-volatile memory 56, constants, programs, etc., for the operation of the system control unit 50 are stored. The program here refers to the program for executing the various flowcharts described later in the present embodiment.
[0019] The lens 103 is composed of multiple lenses, but in FIG. 2, it is shown simplified as only one lens. The system control unit 50 communicates with the lens system control circuit 4 and controls the aperture 1 via the aperture drive circuit 2 to achieve aperture control. Also, it focuses on the subject by displacing the focus lens within the lens 103 via the AF drive circuit 3.
[0020] The imaging unit 22 is an imaging device composed of a CCD, a CMOS element, etc. that converts an optical image into an electrical signal. The imaging unit 22 incorporates an A / D converter and is used to convert the analog signal output from the imaging unit 22 into a digital signal. Imaging by the imaging unit 22 is synchronized with the horizontal synchronization signal and the vertical line synchronization signal output from a timing generator unit (not shown), and outputs the image data of one frame as frame data at the period of the vertical line synchronization signal. The imaging unit 22 is a frame-based synchronous sensor with respect to the event sensor 163, which is an asynchronous event-based sensor described later.
[0021] The image processing unit 24 performs resizing processing such as predetermined pixel interpolation and reduction, and color conversion processing on the data from the imaging unit 22 or the data from the memory control unit 15 described later. Also, the image processing unit 24 performs predetermined arithmetic processing using the captured image data. Based on the arithmetic result obtained by the image processing unit 24, the system control unit 50 performs exposure control and distance measurement control. Thereby, AF (Auto Focus) processing and AE (Automatic Exposure) processing in the TTL (Through-The-Lens) method are performed. In the image processing unit 24, further, predetermined arithmetic processing is performed using the captured image data, and AWB (Auto White Balance) processing in the TTL method is performed based on the obtained arithmetic result. Also, picture style processing that can selectively set whether to make the captured image a color image or a monochrome image, etc. as shooting parameters is possible.
[0022] The memory control unit 15 controls the transmission and reception of data between the imaging unit 22, the image processing unit 24, and the memory 32. Output data from the imaging unit 22 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and image data for display on the display unit 29.
[0023] Furthermore, memory 32 also serves as memory for image display (video memory). Display image data written to memory 32 is displayed by the display unit 29 via the memory control unit 15.
[0024] The display unit 29 displays information on a display device such as an LCD or organic EL display in accordance with the signal from the memory control unit 15. By sequentially transferring the image data stored in the memory 32 to the display unit 29 and displaying it, a continuous display of the captured image can be performed.
[0025] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of the built-in clock.
[0026] The operation unit 70 consists of various operating components that act as input units for receiving user input. The operation unit 70 includes, for example, a voice UI (User Interface) and a touchpad. The touchpad is mounted on the side (not shown) of the wearable terminal 10. The system control unit 50 can detect operations or states on the touchpad. The position coordinates of the finger touching the touchpad are notified to the system control unit 50 via the internal bus, and the system control unit 50 determines what kind of operation (touch operation) was performed on the touchpad based on the notified information. The touchpad may use any type of touch panel from various types such as resistive, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, and optical sensor types, and two or more different types of touch panels may be calibrated.
[0027] The power control unit 31 consists of a battery detection circuit, a DC-DC converter, a switch circuit for switching which blocks are energized, and the like, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 31 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage to each part, including the recording medium 200, for the required period of time. The power supply unit 30 consists of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and lithium-ion batteries, an AC adapter, and the like.
[0028] The recording medium I / F17 is an interface to the recording medium 200, such as a memory card or hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of semiconductor memory, magnetic disks, etc.
[0029] The communication unit 54 connects wirelessly or via a wired cable to transmit and receive data such as video and audio signals with external devices. The communication unit 54 can also connect to a wireless LAN (Local Area Network) or the internet. Furthermore, the communication unit 54 can communicate with external devices using Wi-Fi (Wireless Fidelity) (registered trademark), Bluetooth (registered trademark), and Bluetooth Low Energy.
[0030] The eyepiece detection unit 57 is a wear detection sensor that detects whether or not a user is wearing the wearable terminal 10. The system control unit 50 can switch the wearable terminal 10 on (power on) / off (power off) according to the state detected by the eyepiece detection unit 57. The eyepiece detection unit 57 can use, for example, an infrared proximity sensor, and may be configured to detect the approach of some object to the eyepiece 16. When an object approaches, infrared light emitted from the light-emitting part (not shown) of the eyepiece detection unit 57 is reflected and received by the light-receiving part (not shown) of the infrared proximity sensor. The amount of infrared light received can be used to determine how close the object is to the eyepiece 16. Note that the infrared proximity sensor is just one example, and other sensors such as a capacitive type may be used.
[0031] The eyeball detection unit 161 consists of an eyeball detection lens 162 (described later), an event sensor 163, and an event data calculation unit 164, and is capable of detecting eyeball information of a user wearing the wearable terminal 10.
[0032] The infrared light emitted from the infrared light-emitting diode 58 is reflected by the eyeball (eye) 166, and this reflected infrared light is imaged onto the imaging surface of the event sensor 163 through the eyeball detection lens 162.
[0033] The event sensor 163 is an event-based vision sensor that detects changes in the brightness of light incident on each pixel and outputs information about the pixel where the brightness change occurred, asynchronously from other pixels. The data output from the event sensor 163 (hereinafter referred to as event data) includes, for example, the position coordinates of the pixel where the brightness change (event) occurred, the polarity (positive or negative) of the brightness change, and timing information corresponding to the time the event occurred. Compared to existing frame-based synchronous sensors such as the imaging unit 22, the event sensor 163 eliminates redundancy in the output information and features high-speed operation, high dynamic range, and low power consumption. On the other hand, the event data is asynchronously output for each pixel. In order to determine the relationship between event data, it is necessary to accumulate event data that occurs over a predetermined period of time and perform various calculations on the results.
[0034] The event data calculation unit 164 is a calculation unit for detecting eyeball information based on event data output continuously and asynchronously from the event sensor 163. For example, it accumulates event data that occurs over a predetermined period of time and processes them as a set of data to determine the presence or absence of eyeball information. By changing the accumulation time for event data, it is possible to detect multiple eyeball information with different occurrence speeds. Eyeball information includes eye movements, such as gaze position information, saccade information including the direction and speed of saccades, and microsaccade information including the frequency and amplitude of microsaccades. It may also include pupil information including pupil size and its changes, and blink information such as blink speed and number of blinks. The detectable eyeball information is merely an example and is not limited to these. The event data calculation unit 164 may be configured to map the event data for the accumulation time as image data for one frame based on the event occurrence coordinates and perform image processing. This makes it possible to obtain multiple eyeball information from event data using the method conventionally used in frame-based image data processing.
[0035] The user state determination unit 165 is a determination unit that determines the user's state based on eyeball information detected by the event data calculation unit 164. For example, it can determine the gaze range or degree of gaze (overall view) from the frequency and amplitude of microsaccades. Here, gaze range is synonymous with attention range and focus range. The degree of gaze is an index that is higher the narrower the gaze range and lower the wider it is. Overall view is defined as the opposite of degree of gaze. In addition, it can determine the user's level of concentration (state of concentration) or fatigue from the frequency and amplitude of microsaccades, pupil size and change amount, and blinking speed and number. Furthermore, preferences such as facial preference are related to the speed of microsaccades and pupil diameter, and can be determined from both parameters. The user state determination unit 165 takes parameters related to each of these, such as microsaccades, blinking, pupil, and the identification results of the subject identification unit 70, as input. The user state determination unit 165 can be composed of a neural network or algorithm that outputs indicators representing the user state, such as gaze range, gaze intensity (antonym: overarching view), concentration level (antonym: fatigue level), and preference level (hereinafter referred to as user state information). However, the configuration of the user state determination unit 165 is not limited to the above, and other determination means may be used. The eye information and determination results used in the user state determination unit 165 are not limited to the above, and other configurations may be used.
[0036] The eye-tracking input setting unit 167, via the system control unit 50, sets whether eye-tracking detection by the eyeball detection unit 161 is enabled or disabled. Alternatively, it can set parameters and detection conditions related to the event data calculation unit 164 and the user state determination unit 165. These can be arbitrarily set by the user via menu settings.
[0037] Furthermore, the system control unit 50 can acquire information on which area of the display unit 29 the captured subject (real object) is displayed in and at what size. In addition, the system control unit 50 can also acquire information on which area of the display unit 29 the user is looking at, using the eyeball detection unit 161. This allows the system control unit 50 to determine which area of the subject the user is looking at.
[0038] The content sharing degree determination unit 168 receives content images, gaze position information, and gaze range (hereinafter referred to as gaze information) from the sender's wearable terminal 510 via the communication unit 54. The content sharing degree determination unit 168 also uses the gaze information of the receiver's wearable terminal 10 user when viewing the content image to determine the content sharing degree, which indicates the degree to which the content was shared as intended by the sender's wearable terminal 510 user. Details will be described later in Figures 3 and 5, but in the example described later, whether the content was shared as intended is determined by the degree of difference (degree of agreement) in the gazed areas between the user sending the content and the user receiving the content. Note that the determination may be made based on the difference in the gazed areas without considering the order of gaze or the duration of gaze, or it may be made while considering the order of gaze and the duration of gaze.
[0039] The notification control unit 171 notifies the user, via display or sound, that a request to share a visual image has been made by another user, based on the results of the user status determination unit 165 and the content sharing degree determination unit 168. Other means of notification are also acceptable. Details will be described later in Figure 5.
[0040] The display control unit 172 changes the display on the display unit 29 based on the determination results of the user state determination unit 165 and the content sharing degree determination unit 168. Details will be described later in Figure 5.
[0041] In Figure 2(b), the aperture 501, aperture drive circuit 502, AF drive circuit 503, and lens system control circuit 504 correspond to the aperture 1, aperture drive circuit 2, AF drive circuit 3, and lens system control circuit 4 in Figure 2(a). Also in Figure 2(b), the memory control unit 515, eyepiece unit 516, recording medium I / F 517, imaging unit 522, and image processing unit 524 correspond to the memory control unit 15, eyepiece unit 16, recording medium I / F 17, imaging unit 22, and image processing unit 24 in Figure 2(a). Furthermore, in Figure 2(b), the display unit 529, power supply unit 530, power supply control unit 531, and memory 532 correspond to the display unit 29, power supply unit 30, power supply control unit 31, and memory 32 in Figure 2(a). Furthermore, in Figure 2(b), the system control unit 550, system memory 552, system timer 553, and lens 603 correspond to the system control unit 50, system memory 52, system timer 53, and lens 103 in Figure 2(a). Also, in Figure 2(b), the communication unit 554, non-volatile memory 556, eyepiece detection unit 557, infrared light-emitting diode 558, and operation unit 570 correspond to the communication unit 54, non-volatile memory 56, eyepiece detection unit 57, infrared light-emitting diode 58, and operation unit 70 in Figure 2(a). Furthermore, in Figure 2(b), the eyeball detection unit 661 and eyeball detection lens 662 correspond to the eyeball detection unit 161 and eyeball detection lens 162 in Figure 2(a). Furthermore, in Figure 2(b), the event sensor 663 and the event data calculation unit 664 correspond to the event sensor 163 and the event data calculation unit 164 in Figure 2(a). Also, in Figure 2(b), the user state determination unit 665, the eyeball (eye) 666, the gaze input setting unit 667, and the content sharing unit 668 correspond to the user state determination unit 165, the eyeball (eye) 166, the gaze input setting unit 167, and the content sharing unit 168 in Figure 2(a). Furthermore, in Figure 2(b), the notification control unit 671, the display control unit 672, and the recording medium 700 correspond to the notification control unit 171, the display control unit 172, and the recording medium 200 in Figure 2(a).
[0042] <Specific examples of gaze range detection> Next, an example of gaze range detection according to the first embodiment of the present invention will be described with reference to Figures 4(a), 4(b), 4(c), and 4(d). Here, Figures 4(a) and 4(b) are microsaccade information, which is one of the eyeball information acquired by the event data calculation unit 164. Figures 4(c) and 4(d) are diagrams representing the user's gaze state on the display 29.
[0043] Figure 4(a) shows the microsaccade waveform when the gaze range is relatively wide. The vertical axis represents the position of the pupil center in the ocular-center coordinate system (expressed as an angle), and the horizontal axis represents time. 401 shows the change in eye position when a microsaccade occurs. The wider the gaze range, the larger the amplitude of eye movement and the higher the oscillation (lower the attenuation rate) tend to be. Also, the frequency of microsaccade occurrences over a certain period, as shown in 402, tends to be higher.
[0044] On the other hand, Figure 4(b) shows the microsaccade waveform when the gaze range is relatively narrow. 403 shows the change in eye position per unit time when microsaccades occur. The narrower the gaze range, the smaller the amplitude of eye movement and the lower the oscillation (higher the attenuation rate) tend to be. Also, the frequency of microsaccade occurrence tends to be lower in 404, which represents the same period as 402. By using these eye movement characteristics as parameters, the width of the gaze range can be estimated.
[0045] Figure 4(c) shows the user's gaze state when the gaze range is relatively wide, as in Figure 4(a). Image 410 shows the area visible to the user via the wearable device 10, and includes the first object 406 and the second object 407. The first gaze range 405 is shown by a dotted line, which represents the gaze range of the predetermined size mentioned above. The user's gaze range 408 is shown by a solid line, and the user's gaze range 408 is larger than the first gaze range 405. This assumes a state where the user is looking over relatively far ahead. In other words, it is a state where the user is not looking (looking over) as much as the standard degree of gaze.
[0046] Next, Figure 4(d) shows the user's gaze state when the gaze range is relatively narrow, as in Figure 4(b). The user's gaze range 409 is shown by a solid line, and the user's gaze range 409 is smaller than the first gaze range 405. This assumes a relatively focused state in which the user is looking at the first object 406. In other words, the user is looking at it more than the baseline level of focus.
[0047] <Flowchart for determining content sharing level and notification methods> The following explanation, with reference to Figure 3, describes how to determine the degree of content sharing and how to notify users when a user of the receiving wearable device 10 views content images received from a user of the transmitting wearable device 510.
[0048] Figure 3 is a flowchart illustrating a method for determining the degree of content sharing and a method for notifying the degree of content sharing between two wearable terminals, a receiver and a transmitter, according to a first embodiment of the present invention. Each process in the flowchart of Figure 3 is realized by the system control unit 50 of the receiver wearable terminal 10, which expands the program stored in the non-volatile memory 56 into the system memory 52 and executes it, controlling each functional block. Similarly, the system control unit 550 of the transmitter wearable terminal 510 expands the program stored in the non-volatile memory 556 into the system memory 552 and executes it, controlling each functional block.
[0049] Step S301 begins with the receiving wearable terminal 10 being activated, and the display image data written to the memory 32 becoming ready for display by the display unit 29 via the memory control unit 15.
[0050] Furthermore, step S306 begins with the transmission-side wearable terminal 510 being activated, and the display image data written to memory 532 becoming displayable by the display unit 529 via the memory control unit 515.
[0051] In step S302, the system control unit 550 records the image of the subject viewed by the user of the transmitting wearable terminal 510 (hereinafter referred to as the transmitting user) as a content image in the memory 532 via the imaging unit 522. At the same time, the system control unit 550 acquires the movement (gaze information) of the transmitting user's eyeballs (eyes) 666 via the event sensor 663 and similarly records it in the memory 532.
[0052] In step S303, the system control unit 550 transmits the content image and gaze information recorded in step 302 to the receiving wearable terminal 10 via the communication unit 554.
[0053] In step S307, the system control unit 50 of the receiving wearable terminal 10 receives the content image and gaze information transmitted from the transmitting wearable terminal in step S303 via the communication unit 54. The system control unit 50 may also notify the user when the content image and gaze information are received. The notification method may be different depending on whether both the content image and gaze information are received, or whether only the content image is received and gaze information is not. For example, if both the content image and gaze information are received, the system control unit 50 may further notify the user that gaze information is included.
[0054] In step S308, the system control unit 50 displays the content image received in S307 by the user of the receiving wearable terminal 10 (hereinafter referred to as the receiving user) on the display unit 29 via the display control unit 172. It also records the movement of the receiving user's eyeballs (eyes) 166 during the period in which the content image is being played.
[0055] In step S309, the system control unit 50 uses the content sharing degree determination unit 168 to determine the degree to which the content was shared as intended by the sender, based on the gaze information of the sending user and the gaze information of the receiving user. The method for determining the content sharing degree will be described later with reference to Figure 5.
[0056] In step S310, the system control unit 50 transmits the content sharing level determined in step 309 to the transmitting wearable terminal 510 via the communication unit 54, and terminates the flow for the receiving wearable terminal 10. If the system control unit 50 determines that the content sharing level is low, it may determine that the shared content has not been viewed as intended by the transmitting user and may display the content image again.
[0057] In step S304, the system control unit 550 of the transmitting wearable terminal 510 receives the content sharing degree determination result determined by the receiving wearable terminal 10 via the communication unit 554.
[0058] In step S305, the system control unit 550 of the transmitting wearable terminal 510 displays the content sharing degree determination result received in step S304 on the display unit 529, and the flow of the transmitting wearable terminal 510 ends.
[0059] <Specific examples of methods for determining the degree of content sharing> Next, the determination of the content sharing degree in step S309 of the receiving wearable terminal 10 will be explained with reference to Figures 5(a), 5(b), 5(c), 5(d), and 5(e).
[0060] Figures 5(a), 5(b), 5(c), 5(d), and 5(e) illustrate a method for determining the degree of content sharing and a method for notifying the degree of content sharing between two wearable terminals, a receiver and a transmitter, according to a first embodiment of the present invention.
[0061] <Examples of high content sharing> First, let's refer to Figure 5(a) to explain an example of a case where a high degree of content sharing is determined. The upper image shows the user state determination unit 165 of the receiving wearable terminal 10 superimposing the gaze information of the receiving user onto the content image 700. The lower image shows the user state determination unit 665 of the transmitting wearable terminal 510 superimposing the gaze information of the transmitting user onto the content image 700. Note that the content image is the same for both the receiving and transmitting sides, and it is assumed that a still image is displayed continuously for a predetermined period of time. In addition, time intervals for determining the degree of matching are defined at predetermined intervals, and in Figure 5(a), it is divided into three time intervals. Here, in this example, it is assumed that the wearable terminals 10 and 510 divide the time intervals at which the user is detected to have been fixated on the image, and that time intervals at which the user simply moved their gaze are not detected. The transmitting user may also set the time intervals at which they were fixed on the image.
[0062] In the receiving side of Figure 5(a), in the first time interval 713, the receiving user's gaze range 701 and the gaze information trajectory 702 are superimposed, with the gaze information trajectory 702 being contained within the gaze range 701. In the second time interval 714, the receiving user's gaze range 703 and the gaze information trajectory 704 are superimposed, with the gaze information trajectory 704 being contained within the gaze range 703. In the third time interval 715, the receiving user's gaze range 705 and the gaze information trajectory 706 are superimposed, with the gaze information trajectory 706 being contained within the gaze range 705. Similarly, on the transmitting side, in the first time interval 713, the receiving user's gaze range 707 and the gaze information trajectory 708 are superimposed, with the gaze information trajectory 708 being contained within the gaze range 707. Furthermore, in the second time interval 714, the receiving user's gaze range 709 and the gaze information trajectory 710 are superimposed, with the gaze information trajectory 710 being contained within the gaze range 709. In the third time interval 715, the receiving user's gaze range 711 and the gaze information trajectory 712 are superimposed, with the gaze information trajectory 712 being contained within the gaze range 711. Also, the example in Figure 5(a) shows an example in which multiple relatively narrow gaze ranges (viewing a specific subject) are detected. Furthermore, for example, if gaze information indicating that the user is looking at a subject is detected, the time interval and region of the gaze are stored, and the degree of sharing is determined.
[0063] Furthermore, in Figure 5(a), it is assumed that in the first time interval 713, the second time interval 714, and the third time interval 715, the gaze ranges of the receiving user and the transmitting user are almost identical. In such a case, the receiving user is shifting their gaze as intended by the transmitting user, and the degree of content sharing is determined to be high. The degree of gaze range agreement is defined as the degree to which the gaze ranges viewed by the transmitting and receiving users match by a predetermined area or more, and the gaze range threshold can be arbitrarily determined. That is, in a case like Figure 5(a), the system control unit 50 of the receiving wearable terminal 10 compares the gaze range areas viewed by the transmitting and receiving users in each of the three time intervals via the content sharing degree determination unit 168. The system control unit 50 also determines that the degree of content sharing is high because the gaze range areas match by a threshold or more in all three time intervals. In this example, we assumed a scenario where the order of focus was the same. However, a high degree of content sharing may be determined only when the order is the same, or it may be determined that a high degree of content sharing is determined if the areas of focus are similar, regardless of the order. Furthermore, the degree of content sharing may be determined not only from the area of focus but also from the time spent focusing.
[0064] Furthermore, in Figure 5(a), if it is determined that the content sharing level is high, the result of this determination is sent from the wearable terminal 510 to the wearable terminal 10. In this example, an image 716 indicating that the content image was shared as intended is sent from the wearable terminal 510 to the wearable terminal 10. The content sent to the wearable terminal 510 is controlled to change according to the content sharing level, and in Figure 5(a), the avatar's facial expression is controlled to change. In this example, the avatar's facial expression is used to represent the content sharing level, but it may be expressed in another way. For example, a numerical value such as a percentage may be used.
[0065] <Examples of content sharing with a relatively high degree> Next, with reference to Figure 5(b), an example of a case where the degree of content sharing is determined to be somewhat high will be explained. The upper image shows the user state determination unit 165 of the receiving wearable terminal 10 superimposing the gaze information of the receiving user onto the content image 720. The lower image shows the user state determination unit 665 of the transmitting wearable terminal 510 superimposing the gaze information of the transmitting user onto the content image 720. Note that the content image is the same for both the receiving and transmitting sides, and it is assumed that a still image is displayed continuously for a predetermined period of time. Furthermore, time intervals for determining the degree of matching are defined at predetermined intervals, and in Figure 5(b), it is divided into three time intervals.
[0066] In the receiving side of Figure 5(b), in the first time interval 733, the receiving user's gaze range 721 and the gaze information trajectory 722 are superimposed, with the gaze information trajectory 722 being contained within the gaze range 721. In the second time interval 734, the receiving user's gaze range 723 and the gaze information trajectory 724 are superimposed, with the gaze information trajectory 724 being contained within the gaze range 723. In the third time interval 735, the receiving user's gaze range 725 and the gaze information trajectory 726 are superimposed, with the gaze information trajectory 726 being contained within the gaze range 725. Similarly, on the transmitting side, in the first time interval 733, the receiving user's gaze range 727 and the gaze information trajectory 728 are superimposed, with the gaze information trajectory 728 being contained within the gaze range 727. Furthermore, in the second time interval 734, the receiving user's gaze range 729 and the trajectory of the gaze information 730 are superimposed, with the trajectory of the gaze information 730 being contained within the gaze range 729. Also, in the third time interval 735, the receiving user's gaze range 731 and the trajectory of the gaze information 732 are superimposed, with the trajectory of the gaze information 732 being contained within the gaze range 731.
[0067] Furthermore, in Figure 5(b), in the second time interval 734, the receiving user and the transmitting user are viewing different subjects. That is, there is a difference between the gaze range 723 and gaze information trajectory 724 of the receiving wearable terminal 10 and the gaze range 729 and gaze information trajectory 730 of the transmitting wearable terminal 510. Therefore, the system control unit 50 determines that the degree of content sharing is somewhat high, as the gaze range areas match by more than a threshold in two of the three time intervals.
[0068] Furthermore, in Figure 5(b), if it is determined that the degree of content sharing is high, the result of this determination is transmitted from the wearable terminal 510 to the wearable terminal 10. In this example, image 736, which indicates that the content image was shared somewhat as intended, is transmitted from the wearable terminal 510 to the wearable terminal 10.
[0069] <Examples of low content sharing> Next, with reference to Figure 5(c), an example of a case where the degree of content sharing is determined to be low will be explained. The upper image shows the user state determination unit 165 of the receiving wearable terminal 10 superimposing the gaze information of the receiving user onto the content image 740. The lower image shows the user state determination unit 665 of the transmitting wearable terminal 510 superimposing the gaze information of the transmitting user onto the content image 740. Note that the content image is the same for both the receiving and transmitting sides, and it is assumed that a still image is displayed continuously for a predetermined period of time. Furthermore, time intervals for determining the degree of matching are defined at predetermined intervals, and in Figure 5(c), it is divided into three time intervals.
[0070] In the receiving side of Figure 5(c), in the first time interval 753, the receiving user's gaze range 741 and the gaze information trajectory 742 are superimposed, with the gaze information trajectory 742 being contained within the gaze range 741. In the second time interval 754, the receiving user's gaze range 743 and the gaze information trajectory 744 are superimposed, with the gaze information trajectory 744 being contained within the gaze range 743. In the third time interval 755, the receiving user's gaze range 745 and the gaze information trajectory 746 are superimposed, with the gaze information trajectory 746 being contained within the gaze range 745. Similarly, on the transmitting side, in the first time interval 753, the receiving user's gaze range 747 and the gaze information trajectory 748 are superimposed, with the gaze information trajectory 748 being contained within the gaze range 747. Furthermore, in the second time interval 754, the receiving user's gaze range 749 and the trajectory of the gaze information 750 are superimposed, with the trajectory of the gaze information 750 being contained within the gaze range 749. Also, in the third time interval 755, the receiving user's gaze range 751 and the trajectory of the gaze information 752 are superimposed, with the trajectory of the gaze information 752 being contained within the gaze range 751.
[0071] Furthermore, in Figure 5(c), the gaze range matches only in the third time interval, 755, out of the three time intervals. In such cases, it is determined that the receiving user did not shift their gaze as intended by the sending user, and the degree of content sharing is judged to be low.
[0072] Furthermore, in Figure 5(c), if it is determined that the content sharing level is low, the result of this determination is sent from the wearable terminal 510 to the wearable terminal 10. In this example, image 756, which indicates that the content image was not shared as intended, is sent from the wearable terminal 510 to the wearable terminal 10.
[0073] <Another example of a high degree of content sharing> Figure 5(d) illustrates another example of a case where a high degree of content sharing is determined. Figure 5(d) shows an example where the gaze range detected by the receiving wearable terminal 10 and the transmitting wearable terminal 510 is wide, that is, when the content is viewed from above. Here, the upper image is a diagram in which the user state determination unit 165 of the receiving wearable terminal 10 superimposes the gaze information of the receiving user onto the content image 760. The lower image is a diagram in which the user state determination unit 665 of the transmitting wearable terminal 510 superimposes the gaze information of the transmitting user onto the content image 760. Note that the content image is the same for both the receiving and transmitting sides, and it is assumed that a still image is displayed continuously for a predetermined time. Furthermore, time intervals for determining the degree of agreement are defined at predetermined time intervals, and in Figure 5(d), it is divided into one time interval 765 in which the content was viewed from above.
[0074] In the receiving side of Figure 5(d), in one time interval 765, the receiving user's gaze range 761 and the trajectory of the gaze information 762 are superimposed, and the trajectory of the gaze information 762 is contained within the gaze range 761. Similarly, on the transmitting side, in the first time interval 765, the receiving user's gaze range 763 and the trajectory of the gaze information 764 are superimposed, and the trajectory of the gaze information 764 is contained within the gaze range 763. In this way, when the entire scene is viewed over a single time interval, the content sharing degree determination unit 168 of the receiving wearable terminal 10 compares the coordinates of the gaze ranges (overall view ranges) seen by the transmitting and receiving users. It also determines that the content sharing degree is high if one out of the gaze ranges match. The example in Figure 5(d) shows an example in which a relatively wide gaze range (viewing the entire scene of the subject) was detected.
[0075] Furthermore, in Figure 5(d), if it is determined that the degree of content sharing is high, the result of this determination is sent from the wearable terminal 510 to the wearable terminal 10. In this example, an image 766 indicating that the content image was shared as intended is sent from the wearable terminal 510 to the wearable terminal 10.
[0076] <Another example of low content sharing> Next, with reference to Figure 5(e), another example of a case where the degree of content sharing is determined to be low will be explained. The upper image shows the user state determination unit 165 of the receiving wearable terminal 10 superimposing the gaze information of the receiving user onto the content image 780. The lower image shows the user state determination unit 665 of the transmitting wearable terminal 510 superimposing the gaze information of the transmitting user onto the content image 780. Note that the content image is the same for both the receiving and transmitting sides, and it is assumed that a still image is displayed continuously for a predetermined period of time. Furthermore, time intervals for determining the degree of matching are defined at predetermined intervals, and in Figure 5(e), the receiving side is divided into two time intervals, while in Figure 5(e), the transmitting side is divided into one time interval.
[0077] In the receiving side of Figure 5(e), in the first time interval 785, the receiving user's gaze range 781 and the gaze information trajectory 782 are superimposed, with the gaze information trajectory 782 being contained within the gaze range 781. In the second time interval 786, the receiving user's gaze range 783 and the gaze information trajectory 784 are superimposed, with the gaze information trajectory 784 being contained within the gaze range 783. Here, the receiving side shows an example where multiple relatively narrow gaze ranges (viewing a specific subject) were detected.
[0078] Next, in the transmitting side of Figure 5(e), in one time interval 793, the receiving user's gaze range (overview range) 791 and the trajectory of the gaze information 792 are superimposed, and the trajectory of the gaze information 792 is contained within the gaze range (overview range) 791. Here, the transmitting side shows an example in which a relatively wide gaze range (the entire subject is visible) is detected.
[0079] Furthermore, in Figure 5(e), if it is determined that the content sharing level is low, the result of this determination is sent from wearable terminal 510 to wearable terminal 10. In this example, image 794, which indicates that the content image was not shared as intended, is sent from wearable terminal 510 to wearable terminal 10.
[0080] In this way, the degree of content sharing can be determined based on the viewing range of the sending user and the receiving user, and the result of this determination can be notified from the receiving user to the sending user.
[0081] In this embodiment, the degree of content sharing is determined by the coordinate position of the gaze area, but it can also be performed using comparison methods linked to the gaze area, such as determining the degree of agreement of the trajectory of gaze information or objects present within the gaze area.
[0082] In the above-mentioned content images, for the sake of clarity, the explanation included diagrams where gaze information and gaze range are superimposed within the content image. However, the shared content images may not include gaze information or gaze range. Alternatively, the content image may be sent with either gaze information or gaze range gaze superimposed, and the system may be configured to allow the receiving user to switch between displaying at least one of the gaze information or gaze range gaze superimposed, depending on their choice. Similarly, the system may be configured to allow the sending user to switch between displaying at least one of the gaze information or gaze range gaze superimposed, depending on their choice. Furthermore, while the gaze range was explained as the area encompassing the gaze trajectory, instead of the gaze range, a specific object that the sending user gazed at may be highlighted according to the gaze trajectory. By displaying gaze information and gaze range superimposed, the receiving user can confirm what the sending user gazed at, making it easier to share content images as intended.
[0083] Here, Figure 5(b) is described as an example where content was shared to some extent according to the sender user's intentions, i.e., an example with a relatively high degree of content sharing. However, even if the relationship is as described in Figure 5(b), it may be judged as having a low degree of content sharing. For example, for each user, criteria for determining a high degree of content sharing may be set based on indicators such as the difference in the viewing range, the order in which the content was viewed, and the viewing time. Alternatively, the degree of content sharing may be determined using a deep learning model that has been trained on these criteria. Note that instead of still images, videos with a predetermined playback time may be shared.
[0084] As described above, this embodiment provides an electronic device that makes it possible to confirm whether or not another user has viewed a specific area of focus within a content image that the user intends to share.
[0085] (Variation 1) In the first embodiment described above, an example was explained in which the content sharing degree determination result is notified from the receiving user to the sending user, but the embodiment is not limited to this. For example, as a modification, when the sending wearable terminal 510 is notified that the content sharing degree is low, the sending user may be asked to choose whether or not to share the content image again. In this case, if the sending user chooses to share the content image again, the same content image may be sent again from the sending wearable terminal 510 to the receiving wearable terminal 10. Alternatively, when the sending wearable terminal 510 is notified that the content sharing degree is low, the content image may be shared again without asking the sending user to choose whether or not to share the content image again.
[0086] (Modification 2) In the first embodiment described above, an example of sharing content images without specifying an application was explained, but the invention is not limited to this. For example, as a modification, the process from sharing such content images to notifying the result of the content sharing determination of the content images may be implemented using the messaging function of an SNS (Social Networking Service) or the like.
[0087] In one-on-one messaging, the user sending the content image needed to not only share the image but also to specify in text which parts they wanted the recipient to review. Furthermore, several exchanges of communication with the recipient were necessary to ensure the content image was shared as intended.
[0088] In this case, for example, if the recipient of the content image can confirm the area that the sender of the content image actually focused on, the sender of the content image will have less trouble sending text separately. Also, by determining the degree of content sharing and notifying the sender of the content image of the result, they can know whether the content image was shared as intended. Furthermore, if the content image was not shared as intended, they can take action to share it again as intended. It is also possible for the sender of the content image to send the content image to an unspecified number of users, and for the recipients of that unspecified number of users to be able to confirm the degree of content sharing. In this case, the recipients' degree of content sharing could be confirmed, for example, in a ranking format. It is also possible for the sender of the content image to receive notification of the content sharing degree determination result when sharing the content image to an individual, but not when sharing the content image to an unspecified number of people.
[0089] (Other embodiments) Furthermore, the present invention can also be realized by performing the following process: that is, supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or control unit or MPU, etc.) of the system or device read and execute the program code. In this case, the program and the storage medium storing the program constitute the present invention.
[0090] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.
[0091] Furthermore, each functional unit in each of the above embodiments (each modified example) may or may not be individual hardware. The functions of two or more functional units may be implemented by common hardware. Each of the multiple functions of a single functional unit may be implemented by individual hardware. Two or more functions of a single functional unit may be implemented by common hardware. In addition, each functional unit may or may not be implemented by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. The functions of at least some of the functional units of the device may be implemented by the processor reading and executing the control program from the memory.
[0092] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.
[0093] Furthermore, in each of the examples described above, "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPUs) and specialized processors (e.g., GPUs, ASICs, FPGAs, and programmable logic devices, etc.).
[0094] This embodiment includes the following configurations, methods, and programs.
[0095] [Configuration 1] An information processing device, Receiving means for receiving an image and a first gaze range in the image that includes the trajectory of the gaze of a first user using the external device over a predetermined period of time, A display control means that controls the display unit to display the image received by the receiving means, A detection means for detecting a second gaze range that includes the trajectory of the gaze of a second user using the information processing device over a predetermined period of time, An acquisition means for acquiring first information based on the difference between the first gaze range and the second gaze range, The transmission means for transmitting the first information to the external device is An information processing device characterized by the following:
[0096] [Configuration 2] The display control means displays the first gaze range or the trajectory of the first user's gaze superimposed on the image. The information processing device according to configuration 1, characterized by the above.
[0097] [Configuration 3] The display control means switches the display of the first gaze range or the trajectory of the first user's gaze on the image based on the instruction of the second user. The information processing apparatus according to configuration 2, characterized in that...
[0098] [Structure 4] The receiving means receives an image in which the first gaze range or the trajectory of the first user's gaze is superimposed on the image, based on the settings in the external device. An information processing device according to any one of configurations 1 to 3.
[0099] [Composition 5] The display control means highlights the object that the first user is looking at. An information processing apparatus according to any one of configurations 1 to 4, characterized by the above.
[0100] [Composition 6] The first piece of information is information indicating the degree to which the first user was able to share the object in the image that they wanted to share with the second user. An information processing device according to any one of configurations 1 to 5, characterized by the above.
[0101] [Composition 7] The first information is determined based on the number of intervals in which the difference between the first gaze range and the second gaze range is less than a threshold, among a plurality of intervals that divide the period during which the first user viewed the image. An information processing device according to any one of configurations 1 to 6.
[0102] [Structure 8] The display control means controls the display to redisplay the image if the degree to which objects in the image were shared is lower than a predetermined threshold. The information processing apparatus according to configuration 6 or 7, characterized by the above.
[0103] [Composition 9] When the receiving means receives the image and the first gaze range, it further has a notification means for notifying that the image includes the first gaze range. An information processing device according to any one of configurations 1 to 8.
[0104] [Configuration 10] The transmission means modifies the content to be transmitted to the external device according to the difference between the first gaze range and the second gaze range. An information processing device according to any one of configurations 1 to 9, characterized by the above.
[0105] [Composition 11] Receiving means for receiving an image from an external device that includes first gaze information relating to the gaze of a first user using the external device, A display control means that controls the display unit to display the image received by the receiving means, A detection means for detecting second gaze information relating to the gaze of a second user using the information processing device, An acquisition means for acquiring first information based on the difference between the first gaze information and the second gaze information, The transmission means for transmitting the first information to the external device is An information processing device characterized by the following:
[0106] [system] An information processing system including a first information processing device and a second information processing device, The first information processing device is A first display control means that controls the display unit to show an image, A first detection means for detecting a first gaze range in the image that includes the trajectory of the gaze of a first user using the first information processing device over a predetermined period of time, The device comprises a first transmission means for transmitting the aforementioned image and the first gaze range to the second information processing device, The second information processing device is A second receiving means receives the image and the first viewing range from the first information processing device, A second display control means controls the display unit to display the image received by the second receiving means, A second detection means for detecting a second gaze range in the aforementioned image that includes the trajectory of the gaze of a second user using the second information processing device over a predetermined period of time, A second acquisition means for acquiring first information based on the difference between the first gaze range and the second gaze range, The system includes a second transmission means for transmitting the first information to the first information processing device. An information processing system characterized by the following:
[0107] [program] A program for causing a computer to function as one of the means of an information processing device described in any one of items 1 to 11.
[0108] [method] The steps of receiving an image from an external device and a first gaze range in the image that includes the trajectory of the gaze of a first user using the external device over a predetermined period of time, The steps include controlling the receiving means to display the image received on the display unit, The steps include detecting a second gaze range that includes the trajectory of the gaze of a second user using the information processing device over a predetermined period of time, A step of acquiring first information based on the difference between the first gaze range and the second gaze range, The step of transmitting the first information to the external device is included. A control method for an information processing device characterized by the following features.
Claims
1. An information processing device, Receiving means for receiving an image and a first gaze range in the image that includes the trajectory of the gaze of a first user using the external device over a predetermined period of time, A display control means that controls the display unit to display the image received by the receiving means, A detection means for detecting a second gaze range that includes the trajectory of the gaze of a second user using the information processing device over a predetermined period of time, An acquisition means for acquiring first information based on the difference between the first gaze range and the second gaze range, The transmission means for transmitting the first information to the external device is An information processing device characterized by the following:
2. The display control means displays the first gaze range or the trajectory of the first user's gaze superimposed on the image. The information processing apparatus according to feature 1.
3. The display control means switches the display of the first gaze range or the trajectory of the first user's gaze on the image based on the instruction of the second user. The information processing apparatus according to feature 2.
4. The receiving means receives an image in which the first gaze range or the trajectory of the first user's gaze is superimposed on the image, based on the settings in the external device. The information processing apparatus according to feature 1.
5. The display control means highlights the object that the first user is looking at. The information processing apparatus according to feature 1.
6. The first piece of information is information indicating the degree to which the first user was able to share the object in the image that they wanted to share with the second user. The information processing apparatus according to feature 1.
7. The first information is determined based on the number of intervals in which the difference between the first gaze range and the second gaze range is less than a threshold, among a plurality of intervals that divide the period during which the first user viewed the image. The information processing apparatus according to feature 1.
8. The display control means controls the display to redisplay the image if the degree to which objects in the image were shared is lower than a predetermined threshold. The information processing apparatus according to feature 6.
9. When the receiving means receives the image and the first gaze range, it further has a notification means for notifying that the image includes the first gaze range. The information processing apparatus according to feature 1.
10. The transmission means modifies the content to be transmitted to the external device according to the difference between the first gaze range and the second gaze range. The information processing apparatus according to feature 1.
11. Receiving means for receiving an image from an external device that includes first gaze information relating to the gaze of a first user using the external device, A display control means that controls the display unit to display the image received by the receiving means, A detection means for detecting second gaze information relating to the gaze of a second user using the information processing device, An acquisition means for acquiring first information based on the difference between the first gaze information and the second gaze information, The transmission means for transmitting the first information to the external device is An information processing device characterized by the following:
12. An information processing system including a first information processing device and a second information processing device, The first information processing device is A first display control means that controls the display unit to display an image, A first detection means for detecting a first gaze range in the image that includes the trajectory of the gaze of a first user using the first information processing device over a predetermined period of time, The device comprises a first transmission means for transmitting the aforementioned image and the first gaze range to the second information processing device, The second information processing device is The first information processing device receives the image and the first viewing range, and a second receiving means receives them. A second display control means controls the display unit to display the image received by the second receiving means, A second detection means for detecting a second gaze range in the aforementioned image that includes the trajectory of the gaze of a second user using the second information processing device over a predetermined period of time, A second acquisition means for acquiring first information based on the difference between the first gaze range and the second gaze range, The system includes a second transmission means for transmitting the first information to the first information processing device. An information processing system characterized by the following:
13. A program for causing a computer to function as each of the means of the information processing apparatus described in claim 1.
14. The steps of receiving an image from an external device and a first gaze range in the image that includes the trajectory of the gaze of a first user using the external device over a predetermined period of time, The steps include controlling the receiving means to display the image received by the receiving means on the display unit, The steps include detecting a second gaze range that includes the trajectory of the gaze of a second user using the information processing device over a predetermined period of time, A step of acquiring first information based on the difference between the first gaze range and the second gaze range, The step of transmitting the first information to the external device is included. A control method for an information processing device characterized by the following features.