Display control device

Abstract

To expand the variety of display modes for documents in a virtual space.SOLUTION: A server 10 displays a document DC including a plurality of pages in a virtual space VS accessed by a plurality of users U-1 to U-3. A first display control unit 111 displays, in a first area R1 of the virtual space VS, a main display page image G1 showing at least one page of the document DC. A second display control unit 112 displays, in a second area R2 of the virtual space VS, at least one thumbnail image G2 including a page of the document DC other than the page displayed as the main display page image G1. A gaze acquisition unit 113 acquires a gaze point of at least one of the plurality of users U-1 to U-3. A position determination unit 114 determines a position of the second area R2 based on the gaze point.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a display control device. [Background technology]

[0002] Conventionally, there are cases where a discussion such as a meeting is conducted by presenting documents (materials) to multiple members. For example, the following Patent Document 1 describes a technology for supporting information sharing in a meeting held to discuss various matters related to the treatment of a patient when a team of multiple medical professionals treats a patient as a team. Prior Document 1 describes how materials that are highly popular among team members in a meeting are extracted (0032), how the extracted materials that are highly popular are registered as best practices, and how the images adopted as best practices are used, for example, when a meeting is held regarding another patient (0033). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-90571 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, XR (Extended Reality) technologies such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality) have become widespread. XR technology enables access to virtual space, and users in physically distant locations can experience a meeting or the like in the same space. On the other hand, the display mode of documents used in meetings or the like is often the same in virtual space and real space.

[0005] An object of the present invention is to broaden the range of display modes of documents in a virtual space. [Means for solving the problem]

[0006] A display control device according to a preferred embodiment of the present disclosure is a display control device that displays a document including multiple pages in a virtual space accessed by multiple users, and includes a first display control unit that displays a main display page image indicating at least one page of the document in a first area of ​​the virtual space, a second display control unit that displays at least one thumbnail image including a page of the document other than the at least one page in a second area of ​​the virtual space, a gaze acquisition unit that acquires a gaze point of at least one of the multiple users, and a position determination unit that determines a position of the second area based on the gaze point. Effect of the Invention

[0007] According to the present disclosure, it is possible to expand the range of display modes of documents in a virtual space. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a system 1 according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a user terminal 20. [Diagram 3] FIG. 2 is a block diagram showing the configuration of a server 10. [Figure 4] FIG. 2 is a diagram illustrating a document DC. [Diagram 5] FIG. 2 is a diagram showing a schematic arrangement of avatars A-1 to A-3 in a virtual space VS. [Figure 6] FIG. 2 is a diagram showing a schematic display of a screen SC. [Figure 7] 13 is a diagram showing another example of setting the second region R2. FIG. [Figure 8] 13 is a diagram showing another example of setting the second region R2. FIG. [Figure 9]13 is a diagram showing an example of a display mode of a second region R2. FIG. [Figure 10] 4 is a flowchart showing the operation of the processing device 103. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A. Embodiment A-1. System configuration FIG. 1 is a block diagram showing a configuration of a system 1 according to an embodiment. The system 1 includes a server 10 and a plurality of user terminals 20 (20-1 to 20-n, n is an integer equal to or greater than 2). The plurality of user terminals 20 are used by different users U (U-1 to Un). The server 10 and the user terminals 20 are connected via a network N. Although three user terminals 20 are shown in FIG. 1, the number of user terminals 20 included in the system 1 may be two, or four or more.

[0010] The system 1 realizes access to a virtual space VS (see FIG. 5, etc.) using a user terminal 20. A user U can access the virtual space VS using the user terminal 20 and have various experiences in the virtual space VS by moving an avatar A that resembles the user. In this embodiment, a case where a plurality of users U (more specifically, a plurality of avatars A-1 to A-3 corresponding to a plurality of users U-1 to U-3, see FIG. 5, etc.) gather in the virtual space VS and hold a conference will be mainly described. In this embodiment, the system 1 is assumed to use VR technology.

[0011] The user terminal 20 is, for example, a head mounted display, a smartphone, a tablet, a mobile information processing terminal such as a personal computer, etc. In this embodiment, it is assumed that the user terminal 20 is a head mounted display.

[0012] The server 10 manages virtual space data VD (see FIG. 3) for outputting images showing the virtual space VS and sounds generated in the virtual space VS on the user terminal 20. The server 10 is an example of a display control device that displays a document DC (see FIG. 4) including multiple pages in the virtual space VS accessed by multiple users U-1 to U-3.

[0013] A-2. User terminal 20 2 is a block diagram showing the configuration of a user terminal 20. The user terminal 20 includes a display device 201, an input device 202, a communication device 203, a microphone 204, a speaker 205, a camera 206, a storage device 208, a processing device 209, and a bus 220 that interconnects these devices.

[0014] The display device 201 is a display device (for example, various display panels such as a liquid crystal display panel or an organic EL display panel) that displays information to the outside. The input device 202 is an input device that accepts input from the outside (for example, a sensor, switch, or button that detects the movement of the user U, or may be a keyboard, mouse, microphone 204, etc.). The display device 201 and the input device 202 may be integrated (for example, a touch panel). The communication device 203 has an interface that can be connected to the network N, and communicates with other devices connected to the network N using wireless communication or wired communication.

[0015] The microphone 204 picks up sounds made by the user U and sounds generated around the user U, and generates voice data. The speaker 205 generates sounds based on voice data acquired from a device other than the user terminal 20. Using the microphone 204 and the speaker 205, the user U can converse with an avatar A of another user U in the virtual space VS, and can listen to sounds generated in the virtual space VS.

[0016] The camera 206 captures an image of a subject and outputs image data corresponding to the captured image. The camera 206 has, for example, an imaging optical system and an imaging element. The imaging optical system is an optical system including at least one imaging lens. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor. In this embodiment, the camera 206 is used to detect the direction of the line of sight (point of gaze) of the user U. Therefore, when the user U uses the user terminal 20, the camera 206 is disposed at a position where the eyes of the user U are included in the imaging range.

[0017] The storage device 208 is a recording medium readable by the processing device 209. The storage device 208 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), and an Electrically Erasable Programmable Read Only Memory (EEPROM). The volatile memory is, for example, a Random Access Memory (RAM). The storage device 208 stores a program PG2. The program PG2 is a program for operating the user terminal 20.

[0018] The processing device 209 includes one or more central processing units (CPUs). The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0019] The processing device 209 reads the program PG1 from the storage device 208. The processing device 209 executes the program PG2 to function as a virtual space access unit 211 and a gaze detection unit 212. These functional units may be configured by circuits such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array).

[0020] A-3. Server 10 3 is a block diagram showing the configuration of the server 10. The server 10 includes a communication device 101, a storage device 102, a processing device 103, and a bus 109 that interconnects these devices.

[0021] The communication device 101 includes an interface connectable to a network N, and communicates with other terminals connected to the network N using wireless or wired communication.

[0022] The storage device 102 is a recording medium readable by the processing device 103. The storage device 102 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM. The storage device 102 stores a program PG1. The program PG1 is a program for operating the server 10.

[0023] The storage device 102 also stores virtual space data VD. The virtual space data VD includes, for example, facility model data indicating the three-dimensional shape of a facility installed in the virtual space VS, and avatar data indicating an avatar A that is the appearance of a user U in the virtual space VS.

[0024] The storage device 102 also stores document data DD. The document data DD is data corresponding to a document DC (see FIG. 4). The document DC includes, for example, text and images, and is used to transmit information. In this embodiment, the document DC includes multiple pages. In this embodiment, the document DC is also assumed to be a document whose contents are organized in units of pages, such as presentation materials. Note that the organization of contents in a document DC may not only be on a per-page basis, but may also be on a per-page basis.

[0025] FIG. 4 is a diagram showing a schematic diagram of a document DC. The document DC shown in FIG. 4 is, for example, a presentation material. The document DC shown in FIG. 4 has 14 pages. In the diagram, the page number X (X=an integer from 1 to 14) of each page is represented as "PX." Page P1 is the cover page of the document DC. For example, the title and creator name of the document DC are written on the cover page.

[0026] Page P2 is the table of contents of document DC. In the table of contents, the title and page number of the page following the table of contents (the main text of document DC) are written. A person viewing document DC can grasp the outline of document DC by looking at the table of contents page. The pages following the table of contents are hierarchically organized according to heading levels. In this embodiment, there are three heading levels: "major heading", "medium heading", and "minor heading". The heading levels correspond to, for example, the hX tag (X=an integer from 1 to 6) in all images in HTML (HyperText Markup Language).

[0027] The main heading corresponds to the title or summary of each chapter when the entire document DC is divided into multiple chapters. The sub-heading corresponds to the title or summary of each chapter when the chapters distinguished by the main heading are further divided into multiple chapters. The sub-heading corresponds to the title or summary of each chapter when the chapters distinguished by the sub-heading are further divided into multiple chapters. That is, the document DC is hierarchized by heading level. In this embodiment, the document DC has three heading levels, so it can be said to be hierarchized into three levels. Note that in this embodiment, the cover and table of contents are not included in the hierarchy by heading level. In addition, the types of headings are not limited to three types, and may be one or two types, or four or more types.

[0028] Depending on the type of document data DD, when a selection operation is performed on the text display of a title shown in the table of contents, the display is set to transition to the page of the title. More specifically, a link to a page corresponding to the title is attached to the text display of a title shown in the table of contents, and when the user U selects the text display, the content of the page linked to the text display is displayed.

[0029] In this embodiment, the document data DD is stored in the server 10, but the document data DD may be stored, for example, in the user terminal 20, or in a terminal other than the server 10 and the user terminal 20 connected to the network N.

[0030] The processing unit 103 includes one or more CPUs. The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0031] By executing the program PG1, the processing device 103 functions as a data acquisition unit 110, a first display control unit 111, a second display control unit 112, a line of sight acquisition unit 113, a position determination unit 114, a selection unit 115, a search unit 116, and a priority determination unit 117. At least some of these functions may be configured by circuits such as a DSP, an ASIC, a PLD, and an FPGA. Details of these functional units will be described later.

[0032] A-4. Details of Processing Devices 103 and 208 Next, the details of each functional unit realized by the processing device 103 of the server 10 and the processing device 209 of the user terminal 20 will be described.

[0033] The virtual space access unit 211 of the user terminal 20 controls access to the virtual space VS. For example, when the user U starts an application for accessing the virtual space VS, the virtual space access unit 211 transmits a transmission request for the virtual space data VD to the server 10. The virtual space access unit 211 causes the display device 201 to display an image showing the virtual space VS based on the virtual space data VD received from the server 10. The virtual space access unit 211 also causes the speaker 205 to output a sound in the virtual space VS based on the virtual space data VD received from the server 10. The virtual space access unit 211 also accepts an input to the input device 202 (for example, an input instructing the movement of the avatar A), generates a transmission request for new virtual space data VD based on the contents of the input, and transmits it to the server 10.

[0034] The gaze detection unit 212 detects the gaze point of the user U. The gaze detection unit 212 acquires an image of the user U's eyes captured by the camera 206 of the user terminal 20, for example, and detects the direction of the user U's gaze. The gaze detection unit 212 may, for example, calibrate the gaze direction of the user U and the image of the user U's eyes in advance. The calibration is performed, for example, by displaying an object on the display device 201, instructing the user U to follow the object with his / her eyes, and capturing an image of the user U's eyes with the camera 206. By analyzing the relationship between the image thus obtained and the display position of the object on the display device 201, it is possible to associate the image of the user U's eyes with the location (gazing point) of the display device 201 where the user U is directing his / her gaze. The location (gazing point) of the display device 201 where the user U is directing his / her gaze is indicated, for example, by display coordinates described later.

[0035] Furthermore, the gaze detection unit 212 transmits the detected gaze point of the user U to the server 10. If the coordinates of the image showing the virtual space VS received from the server 10 are defined as image coordinates, and the coordinates of the display surface of the display device 201 are defined as display coordinates, the correspondence between the image coordinates and the display coordinates is known. Therefore, the gaze detection unit 212 may convert the gaze point of the user U indicated by the display coordinates, for example, into image coordinates and then transmit it to the server 10. In other words, the gaze detection unit 212 may detect which position in the virtual space VS the user U is looking at, and transmit position information indicating that position to the server 10.

[0036] The data acquisition unit 110 of the server 10 acquires document data DD of a document DC to be displayed in the virtual space VS. The data acquisition unit 110 accepts a designation of a document DC from, for example, a user U, and acquires document data DD corresponding to the document DC. If the document data DD is stored in the storage device 102 of the server 10, the data acquisition unit 110 reads the document data DD from the storage device 102. If the document data DD is stored in a terminal other than the server 10, the data acquisition unit 110 receives the document data DD from the terminal via, for example, a network N.

[0037] The first display control unit 111 displays a main display page image G1 showing at least one page of the document DC in a first region R1 of the virtual space VS. The second display control unit 112 displays at least one thumbnail image G2 including a page (hereinafter sometimes referred to as a "page waiting to be displayed") other than the at least one page displayed as the main display page image G1 of the document DC in a second region R2 of the virtual space VS.

[0038] FIG. 5 is a diagram showing a schematic arrangement of avatars A-1 to A-3 in a virtual space VS. FIG. 5 is a diagram showing the avatars A-1 to A-3 and the screen SC as viewed from above. In the example of FIG. 5, the avatars A-1 to A-3 are lined up horizontally facing the virtual screen SC. The first display control unit 111 and the second display control unit 112 display a document DC on the screen SC based on the document data DD. It is assumed that the avatars A-1 to A-3 are holding a discussion (a meeting, etc.) while looking at the document DC displayed on the screen SC. The avatar A-1 serves as the facilitator in the discussion.

[0039] Fig. 6 is a diagram showing a schematic view of the display of the screen SC. Fig. 6 is a diagram showing the screen SC as viewed from the direction of the arrow B in Fig. 5 (behind the avatars A-1 to A-3). The screen SC has a first area R1 and a second area R2. The first area R1 is an area surrounded by a dashed line, and the second area R2 is an area surrounded by a dotted line. In the example of Fig. 6, the second area R2 includes the first area R1.

[0040] In the first region R1, a main display page image G1 showing at least one page of the document DC is displayed. In the example of FIG. 6, the main display page image G1 displays one page of the document DC. The page displayed as the main display page image G1 is, for example, a page showing the content currently being discussed in a discussion. The main display page image G1 is displayed at a size (resolution) that allows the avatars A-1 to A-3 to grasp the content.

[0041] In the second region R2, a plurality of thumbnail images G2 including pages (pages waiting to be displayed) other than the page displayed as the main display page image G1 of the document DC are displayed. When the user U performs a predetermined operation on any of the thumbnail images G2, the page corresponding to the thumbnail image G2 is displayed as the main display page image G1. The predetermined operation is arbitrary, but for example, the user U may hold an input device (e.g., a smartphone, etc.) capable of emitting an operation light (ray), apply the operation light to a desired thumbnail image G2, and tap to switch the main display page image G1. In addition, for example, the user U may stare at the thumbnail image G2 of a desired page for a predetermined period of time or more, so that the main display page image G1 is switched to the desired page. For example, the main display page image G1 may be switched only by the facilitator, or may be switched by discussion participants other than the facilitator.

[0042] In the example of FIG. 6, ten thumbnail images G2 are arranged in a circle in the second region R2 so as to surround the main display page image G1. Unlike the main display page image G1, the thumbnail images G2 do not need to allow the avatars A-1 to A-3 to grasp the contents of the pages waiting to be displayed in detail. For example, it is sufficient for the avatars A-1 to A-3 to grasp which page of the document DC each thumbnail image G2 corresponds to (for example, page number). The pages waiting to be displayed displayed as the thumbnail image G2 may include the page displayed as the main display page image G1. In addition, the pages waiting to be displayed displayed as the thumbnail image G2 may change in conjunction with the page displayed as the main display page image G1. For example, it is assumed that the pages waiting to be displayed displayed as the thumbnail image G2 are ten pages including those before and after the main display page image G1. In this case, if the page displayed as the main display page image G1 is page P6, thumbnail images G2 of pages P1 to P10 may be displayed, and if the page displayed as the main display page image G1 is page P7, thumbnail images G2 of pages P2 to P11 may be displayed, etc. A plurality of thumbnail images G2 are not necessarily displayed. For example, if the document DC has two pages, one thumbnail representing a page other than the page displayed as the main display page image G1 is displayed in the second region R2.

[0043] Here, the position of the second area R2 is determined based on the gaze points of the participants in the discussion. In addition, in FIG. 6, ten thumbnail images G2 are illustrated in the second area R2, but if the number of pages in the document DC exceeds ten, it is not possible to display all the pages as thumbnail images G2. It is necessary to select the pages to be displayed in some way. Below, the details of [1] a method for determining the position of the second area R2 and [2] a method for selecting the thumbnail image G2 to be displayed in the second area R2 will be described.

[0044] [1] Method for determining the position of the second region R2 As described above, the display position of the second region R2 is determined based on the gaze points of the participants of the discussion. More specifically, the gaze acquisition unit 113 shown in FIG. 3 acquires the gaze point of at least one of the multiple users U. A target person from whom the gaze acquisition unit 113 acquires a gaze point is referred to as a "gaze acquisition target." The gaze acquisition unit 113 acquires the gaze point of a specific one of the multiple users U. Specifically, for example, when a facilitator is set in a discussion using the document DC, the gaze acquisition unit 113 may acquire the gaze point of the facilitator. As described above, in this embodiment, the user U-1 is the facilitator. Therefore, the gaze acquisition unit 113 may acquire the gaze point of the user U-1.

[0045] Furthermore, when there is a presenter making a presentation using the document DC, the line of sight acquisition unit 113 may acquire the gaze point of the presenter. Furthermore, when there is a user U who needs visual consideration among the multiple users U, the line of sight acquisition unit 113 may acquire the gaze point of the user U who needs visual consideration.

[0046] Furthermore, the gaze acquiring unit 113 may acquire the gaze point of each of the multiple users U (all users U). Alternatively, the gaze acquiring unit 113 may acquire the gaze point of at least one user U having a specific attribute among the multiple users U. The specific attribute may be, for example, a co-presenter when multiple people make a presentation using a document DC, a co-creator of a document DC when multiple people jointly create the document DC, a user U who is more closely related to the topic of the current discussion than other users U, or a user U who requires more visual consideration than other users U.

[0047] The gaze acquisition unit 113 acquires, for example, a gaze point of the user U detected by the gaze detection unit 212 of the user terminal 20 held by the gaze acquisition target person. Alternatively, the gaze acquisition unit 113 may acquire an image of the eyes of the gaze acquisition target person captured by the camera 206 of the user terminal 20 held by the gaze acquisition target person from the user terminal 20. In this case, the gaze acquisition unit 113 detects the gaze point of the user U based on the image of the eyes of the gaze acquisition target person.

[0048] The position determination unit 114 determines the position of the second region R2 based on the gaze point of the gaze acquisition target. When the gaze acquisition target is one specific person, the position determination unit 114 determines the position of the second region R2 so as to include the gaze point of the specific person. For example, FIG. 6 illustrates a case where the specific person is a user U-1 who is a facilitator. The gaze point M-1 of the user U-1 (illustrated as the gaze point of the avatar A-1 in FIG. 6) is on the main display page image G1 of the first region R1. The position determination unit 114 sets the second region R2 so as to surround the first region R1 including the gaze point M-1. The second display control unit 112 displays the thumbnail image G2 in a position of the second region R2 that does not overlap with the first region R1.

[0049] The arrangement of the thumbnail images G2 in the second region R2 is arbitrary. For example, the thumbnail image G2 of the first page, the thumbnail image G2 of the second page, etc. may be displayed in order from the end of the second region R2. Also, the thumbnail image G2 of a page with high importance may be displayed at a position with high visibility to the user U-1 corresponding to the facilitator, for example. The position with high visibility is, for example, a position close to the avatar A-1 corresponding to the user U-1. Also, the position with high visibility may be determined based on the display characteristics of the user terminal 20, for example.

[0050] Furthermore, a page with high importance may be, for example, a page before or after the page displayed as the main display page image G1, or a page with a high heading level. Numbers 1 to 10 are displayed on the thumbnail image G2 in Fig. 6. These numbers indicate the importance of the thumbnail image G2. The thumbnail image G2 with number 1, which has the highest importance, is placed in a position closest to the avatar A-1. Thumbnail images G2 with numbers 2 and 3 are placed to the left and right of thumbnail image G2 with number 1, and the further away from the avatar A-1 the thumbnail image G2 with lower importance (larger number) is displayed.

[0051] Fig. 7 is a diagram showing another setting example of the second region R2. In Fig. 7, the second region R2 is set to an area that includes the gaze point M-1 and is closer to the avatar A-1 than in Fig. 6. By setting the second region R2 to an area closer to the avatar A-1, the user U-1 can easily view the display of the thumbnail image G2, and can easily find the desired thumbnail image G2.

[0052] Fig. 8 is a diagram showing another example of setting the second region R2. In Fig. 6 and Fig. 7, the gaze point M-1 of the avatar A-1 overlaps with the main display page image G1. In Fig. 8, the gaze point M-2 of the avatar A-1 is in a position that does not overlap with the main display page image G1, and the second region R2 is also set in a position that does not overlap with the first region R1.

[0053] In this way, by determining the position of the second region R2 based on the facilitator's gaze point, the facilitator can increase the visibility of the thumbnail image G2. If the position of the second region R2 changes frequently following changes in the facilitator's gaze point, it may cause other users U to feel uncomfortable. Therefore, the position of the second region R2 may be fixed by the facilitator's instruction (for example, any action such as blinking a predetermined number of times). Also, for example, if the facilitator's gaze point remains in one position for a predetermined period of time or more, the position of the second region R2 may be changed.

[0054] Also, when the gaze acquisition targets are all users U, the gaze acquisition unit 113 acquires the gaze points of each of the multiple users U. The position determination unit 114 determines the position of the second region R2 so as to include a concentration point where the gaze points of each of the multiple users U gather in the virtual space VS. The concentration point may be, for example, the center of gravity of the gaze points of each user U, or may be a point where the largest number of users U are gazing. In this embodiment, the number of users U is three, but in a meeting with several dozen or more participants, for example, it is also effective to set the second region R2 based on the concentration point. In this case, the second display control unit 112 displays at least one thumbnail image G2 in a position of the second region R2 that does not overlap with the first region R1.

[0055] Furthermore, when the gaze acquisition target is at least one user U having a specific attribute among a plurality of users U, the second region R2 is determined as follows. When there is one user U having a specific attribute: the position determination unit 114 determines the position of the second region R2 so as to include the gaze point of the one user U having the specific attribute. When there are multiple users U with a specific attribute: the position determination unit 114 determines the position of the second region R2 so as to include a concentration point where the gaze points of multiple users U with a specific attribute gather in the virtual space VS.

[0056] [2] How to select thumbnail image G2 to be displayed in second area R2 Next, a method for selecting thumbnail images G2 to be displayed in the second area R2 will be described. If thumbnail images G2 of all pages of document DC can be displayed in the second area R2 at an appropriate size, thumbnail images G2 of all pages can be displayed. On the other hand, if the size of the second area R2 is small compared to the number of pages of document DC (if thumbnail images G2 cannot be displayed at an appropriate size), thumbnail images G2 of all pages of document DC cannot be displayed. Also, for example, if document DC has a large number of pages, even if the size of the second area R2 is sufficient, it may be difficult to select a desired page, so it may be better to narrow down the pages on which thumbnail images G2 are displayed.

[0057] 3 selects a page to be displayed as thumbnail image G2 from among the pages waiting to be displayed. The second display control unit 112 displays the thumbnail image G2 of the page selected by the selection unit 115.

[0058] The selection unit 115, for example, extracts pages with a high heading level and displays them as thumbnail images G2. As shown in FIG. 4, the total number of pages in the document DC is 14, with 2 pages corresponding to main headings, 4 pages corresponding to sub-headings, and 6 pages corresponding to sub-headings. For example, when it is desired to reduce the number of thumbnail images G2 to 8 or less, the selection unit 115 displays, for example, a total of 6 pages, including pages corresponding to main headings and pages corresponding to sub-headings, as thumbnail images G2. Also, when it is desired to reduce the number of thumbnail images G2 to 4 or less, the selection unit 115 displays, for example, 2 pages corresponding to main headings as thumbnail images G2. This allows the user U to grasp the overview of the entire document DC from a limited number of thumbnail images G2.

[0059] Fig. 9 is a diagram showing an example of the display mode of the second region R2. In Fig. 9, as in Fig. 8, the gaze point M-2 of the avatar A-1 is located at a position that does not overlap with the main display page image G1. In the second region R2, thumbnail images G2 of pages P3 and P10 corresponding to main headings and thumbnail images G2 of pages P4, P7, P11, and P12 corresponding to subheadings are displayed.

[0060] In Fig. 9, these thumbnail images G2 are displayed hierarchically according to heading level. That is, the thumbnail image G2 of page P3, which is a main heading, is connected to the thumbnail images G2 of pages P4 and P7, which are sub-headings below it, by a line indicating the hierarchy. Also, the thumbnail image G2 of page P10, which is a main heading, is connected to the thumbnail images G2 of pages P11 and P12, which are sub-headings below it, by a line indicating the hierarchy. In Fig. 9, pages at higher heading levels (main headings) are displayed so as to be located closer to avatar A-1.

[0061] In this way, the selection unit 115 selects pages that are equal to or smaller than the maximum number of thumbnail images G2 that can be displayed in the second area R2, starting from the page with the highest heading level. Then, the second display control unit 112 displays thumbnail images G2 corresponding to multiple pages in the document DC in a hierarchical manner based on the heading level. In this way, the user U can easily understand the relationship between the pages in the document DC. Note that, if the second area R2 is sufficiently large, all pages may be displayed in a hierarchical manner based on the heading level without selection by the selection unit 115.

[0062] As another example, the selection unit 115 may select pages by extracting pages in the document DC for example every N pages (N is an integer equal to or greater than 2). The selection unit 115 may also select pages by extracting pages randomly while maintaining the context within the document DC.

[0063] Also, the pages to be displayed as thumbnail images G2 may be selected based on whether or not they contain a predetermined keyword. That is, pages containing a predetermined search string (keyword) may be searched for within the document DC, and thumbnail images G2 of the searched pages may be displayed in the second area R2. Such a search may be used not only when the number of thumbnail images G2 to be displayed is limited, but also when only pages containing a predetermined keyword are displayed for discussion or when searching for a predetermined page.

[0064] 3 accepts input of a search string from at least one of a plurality of users U, and searches for a page including the search string from document DC. The search unit 116 accepts input of the search string, for example, through input to a user interface superimposed and displayed on the virtual space VS, or through speech (voice input) by user U. Furthermore, if there is no page including the search string of the search string, the search unit 116 displays a result such as "no match."

[0065] When multiple pages containing the search string are found, the priority determination unit 117 determines the priority of each of the multiple pages containing the search string based on at least one of the number of times the search string appears on each of the multiple pages containing the search string, the position of the search string, the font attributes of the search string, or the heading level of the page containing the search string.

[0066] For example, the priority determination unit 117 assigns a higher priority to a page in which the search string appears more frequently. The priority determination unit 117 also assigns a higher priority to a page in which the search string is used in the title or a heading within the page. The priority determination unit 117 also assigns a higher priority to a page in which a font attribute different from that of other strings is used for the search string (use of bold, use of a conspicuous color, use of a large font point, etc.). The priority determination unit 117 also assigns a higher priority to a page with a higher heading level among pages that include the search string.

[0067] The second display control unit 112 displays thumbnail images G2 of multiple pages including the search string based on the priority. For example, when thumbnail images G2 of pages including the search string cannot be displayed in the second area R2, the second display control unit 112 displays thumbnail images G2 of pages in order of priority in the second area R2. Also, the second display control unit 112 displays thumbnail images G2 of pages in order of priority in a position with high visibility to the user U (especially user U-1 who is the facilitator).

[0068] In this way, when the document DC has a large number of pages, it is possible to search for and display pages that contain a specific keyword. The searched pages are displayed in order of priority, so that the user U can efficiently search for a desired page.

[0069] A-5. Flowchart 10 is a flowchart showing the operation of the processing device 103. The processing device 103 waits until it receives a designation of a document DC to be displayed in the virtual space VS from the user U (step S100: NO). When it receives a designation of a document DC (step S100: YES), the processing device 103 functions as the data acquisition unit 110 and acquires document data DD corresponding to the designated document DC (step S102).

[0070] The processing device 103 functions as a gaze acquisition unit 113 and acquires the gaze point of a gaze acquisition target person (step S104). The gaze acquisition target person may be a facilitator or may be all of the multiple users U. The processing device 103 functions as a position determination unit 114 and determines the position of the second region R2 in the virtual space VS based on the gaze point (step S106).

[0071] The processing device 103 functions as the first display control unit 111 and displays the main display page image G1 in the first region R1 (step S108). In many cases, the initial main display page image G1 is the cover (first page) of the document DC. The processing device 103 also functions as the second display control unit 112 and displays the thumbnail image G2 in the second region R2 (step S110). The page displayed as the thumbnail image G2 may be a page selected by the selection unit 115 or a page searched for by the search unit 116.

[0072] When the processing device 103 receives a switching operation of the main display page image G1 from the user U (step S112), it functions as the first display control unit 111 and switches the display of the main display page image G1. In addition, accompanying the switching of the display of the main display page image G1, the processing device 103 functions as the second display control unit 112 and also switches the display of the thumbnail image G2 as necessary (step S114).

[0073] Until the user U instructs the end of displaying the document DC (step S116: NO), the processing device 103 returns to step S104 and repeats the subsequent processes. When the user U instructs the end of displaying the document DC (step S116: YES), the processing device 103 ends the process of this flowchart.

[0074] A-6. Summary of embodiments As described above, the server 10 according to the embodiment determines the position of the second region R2 in which the thumbnail image G2 of the document DC is displayed based on the gaze point of at least one of the multiple users U who are viewing the document DC. Therefore, the thumbnail image G2 can be displayed at an appropriate position in the virtual space VS, improving the convenience of switching the page to be displayed as the main display page image G1. In addition, the convenience of the user U in understanding the contents of the entire document DC using the thumbnail image G2 is improved.

[0075] The server 10 also determines the position of the second region R2 based on the gaze point of a specific one of the multiple users U, such as a facilitator or a presenter. The facilitator advances the discussion using the document DC, so by determining the position of the second region R2 based on the gaze point of the facilitator, the discussion can proceed smoothly. The presenter gives a presentation using the document DC, so by determining the position of the second region R2 based on the gaze point of the presenter, the presentation can proceed smoothly.

[0076] Furthermore, the server 10 determines the position of the second region R2 based on a convergence point of the gaze points of multiple users U participating in the discussion. When a large number of users U participate in the discussion, the discussion can be smoothly progressed by determining the position of the second region R2 based on the gaze points.

[0077] The server 10 also searches the document DC for pages that include the search string, and determines the priority of multiple pages that include the search string. In the second area R2, thumbnail images G2 of pages that include the search string are displayed based on the priority. Therefore, the page that displays the thumbnail image G2 can be selected based on the content of the page (whether or not the page includes the search string), improving convenience.

[0078] Furthermore, the server 10 displays thumbnails corresponding to each page in a hierarchical structure based on the heading level of each page of the document DC, thereby enabling the user U to intuitively grasp the relationship between multiple pages.

[0079] Furthermore, the server 10 selects a page to be displayed as the thumbnail image G2 from among the pages waiting to be displayed, and displays the thumbnail image G2 of the selected page. Therefore, for example, when the document DC has a large number of pages, the number of thumbnail images G2 to be displayed can be kept to an appropriate number.

[0080] (B:Other) (1) In the above-described embodiment, ROM and RAM, etc. are given as examples of storage devices 102 and 208, but storage devices 102 and 208 may also be a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disc), a smart card, a flash memory device (e.g., a card, a stick, a key drive), a CD-ROM (Compact Disc-ROM), a register, a removable disk, a hard disk, a floppy (registered trademark) disk, a magnetic strip, a database, a server, or any other suitable storage medium.

[0081] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0082] (3) In the above-described embodiment, the input / output information, etc. may be stored in a specific location (e.g., memory) or may be managed using a management table. The input / output information, etc. may be overwritten, updated, or added. The output information, etc. may be deleted. The input information, etc. may be transmitted to another device.

[0083] (4) In the above-described embodiments, the determination may be made based on a value represented by 1 bit (0 or 1), a Boolean (true or false) value, or a numerical comparison (e.g., comparison with a predetermined value).

[0084] (5) The order of the process steps, sequences, flow charts, etc. illustrated in the above-described embodiments may be changed without causing any inconsistency. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0085] (6) Each function illustrated in Figures 2 and 3 is realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically combined, or may be realized by using two or more devices that are physically or logically separated and directly or indirectly connected (for example, by using wires, wirelessly, etc.). The functional block may be realized by combining the one device or the multiple devices with software.

[0086] (7) The programs exemplified in the above-described embodiments should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0087] Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave, etc.), then these wired and / or wireless technologies are included within the definition of transmission media.

[0088] (8) In each of the above embodiments, the terms “system” and “network” are used interchangeably.

[0089] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0090] (10) In the above-described embodiments, the portable device may be a mobile station (MS). A mobile station may be referred to by those skilled in the art as a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other suitable term. In this disclosure, the terms "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.

[0091] (11) In the above-mentioned embodiments, the terms "connected" and "coupled" or any variation thereof refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "accessed". As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and light (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0092] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0093] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, search, inquiry (e.g., searching in a table, database, or other data structure), ascertaining, and the like. Also, "determining" and "determining" may include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), and the like. In addition, "judgment" and "decision" can include considering resolving, selecting, choosing, establishing, comparing, etc., to be a "judgment" or "decision." In other words, "judgment" and "decision" can include considering some action to be a "judgment" or "decision." In addition, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0094] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or" used in this disclosure is not intended to be an exclusive or.

[0095] (15) In this disclosure, where articles have been added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural.

[0096] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0097] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched according to execution. In addition, notification of specific information (e.g., notification that "X is the case") is not limited to being done explicitly, but may be done implicitly (e.g., not notifying the specific information). [Explanation of symbols]

[0098] 1...system, 10...server, 20...user terminal, 110...data acquisition unit, 111...first display control unit, 112...second display control unit, 113...gaze acquisition unit, 114...position determination unit, 115...selection unit, 116...search unit, 117...priority determination unit, 211...virtual space access unit, 212...gaze detection unit, A (A-1 to A-3)...avatar, DC...document, DD...document data, G1...main display page image, G2...thumbnail image, M-1, M-2...gazing point, N...network, R1...first area, R2...second area, SC...screen, SV...virtual space, U (U-1 to U-13)...user, VS...virtual space.

Claims

1. A display control device that displays a document including a plurality of pages in a virtual space accessed by a plurality of users, comprising: a first display control unit that displays a main display page image indicating at least one page of the document in a first area of ​​the virtual space; a second display control unit that displays at least one thumbnail image including a page other than the at least one page of the document in a second area of ​​the virtual space; a gaze acquisition unit for acquiring a gaze point of at least one of the plurality of users; and a position determination unit that determines a position of the second area based on the gaze point. Display control device.

2. The line-of-sight acquisition unit acquires a gaze point of a specific one of the plurality of users, The position determination unit determines a position of the second region so as to include a gaze point of the specific person, the second display control unit displays the at least one thumbnail image in a position in the second area that does not overlap with the first area. The display control device according to claim 1.

3. The line-of-sight acquisition unit acquires a gaze point of each of the plurality of users, The position determination unit determines a position of the second region so as to include a convergence point where gaze points of the respective users gather in the virtual space; the second display control unit displays the at least one thumbnail image in a position in the second area that does not overlap with the first area. The display control device according to claim 1.

4. a search unit that receives an input of a search string from at least one of the plurality of users and searches the document for a page including the search string; a priority determination unit that, when a plurality of pages including the search string are found, determines a priority of each of the plurality of pages including the search string based on at least one of the number of occurrences of the search string in each of the plurality of pages including the search string, the position of the search string, a font attribute of the search string, or an index level of the page including the search string, the second display control unit displays the thumbnail images of a plurality of pages including the search character string based on the priority. The display control device according to claim 1.

5. Each of the plurality of pages has a heading level set, the second display control unit displays the thumbnail images corresponding to the plurality of pages in a hierarchical manner based on the heading level. The display control device according to claim 1.

6. the pages other than the at least one page are multiple, a selection unit that selects a page to be displayed as the thumbnail image from among pages other than the at least one page, the second display control unit displays a thumbnail image of the page selected by the selection unit. The display control device according to claim 1.

Images