Information processing system and program

Abstract

To provide an information processing system and a program capable of displaying a participant to which a user pays attention in a larger size than when participants at a plurality of bases are equally displayed in a plurality of display areas having the same display area in one display device.SOLUTION: CPU31 displays the video images of the participants of the WEB conference at the plurality of other sites in a plurality of display regions having different display areas of the display apparatus 201. In addition, the CPU31 detects the directions of the lines of sight of the users of the display devices 201 using the sensors 204 of the cameras 202. Then, the video images of the participants of the site displayed in the first display area 301 and the second display area 302 are switched so that the participant of the site displayed in the first display area 301 or the second display area 302 to which the line of sight of the user is directed among the participants of the WEB conference displayed in the first display area 301 and the second display area 302 is displayed in the largest size.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system and a program. [Background technology]

[0002] Patent Document 1 discloses a video conferencing system in which an image wall is formed that simultaneously displays all participants at remote locations within a space reproduced on the image wall formed by a video screen, and audio is spatially distributed so that the audio emitted by each remote participant at each site corresponds to the image of the participant projected on the video screen at that site.

[0003] Patent document 2 discloses a communication device equipped with a transmission control unit that controls the transmission of an image captured by a first camera positioned to capture the gaze of a user when the user gazes at a first display area to a first external communication device, and the transmission of an image captured by a second camera positioned so as not to capture the gaze of a user gazing at the first display area or the second display area to a second external communication device.

[0004] Patent document 3 discloses a communication device that includes an imaging unit that captures images within a base, a recording unit that records the point of speech indicating the speaker's location together with the time in a memory unit when a speech is made within the base, a range determination unit that determines a shooting range that includes the recorded multiple speech points when multiple speech points within the base are recorded within a specified period of time, and a transmission unit that transmits video of the determined shooting range to another communication device located at another base. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-081861 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-178355 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-034312 Summary of the Invention [Problem to be solved by the invention]

[0006] In a web conferencing system, multiple participants at multiple locations may connect to the system and hold a conference with the participants who appear on camera. When a web conference is held with multiple locations, the display area displayed on the user's display device corresponds to at least the number of locations. Therefore, the more locations there are, the smaller the display area per location on the user's display device becomes, and the display size of the participants at each location also becomes smaller. As a result, even the participants that the user is paying attention to will remain displayed in a small size.

[0007] The object of the present invention is to provide an information processing system and program that can display participants that a user is paying attention to larger than when participants from multiple locations are displayed evenly in multiple display areas with the same display area within a single display device. [Means for solving the problem]

[0008] The information processing system of the first aspect comprises a processor, a display device, a camera that captures an image of a user viewing the display device, and a sensor that detects the direction of the user's gaze, wherein the processor displays images of participants at multiple other locations in multiple display areas of the display device with different display areas, detects the direction of the user's gaze using the sensor, and switches the images of the participants at the location displayed in the multiple display areas so that the participant at the location displayed in the display area to which the user's gaze is directed is displayed largest among the participants displayed in the multiple display areas.

[0009] In a second aspect of the information processing system, in the information processing system of the first aspect, when there are multiple users and the sensor detects the direction of gaze of more than a predetermined number of the users, the processor switches the images of the participants at the locations displayed in the multiple display areas so that the participant at the location displayed in the display area to which the gazes of more than the predetermined number of users are directed is displayed largest.

[0010] A third aspect of the information processing system is the information processing system of the second aspect, wherein when the processor detects the direction of gaze of more than half of the users using the sensor, it switches the images of the participants at the location displayed in the multiple display areas so that the participant at the location displayed in the display area to which the gaze of more than half of the users is directed is displayed largest.

[0011] In a fourth aspect of the information processing system, in the information processing system of the first aspect, when the user is displayed largest on the display device of a participant at another location, the processor displays the display area displaying the participant at the other location so that it is recognizable that the user is displayed largest on the display device of the participant at the other location.

[0012] In the information processing system of the fifth aspect, in the information processing system of the fourth aspect, a symbol is added to the display area displaying the participants at the other location to indicate that the user is displayed largest on the display device of the participant at the other location.

[0013] In a sixth aspect of the information processing system, in the fourth aspect of the information processing system, the processor displays a display frame in the display area displaying the participants at the other location, indicating that the user is displayed largest on the display device of the participant at the other location.

[0014] An information processing system of a seventh aspect is the information processing system of the first aspect, wherein the multiple display areas have a first display area and a second display area having a larger display area than the first display area, and the processor detects the direction of the user's gaze using the sensor and switches the images of the location participants displayed in the multiple display areas so that the location participants displayed in the display area toward which the gaze is directed are displayed in the second display area, and the location participants displayed in the display area toward which the gaze is not directed are displayed in the first display area.

[0015] An eighth aspect of the information processing system is the seventh aspect of the information processing system, in which, after the sensor detects the direction of gaze in the second display area, if the sensor detects the direction of gaze in the first display area continuously for more than a predetermined time, the images of the participants at the locations displayed in the multiple display areas are switched so that the participant at the location displayed in the first display area to which the gaze has been directed continuously for more than the predetermined time is displayed largest.

[0016] In a ninth aspect of the information processing system, in the information processing system of the first aspect, when the processor switches the images of the participants at the locations displayed in the multiple display areas so that the participant at the location displayed in the display area at which the user's gaze is directed is displayed largest, the processor displays the images in a way that allows the user to recognize the switch in the display area at which the user's gaze is directed.

[0017] In a tenth aspect of the information processing system, in the ninth aspect, the processor controls the display area that is being gazed at to be gradually larger and the display area that is not being gazed at to be gradually smaller when switching the images of the participants at the location displayed in the multiple display areas, so that the participant at the location displayed in the display area that is being gazed at is displayed largest.

[0018] An information processing system of an eleventh aspect is the information processing system of the first aspect, further comprising a plurality of speakers, and the processor outputs the audio of participants at other locations from speakers positioned corresponding to display areas that display images of participants at other locations.

[0019] The program of the 12th aspect causes a computer to execute the steps of displaying images of participants at multiple other locations in multiple display areas of the display device, each having a different display area; and detecting the direction of the gaze of a user viewing the display device using a sensor, and switching the images of the participants at the locations displayed in the multiple display areas so that the participant at the location displayed in the display area to which the gaze is directed is displayed largest among the participants displayed in the multiple display areas. [Effects of the Invention]

[0020] According to the information processing system of the first aspect, it is possible to display participants at multiple locations in a larger size than when participants are displayed evenly in multiple display areas with the same display area within a single display device.

[0021] According to the information processing system of the second aspect, it is possible to enlarge a participant who is being watched by a preset number of users or more out of a plurality of users.

[0022] According to the information processing system of the third aspect, it is possible to enlarge a participant who is being watched by more than half of the users out of a plurality of users.

[0023] According to the information processing system of the fourth aspect, it is possible to make it easier for the user to recognize which of the multiple display areas a participant displayed in is paying attention to.

[0024] According to the information processing system of the fifth aspect, it is possible to make it easier for the user to recognize which of the multiple display areas a participant displayed in is paying attention to.

[0025] According to the information processing system of the sixth aspect, it is possible to make it easier for the user to recognize which of the multiple display areas a participant displayed in is paying attention to.

[0026] According to the information processing system of the seventh aspect, it is possible to display participants that the user is paying attention to larger than when participants from multiple locations are displayed evenly in multiple display areas each with the same display area within a single display device.

[0027] According to the information processing system of the eighth aspect, it is possible to prevent the video of the participants from being changed multiple times within a few seconds.

[0028] According to the information processing system of the ninth aspect, it is possible to make it easier for participants to recognize that their video images have been switched.

[0029] According to the information processing system of the tenth aspect, it is possible to make it easier for participants to recognize that their video images have been switched.

[0030] According to the information processing system of the eleventh aspect, it is possible to make it easier to recognize in which of the multiple display areas a participant is speaking.

[0031] According to the program of the 12th aspect, it is possible to display participants at multiple locations in a larger size than when participants are displayed evenly in multiple display areas with the same display area within a single display device. [Brief explanation of the drawings]

[0032] [Figure 1] 1 is a diagram illustrating a system configuration of an information processing system according to an embodiment of the present disclosure. [Figure 2]FIG. 2 is a block diagram illustrating a hardware configuration of a terminal device according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a block diagram illustrating a hardware configuration of a server according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a diagram illustrating an example of a display device according to an embodiment of the present disclosure. [Figure 5] 10 is a flowchart illustrating an example of a flow of a display screen switching process of a display device according to an embodiment of the present disclosure. [Figure 6] 10A and 10B are diagrams for explaining a display screen switching process of a display device according to an embodiment of the present disclosure. [Figure 7] 10A and 10B are diagrams illustrating an example of a display screen switching process of a display device according to an embodiment of the present disclosure. [Figure 8] 10A and 10B are diagrams illustrating modified examples of the display screen of the display device according to the embodiment of the present disclosure. [Figure 9] 10A and 10B are diagrams illustrating an example of an operation of a speaker provided in a display device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0033] An example of an embodiment of the present disclosure will be described below with reference to the drawings. The same or equivalent components and parts in each drawing are designated by the same reference numerals. The dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0034] An example of an information processing system 10 according to this embodiment will be described with reference to FIG.

[0035] Fig. 1 is a diagram showing an example of a schematic configuration of an information processing system 10 according to this embodiment. As shown in Fig. 1, the information processing system 10 according to this embodiment includes a plurality of, for example, three terminal devices 20A, 20B, and 20C, and a server 30. The terminal devices 20A, 20B, and 20C and the server 30 are connected via a network 40. This network 40 may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like.

[0036] Terminal devices 20A, 20B, and 20C are personal computers (PCs) used by participants in the web conference. A display device 201A, a camera 202A, a speaker 203A, etc. are connected to terminal device 20A installed at the first location SA. A display device 201B, a camera 202B, a speaker 203B, etc. are connected to terminal device 20B installed at the second location SB. A display device 201C, a camera 202C, a speaker 203C, etc. are connected to terminal device 20C installed at the third location SC.

[0037] In this embodiment, terminal devices 20A, 20B, and 20C are each configured in the same way. For this reason, terminal devices 20A, 20B, and 20C may be collectively referred to as "terminal device 20." Furthermore, each of the components connected to terminal devices 20A, 20B, and 20C is configured in the same way. For this reason, display devices 201A, 201B, and 201C may be collectively referred to as "display device 201." Furthermore, cameras 202A, 202B, and 202C may be collectively referred to as "camera 202." Furthermore, speakers 203A, 203B, and 203C may be collectively referred to as "speaker 203."

[0038] The server 30 is a server computer that provides a web conference service in the information processing system 10. Here, the web conference service provided by the server 30 includes online meetings, online presentations, online seminars (webinars), and the like.

[0039] FIG. 2 is a block diagram showing the hardware configuration of the terminal device 20 according to this embodiment.

[0040] As shown in FIG. 2, the terminal device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage device 24 such as a hard disk drive, a communication interface (abbreviated as IF) 25, an input device 26, and an input / output interface (abbreviated as I / O) 27 for inputting and outputting data between each device, etc.

[0041] A display device 201, a camera 202, a speaker 203, and the like are connected to the I / O 27.

[0042] The CPU 21 is a central processing unit that executes various programs and controls each part. That is, the CPU 21 reads the programs from the ROM 22 or the storage device 24 and executes the programs using the RAM 23 as a work area. The CPU 21 controls each of the above components and performs various arithmetic processing in accordance with the programs recorded in the ROM 22 or the storage device 24. In this embodiment, the programs are stored in the ROM 22 or the storage device 24.

[0043] The ROM 22 stores various programs and various data. The RAM 23 serves as a working area and temporarily stores programs or data. The storage device 24 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0044] The communication IF 25 is connected to a network 40 and communicates with other devices such as other terminal devices 20 and a server 30 .

[0045] The input device 26 includes a pointing device such as a mouse and a keyboard, and is used to make various inputs.

[0046] The display device 201 is, for example, a liquid crystal display. The display device 201 displays various information under the control of the CPU 21. In this embodiment, the display device 201 employs a touch panel system and also functions as an input device.

[0047] Camera 202 is a user viewing display device 201 and captures participants in the web conference. Camera 202 is equipped with sensor 204 that detects the direction of the line of sight of the user of display device 201. Camera 202 also has built-in microphone 205 and functions as microphone 205. Microphone 205 acquires the voice of the user of display device 201. Note that microphone 205 may be provided separately from camera 202.

[0048] The speaker 203 outputs the remarks of other participants who are participating in the web conference using other terminal devices 20 connected via the network 40 .

[0049] FIG. 3 is a block diagram showing the hardware configuration of the server 30. As shown in FIG.

[0050] 3, the server 30 includes a CPU 31, a ROM 32, a RAM 33, a storage device 34, and a communication IF 35. Each component is connected to each other so as to be able to communicate with each other via a bus 36. The server 30 may also include an input unit such as a keyboard, a display device such as a display, and the like.

[0051] The CPU 31 is a central processing unit that executes various programs and controls each part. That is, the CPU 31 reads a program from the ROM 32 or the storage device 34 and executes the program using the RAM 33 as a work area. The CPU 31 controls each of the above components and performs various arithmetic processing according to the program recorded in the ROM 32 or the storage device 34. In this embodiment, the ROM 32 or the storage device 34 stores programs and various data.

[0052] The ROM 32 stores various programs and various data. The RAM 33 temporarily stores programs or data as a working area. The storage device 34 is configured with an HDD or SSD and stores various programs including an operating system and various data. The storage device 34 of the server 30 stores various data transmitted from the terminal device 20. In this embodiment, folders accessible by participants in the web conference are created in the storage device 34 of the server 30, and documents such as text files and image files used in the web conference transmitted from the terminal device 20 are stored. The storage device 34 of the server 30 also stores information required for the web conference, such as the date and time of the web conference, connection information such as a connection ID and password, and a list of participants. The server 30 may acquire such web conference information by inputting information entered by participants in the web conference using the terminal device 20.

[0053] The communication IF 35 is for communicating with other devices such as the terminal device 20, and uses standards such as the Internet, intranet, Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0054] Next, the display device 201 in this embodiment will be described using a display device 201A connected to the terminal device 20A at the first site SA.

[0055] 4, a display device 201A used by a user A is connected to a terminal device 20A installed at a first location SA. A camera 202A with a built-in microphone 205 and a speaker 203A are installed on the upper frame of the display device 201A.

[0056] 4, a first display area 301 of the display device 201A displays an image captured by a camera 202C installed at the third location SC, i.e., an image of participant C who is participating in the web conference from the third location SC. Also, a second display area 302 of the display device 201A displays an image captured by a camera 202B installed at the second location SB, i.e., an image of participant B who is participating in the web conference from the second location SB.

[0057] That is, when a web conference starts, the display device 201A displays a video of participant B at the second location SB and a video of participant C at the third location SC in the first display area 301 and the second display area 302, respectively, which have different display areas. The video of the participant at the location displayed in the first display area 301 and the video of the participant at the location displayed in the second display area 302 are switched depending on the direction of user A's gaze. In other words, the video of the participant at the location displayed in the first display area 301 and the second display area 302 are switched so that the participant at the location toward which user A's gaze is directed on the display device 201A is displayed largest. Here, the second display area 302 has a larger display area than the first display area 301 and is displayed as a full screen, for example. The first display area 301 is displayed as a wipe screen, superimposed on a corner of the second display area 302, as shown in FIG. 4, for example.

[0058] 5 is a flowchart showing an example of the flow of the screen switching process of the display device 201 by the server 30. In the following, the display device 201A connected to the terminal device 20A at the first location SA will be used for explanation.

[0059] As a prerequisite, the organizer of the web conference will reserve the conference and set the scheduled conference time, set up a folder to store the conference documents, and store the documents in that folder.

[0060] First, the CPU 31 of the server 30 receives a user ID from each terminal device 20 and performs personal authentication. Then, based on the web conference information stored in the storage device 34, the CPU 31 compares the scheduled conference time at which the authenticated individual will participate with the current time to identify the conference. Then, the CPU 31 establishes conference communication based on the connection information of the identified conference. Then, the CPU 31 displays videos of the web conference participants at multiple other locations in a first display area 301 or a second display area 302 of the display device 201, each having a different display area.

[0061] Specifically, the CPU 31 displays images of participant B at the second location SB and participant C at the third location SC in a first display area 301 or a second display area 302, each having a different display area, on a display device 201A connected to a terminal device 20A at the first location SA.

[0062] In this example, we will explain the case where there is one web conference participant A at the first location SA, one web conference participant B at the second location SB, and two web conference participants C at the third location SC.

[0063] In step S101, as shown in Fig. 6(A), CPU 31 detects the direction of user A's gaze using sensor 204. At this time, CPU 31 determines whether or not the direction of user A's gaze has been detected in an area other than first display area 301. If CPU 31 determines that the direction of user A's gaze has been detected in an area other than first display area 301, for example, in second display area 302, the process proceeds to the next step S102. On the other hand, if CPU 31 determines that the direction of user A's gaze has been detected in first display area 301, the process proceeds to the next step S105.

[0064] 6(B), in step S102, CPU 31 determines whether or not the direction of user A's gaze has been detected in first display area 301 by sensor 204. That is, CPU 31 first detects the direction of user A's gaze in an area other than first display area 301 by sensor 204, and then determines whether or not the direction of user A's gaze has been detected in first display area 301. If CPU 31 determines that the direction of user A's gaze has been detected in first display area 301, the process proceeds to next step S103. On the other hand, if CPU 31 determines that the direction of user A's gaze has not been detected in first display area 301, the process proceeds to next step S105 without switching the screen between the video of participant C at third location SC displayed in first display area 301 and the video of participant B at second location SB displayed in second display area 302.

[0065] In step S103, CPU 31 determines whether sensor 204 has detected the direction of user A's gaze continuously for a predetermined time or longer (e.g., three seconds or longer) in first display area 301. If CPU 31 determines that the direction of user A's gaze has been detected continuously for a predetermined time or longer (e.g., three seconds or longer) in first display area 301, the process proceeds to next step S104. On the other hand, if CPU 31 determines that the direction of user A's gaze has not been detected in first display area 301 for a predetermined time or longer (e.g., three seconds or longer), the process proceeds to next step S105 without switching the screen between the video of participant C at third location SC displayed in first display area 301 and the video of participant B at second location SB displayed in second display area 302.

[0066] 6(B) and 6(C), in step S104, when the sensor 204 detects the direction of user A's gaze continuously in the first display area 301 for a predetermined period of time or more (e.g., three seconds or more), the CPU 31 switches the screen between the video of participant C at the third location SC in the first display area 301 and the video of participant B at the second location SB in the second display area 302. That is, the CPU 31 switches the video of the participants at each location displayed in the first display area 301 and the second display area 302. In other words, the CPU 31 displays the video of participant B at the second location SB in the first display area 301 of the display device 201A, and displays the video of participant C at the third location SC in the second display area 302 of the display device 201A.

[0067] That is, CPU 31 displays the participant of the location displayed in first display area 301 or second display area 302, whichever is the location that user A is looking at, at the largest size among the participants of the web conference displayed in first display area 301 and second display area 302. In other words, CPU 31 displays the participant of the location displayed in the display area that user A is looking at in second display area 302, and displays the participant of the location displayed in the display area that user A is not looking at in first display area 301.

[0068] Then, when switching the images of the participants at the respective locations displayed in the first display area 301 and the second display area 302, for example, so that participant C at the location displayed in the first display area 301 at which user A is looking is displayed largest, the CPU 31 controls the display so that user A can recognize the switch from the first display area 301 at which user A is looking to the second display area 302.

[0069] 7, CPU 31 controls the display so that the image of participant C in first display area 301, which is directed by user A's gaze, is gradually enlarged and displayed in second display area 302. At the same time, CPU 31 controls the display so that the image of participant B in second display area 302, which is not directed by user A's gaze, is gradually reduced and displayed in first display area 301.

[0070] In step S105, the CPU 31 of the server 30 determines whether the conference has ended. If the CPU 31 determines that the conference has ended, the process ends. On the other hand, if the CPU 31 does not determine that the conference has ended, the process returns to step S101.

[0071] That is, after the CPU 31 detects the direction of the user's gaze in the second display area 302 other than the first display area 301 using the sensor 204, if the CPU 31 detects the direction of the user's gaze in the first display area 301 continuously for more than a preset time, the CPU 31 switches the images of the participants at the location displayed in the first display area 301 and the second display area 302 so that the participant at the location displayed in the first display area 301 at which the gaze is directed continuously for more than the preset time is displayed largest.

[0072] That is, CPU 31 of server 30 switches, for display device 201A installed at first location SA, between video transmitted from camera 202B at second location SB, which is the remote party in the conference, and video transmitted from camera 202C at third location SC. Similarly, for display device 201B installed at second location SB, it switches between video transmitted from camera 202A at first location SA, which is the remote party in the conference, and video transmitted from camera 202C at third location SC. Also, for display device 201C installed at third location SC, it switches between video transmitted from camera 202A at first location SA and video transmitted from camera 202B at second location SB.

[0073] In addition, when there are multiple users of the display device 201, if the sensor 204 detects the direction of gaze of more than a predetermined number of the multiple users, the CPU 31 switches the images of the participants at each location displayed in the first display area 301 and the second display area 302 so that the participant at the location displayed in the first display area 301 or the second display area 302 at which the gazes of more than the predetermined number of users are directed is displayed largest.

[0074] Specifically, when CPU 31 detects the direction of gaze of more than half (or more than 60%, or more than two-thirds, etc.) of multiple users of display device 201, CPU 31 switches the images of the participants of each location displayed in first display area 301 and second display area 302 so that the participant of the location displayed in the display area where the gazes of more than half (or more than 60%, or more than two-thirds, etc.) are directed is displayed largest. In other words, for example, if there are three users A of display device 201A, CPU 31 controls so that the participant of the location displayed in the display area where the gazes of two or more users are directed is displayed largest.

[0075] 8(A) and 8(B) are diagrams showing modified examples of the display screen of the display device 201 according to an embodiment of the present disclosure.

[0076] When a user is displayed largest on the display device 201 of a participant at another location, the CPU 31 of the server 30 displays the display area displaying the participant at the other location so that it is recognizable that the user is displayed largest on the display device 201 of the participant at the other location.

[0077] Specifically, as shown in Figure 8(A), when user A is displayed largest on the display device 201B of participant B at the second location SB, the CPU 31 of the server 30 displays, in the second display area 302 displaying participant B at the second location SB, an eye mark 402, for example, as a symbol indicating that user A is displayed largest on the display device 201B of participant B at the second location SB.

[0078] Similarly, when user A is displayed largest on the display device 201C of participant C at the third location SC, the CPU 31 of the server 30 controls the first display area 301 displaying participant C at the third location SC to display, for example, an eye mark 401 as a symbol indicating that user A is displayed largest on the display device 201C of participant C at the third location SC.

[0079] Furthermore, as shown in FIG. 8(B), when user A is displayed largest on the display device 201B of participant B at the second location SB, the CPU 31 of the server 30 may control the display so that a display frame 502 indicating that user A is displayed largest on the display device 201B of participant B at the second location SB is displayed around the second display area 302 displaying participant B at the second location SB.

[0080] Similarly, the CPU 31 of the server 30 may be configured to control the display so that, when user A is displayed largest on the display device 201C of participant C at the third location SC, a display frame 501 indicating that user A is displayed largest on the display device 201C of participant C at the third location SC is displayed around the first display area 301 displaying participant C at the third location SC.

[0081] 9 is a diagram illustrating an example of the operation of the speaker 203 according to an embodiment of the present disclosure. In this embodiment, a participant D is participating in the web conference at a fourth location SD in addition to the first location SA, the second location SB, and the third location SC.

[0082] In this embodiment, speakers 203A-1 and 203A-2 are installed on the right and left frames of the display device 201A, respectively. The CPU 31 of the server 30 controls the audio of the participants at the other locations to be output from the speaker 203A-1 and / or the speaker 203A-2 arranged near the display area displaying the video of the participants at each location.

[0083] Specifically, the CPU 31 of the server 30 outputs the voice of participant C, who is displayed in the first display area 301-1, only from the speaker 203A-1 arranged near the first display area 301-1. The CPU 31 also outputs the voice of participant D, who is displayed in the first display area 301-2, only from the speaker 203A-2 arranged near the first display area 301-2. The CPU 31 also outputs the voice of participant B, who is displayed in the second display area 302, from both the speaker 203A-1 and the speaker 203A-2.

[0084] Then, the CPU 31 of the server 30 controls the display so that the participant of the location displayed in the display area where the gaze of user A is directed by the sensor 204 is displayed largest, and also controls the display so that the voice of the participant of the location displayed in the display area where the gaze of user A is directed is output from both speakers 203A-1 and 203A-2.

[0085] Specifically, the CPU 31 of the server 30 controls the video of participant D at the location displayed in the first display area 301-2, which is the line of sight of the user A, to be gradually enlarged and displayed in the second display area 302, and also controls the audio of participant D output from the speaker 203A-2 to be gradually output from both speakers 203A-1 and 203A-2. At the same time, the CPU 31 controls the video of participant B in the second display area 302 to be gradually reduced in size and displayed in the first display area 301-2, and also controls the audio of participant B output from the speakers 203A-1 and 203A-2 to be gradually output only from the speaker 203A-2.

[0086] That is, the CPU 31 of the server 30 switches the video of the participants at each location in each display area of ​​the display device 201, and also switches the audio of the participants at each location.

[0087] The present invention is not limited to the above-described embodiment, and various modifications and applications are possible within the scope of the gist of the present invention.

[0088] For example, the number of locations described in the above-mentioned embodiment and the number of participants in the web conference at each location (or users of the display device 201 at each location) are merely examples and are not limited to these.

[0089] In the above-described embodiment, a case has been described in which one display device 201 is connected to the terminal device 20 at each base, but the present invention is not limited to this. Two display devices 201 may be connected to the terminal device 20 at each base, with one display device displaying video of the participants at each base and the other display device displaying documents such as text files and image files used in the web conference.

[0090] In the above embodiment, the program is pre-stored (installed) in a ROM or a storage device, but the present invention is not limited to this. The program may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via the network 40.

[0091] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0092] Furthermore, the operations of the processor in each of the above embodiments may be performed not only by a single processor but also by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments and may be changed as appropriate.

[0093] In addition, the term "system" in this embodiment includes both a system made up of multiple devices and a system made up of a single device.

[0094] The techniques of this disclosure are also applicable to programs and program products.

[0095] [Note] Preferred embodiments of the present disclosure will be described below.

[0096] (((1))) a processor; a display device; a camera that captures an image of a user viewing the display device; a sensor that detects the direction of the user's line of sight, The processor: Displaying images of participants at other locations in a plurality of display areas of the display device, each having a different display area; the sensor detects the direction of the user's line of sight, and switches the images of the participants at the locations displayed in the plurality of display areas so that the participant at the location displayed in the display area to which the user's line of sight is directed is displayed largest among the participants displayed in the plurality of display areas; Information processing system.

[0097] (((2))) The information processing system described in (((1)))), wherein when there are multiple users, if the sensor detects the direction of gaze of more than a predetermined number of the users, the processor switches the images of the participants at the locations displayed in the multiple display areas so that the participant at the location displayed in the display area to which the gazes of more than the predetermined number of users are directed is displayed largest.

[0098] (((3))) The information processing system described in (((2))), wherein when the sensor detects the direction of gaze of more than half of the users, the processor switches the images of the participants at the locations displayed in the multiple display areas so that the participant at the location displayed in the display area where the gaze of more than half of the users is directed is displayed largest.

[0099] (((4))) The information processing system described in (((1))) wherein, when the user is displayed largest on the display device of a participant at another location, the processor displays the display area displaying the participant at the other location so that it is recognizable that the user is displayed largest on the display device of the participant at the other location.

[0100] (((5))) The information processing system described in (((4)))), wherein the processor displays a symbol in the display area displaying the participants at the other location indicating that the user is displayed largest on the display device of the participant at the other location.

[0101] (((6))) The information processing system described in (((4)))), wherein the processor displays a display frame in the display area displaying the participants at the other location, indicating that the user is displayed largest on the display device of the participant at the other location.

[0102] (((7))) the plurality of display areas include a first display area and a second display area having a display area larger than that of the first display area, The processor: The information processing system described in (((1)))) detects the direction of the user's gaze using the sensor, and switches the images of the location participants displayed in the multiple display areas so that the location participant displayed in the display area where the gaze is directed is displayed in the second display area, and the location participant displayed in the display area where the gaze is not directed is displayed in the first display area.

[0103] (((8))) The information processing system described in (((7))), wherein, after the processor detects the direction of gaze in the second display area using the sensor, if the processor detects the direction of gaze in the first display area for more than a predetermined time continuously, the processor switches the images of the participants at the locations displayed in the multiple display areas so that the participant at the location displayed in the first display area to which the gaze has been directed for more than the predetermined time continuously is displayed largest.

[0104] (((9))) The processor: An information processing system as described in (((1))), in which, when switching images of participants at different locations displayed in multiple display areas so that the participant at the location displayed in the display area at which the user's gaze is directed is displayed largest, the display area at which the user's gaze is directed is displayed in a manner that allows the user to recognize the switching of the display area at which the user's gaze is directed.

[0105] (((10))) The processor: The information processing system described in (((9))) controls the display area that is being gazed at to be gradually larger and the display area that is not being gazed at to be gradually smaller when switching images of participants at locations displayed in the multiple display areas so that the participant at the location displayed in the display area that is being gazed at is displayed largest.

[0106] (((11))) further comprising a plurality of speakers; An information processing system described in any one of (((1))) to (((10))), wherein the processor outputs the audio of participants at other multiple locations from speakers positioned at positions corresponding to a display area displaying images of participants at the other multiple locations.

[0107] (((12))) displaying images of participants at other locations in a plurality of display areas of the display device, each having a different display area; a step of detecting a direction of gaze of a user viewing the display device by a sensor, and switching images of the participants at the respective locations displayed in the plurality of display areas so that the participant at the location displayed in the display area to which the gaze is directed is displayed largest among the participants displayed in the plurality of display areas; A program that causes a computer to execute the following.

[0108] The effects of the configuration described above will be described below.

[0109] According to the information processing system (((1))), it is possible to display participants in a larger size that the user is paying attention to, compared to when participants from multiple locations are displayed evenly in multiple display areas each with the same display area within a single display device.

[0110] According to the information processing system (((2))), it is possible to enlarge a participant who is being watched by a preset number of users or more out of a plurality of users.

[0111] According to the information processing system (((3))), it is possible to enlarge a participant who is the focus of attention of more than half of the users.

[0112] According to the information processing system (((4))), it is possible to make it easier for the user to recognize which of the multiple display areas the participant displayed in is paying attention to.

[0113] According to the information processing system (((5))), it is possible to make it easier for the user to recognize which of the multiple display areas the participant displayed in is paying attention to.

[0114] According to the information processing system (((6))), it is possible to make it easier for the user to recognize which of the multiple display areas the participant displayed in is paying attention to.

[0115] According to the information processing system (((7))), it is possible to display participants of interest to the user in a larger size than when participants from multiple locations are displayed evenly in multiple display areas with the same display area within a single display device.

[0116] According to the information processing system (((8))), it is possible to prevent the video of the participants from being changed multiple times within a few seconds.

[0117] According to the information processing system (((9))), it is possible to make it easier for participants to recognize that their video images have been switched.

[0118] According to the information processing system (((10))), it is possible to make it easier for participants to recognize that their video images have been switched.

[0119] According to the information processing system (((11))), it is possible to make it easier to recognize in which of a plurality of display areas a participant is speaking.

[0120] According to the program (((12))), it is possible to display the participant that the user is paying attention to in a larger size than when participants from multiple locations are displayed evenly in multiple display areas with the same display area within a single display device. [Explanation of symbols]

[0121] 10 Information Processing Systems 20 Terminal equipment 30 servers 40 Network 201 Display device 202 Camera 203 Speaker 204 Sensors 205 Microphone

Claims

1. a processor; A display device; a camera that captures an image of a user viewing the display device; a sensor that detects the direction of the user's line of sight, The processor: Displaying images of participants at other locations in a plurality of display areas of the display device, each having a different display area; the sensor detects the direction of the user's line of sight, and switches the images of the participants at the locations displayed in the plurality of display areas so that the participant at the location displayed in the display area to which the user's line of sight is directed is displayed largest among the participants displayed in the plurality of display areas; Information processing system.

2. The information processing system of claim 1, wherein when there are multiple users, the processor, when detecting the direction of gaze of more than a predetermined number of the users via the sensor, switches the images of the participants of the locations displayed in the multiple display areas so that the participant of the location displayed in the display area to which the gaze of more than the predetermined number of users is directed is displayed largest.

3. The information processing system of claim 2, wherein when the sensor detects the direction of gaze of more than half of the users, the processor switches the images of the participants of the locations displayed in the multiple display areas so that the participant of the location displayed in the display area to which the gaze of more than half of the users is directed is displayed largest.

4. The information processing system of claim 1, wherein the processor, when the user is displayed largest on the display device of a participant at another location, displays the display area displaying the participant at the other location so that it is recognizable that the user is displayed largest on the display device of the participant at the other location.

5. The information processing system according to claim 4, wherein the processor displays a symbol in the display area displaying the participants at the other location indicating that the user is displayed largest on the display device of the participant at the other location.

6. The information processing system according to claim 4, wherein the processor displays the participants at the other location in a display area that displays the participants at the other location with a display frame indicating that the user is displayed largest on the display device of the participant at the other location.

7. the plurality of display areas include a first display area and a second display area having a display area larger than that of the first display area, The processor: The information processing system of claim 1, wherein the sensor detects the direction of the user's gaze and switches the images of the location participants displayed in the multiple display areas so that the location participants displayed in the display area where the gaze is directed are displayed in the second display area, and the location participants displayed in the display area where the gaze is not directed are displayed in the first display area.

8. The information processing system of claim 7, wherein the processor, after detecting the direction of gaze in the second display area with the sensor, detects the direction of gaze in the first display area continuously for more than a predetermined time, switches the images of the participants of the locations displayed in the multiple display areas so that the participant of the location displayed in the first display area to which the gaze has been directed continuously for more than the predetermined time is displayed largest.

9. The processor:

2. The information processing system of claim 1, wherein when images of participants at different locations are switched across the multiple display areas so that the participant at the location displayed in the display area toward which the user's gaze is directed is displayed largest, the display area toward which the user's gaze is directed is displayed in a manner that allows the user to recognize the switch in the display area toward which the user's gaze is directed.

10. The processor: The information processing system of claim 9, wherein when switching the images of the participants at different locations displayed in the multiple display areas, the display area at which the gaze is directed is gradually enlarged and the display area at which the gaze is not directed is gradually reduced so that the participant at the location displayed in the display area at which the gaze is directed is displayed largest.

11. further comprising a plurality of speakers; 2. The information processing system according to claim 1, wherein the processor outputs the voices of the participants at the other plurality of locations from speakers arranged at positions corresponding to display areas that display images of the participants at the other plurality of locations.

12. displaying images of participants at other locations in a plurality of display areas of the display device, each having a different display area; a step of detecting a direction of gaze of a user viewing the display device by a sensor, and switching images of the participants at the respective locations displayed in the plurality of display areas so that the participant at the location displayed in the display area to which the gaze is directed is displayed largest among the participants displayed in the plurality of display areas; A program that causes a computer to execute the following.

Images