Information processing device, and program

Abstract

To complement a sense of sharing of an online conference site in a conference site using a plurality of terminal devices connected through a network.SOLUTION: An information processing device according to this embodiment includes a display control part for displaying an icon display area for displaying icons of each user in a list in each user terminal in an online conference in which a plurality of users participates by using a plurality of user terminals connected through a network, and a virtual image generation part for displaying a virtual image on an icon common in each user terminal, of at least one person of the plurality of users.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] Web conference systems that allow a conference or the like to be held with a remote party via a network are widely known. Patent Document 1 discloses a technology for a web conference system that allows a user to check the facial expressions of other users and grasp their reactions to speech content without displaying camera images, thereby smoothly communicating in a web conference. In Patent Document 1, a person is recognized from an image captured by a camera, and an icon reflecting the analysis results of the recognized person's actions or facial expressions is generated and transmitted to another terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-83548 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, although it is possible to grasp the reactions of participants during a web conference without displaying camera footage, there is a problem in that there are no events that the participants can commonly recognize, making it difficult to synchronize and harmonize the awareness of the participants.

[0005] The present disclosure has been made in consideration of the above points, and has as its main object to complement the sense of sharing at the conference site in an online conference using multiple terminal devices connected via a network. [Means for solving the problem]

[0006] An information processing device according to one embodiment includes a display control unit that causes each user terminal to display an icon display area that displays a list of icons of each user in an online conference in which multiple users participate using multiple user terminals connected via a network, and a virtual image generation unit that displays a virtual image on the icon of at least one of the multiple users, which is common to each user terminal.

[0007] A program according to one embodiment causes a computer to execute the following processes in an online conference in which multiple users participate using multiple user terminals connected via a network: displaying on each user terminal an icon display area that displays a list of icons of each user; and displaying a virtual image on an icon of at least one of the multiple users, which is common to each user terminal. Effect of the Invention

[0008] According to the present disclosure, it is possible to complement the sense of sharing at the conference site in an online conference using multiple terminal devices connected via a network. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a schematic configuration of an online conference system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing a configuration of a user terminal in FIG. [Diagram 3] FIG. 2 is a diagram illustrating a configuration of a server in FIG. [Figure 4] FIG. 11 is a diagram showing an example of a display screen on a user terminal. [Diagram 5] FIG. 11 is a diagram showing an example of a display screen on a user terminal. [Figure 6] FIG. 11 is a diagram showing an example of a display screen on a user terminal. [Figure 7] FIG. 11 is a diagram showing an example of a display screen on a user terminal. [Figure 8] FIG. 11 is a diagram showing an example of a display screen on a user terminal. [Figure 9] FIG. 1 is a diagram showing a flow of an information processing method by an online conference system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an information processing device and an information processing method according to an embodiment of the present invention will be described with reference to the drawings. However, the present disclosure is not limited to the following embodiments. In addition, the following description and drawings are appropriately simplified for clarity of explanation.

[0011] Teleworking has become widespread, and online conferences with people in remote locations have become commonplace. In work and conferences that are conducted simultaneously in multiple locations, although people can see each other's faces through a screen, there is a problem that they lack a shared sense of the site because they are not in the same space. Therefore, the inventors of the present invention have devised the following invention with the aim of complementing the shared sense of the site of a conference between multiple users in an online conference. Note that when multiple users participate in an online conference, each participant may participate from a different location, or some of the participants may participate in the same location.

[0012] Fig. 1 is a diagram showing a schematic configuration of an online conference system 100 according to an embodiment. As shown in Fig. 1, the online conference system 100 includes a server 10 and a plurality of user terminals 20. The server 10 and the plurality of user terminals 20 are configured to be able to communicate data with each other via a network N.

[0013] Here, the network N is a wired or wireless communication line, for example, the Internet. However, the communication line is not limited to the Internet, and may be a combination of the Internet and another communication line, or may be a communication line other than the Internet. As a communication method, for example, wireless communication or wired communication such as 4G (Generation), 5G, local 5G, Wi-Fi (registered trademark), LTE (Long Term Evolution), etc. may be used. Note that the communication method is not limited to these examples.

[0014] The server 10 has application software for realizing an online conference function, as well as plug-in software for realizing an event distribution function. For example, by accessing the server 10, the user terminal 20 can hold a WEB conference as a host using the online conference function provided by the server 10. In addition, the host user terminal 20 can allow other user terminals 20 to participate in the online conference as guests using user information (ID, password, etc.).

[0015] The server 10 can generate the same event simultaneously on the user terminal 20 of each user participating in an online conference during the online conference by using the event distribution function. This event can complement the sense of commonality at the conference site and encourage lively discussion. In addition, when used at the beginning of a conference, the event has the effect of introductory communication (so-called ice-breaking). This event will be described in detail later.

[0016] As an example, when setting up an online conference, the host user can create a channel for event distribution using the event distribution function of the server 10. The server 10 can link the online conference set up by the host with the channel for event distribution and send an invitation email to the guest user's user terminal 20 to the online conference. By playing back images and sounds based on common parameters as an event on the user terminals 20 of all users participating in the online conference, it becomes possible to synchronize and synchronize the consciousness of the users.

[0017] The invitation email may include, for example, a URL for participating in an event in addition to a URL and a password for participating in an online conference. When participating in an online conference, a guest user may select whether or not to participate in an event. In other words, the multiple users who become guests may include a mixture of people who participate in an online conference where no event occurs and people who participate in an online conference where an event occurs.

[0018] <User terminal> The user terminal 20 is an information terminal used by a user participating in an online conference, and transmits and receives data to and from the server 10 via the network N. The user terminal 20 is, for example, a personal computer, a tablet, a smartphone, etc. FIG. 2 is a block diagram showing the configuration of the user terminal 20 in FIG.

[0019] As shown in FIG. 2, the user terminal 20 includes an image capturing unit 21, an audio input unit 22, an audio output unit 23, a display unit 24, an operation unit 25, a storage unit 26, a memory 27, a communication unit 28, a positioning unit 29, and a processing unit 30.

[0020] The photographing unit 21 is a camera that is built into the frame of the user terminal 20 or is attached separately at an arbitrary position. The photographing unit 21 photographs the faces of users participating in an online conference. For example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or the like can be used as the photographing unit 21. The photographing unit 21 outputs a video signal of the user to the processing unit 30.

[0021] The audio input unit 22 is a microphone that collects the voice spoken by the user, converts it into an audio signal, and outputs it to the processing unit 30. The audio output unit 23 is a speaker, a headphone, or the like that outputs various sounds. For example, the audio output unit 23 outputs the voice spoken by other users transmitted from other user terminals 20 participating in the online conference. In addition, the audio output unit 23 can output an event sound generated by an event distribution function. The audio input unit 22 and the audio output unit 23 may be built into the user terminal 20 or may be externally attached to the user terminal 20.

[0022] The display unit 24 is a liquid crystal display, an organic EL display, or the like. The display unit 24 executes display based on a video signal input from the processing unit 30. The operation unit 25 includes a mouse and a keyboard. The operation unit 25 accepts operations by the user and outputs an operation signal based on the content of the operation to the processing unit 30. Note that a touch panel that integrates the functions of the display unit 24 and the operation unit 25 may be used. The user can perform operations by touching the touch panel.

[0023] The storage unit 26 is an example of a storage device including a non-volatile memory such as a flash memory or a solid state drive (SSD). The storage unit 26 stores various programs for realizing online conferences and event distribution. The memory 27 is a volatile storage device such as a random access memory (RAM), and is a storage area for temporarily holding information when the processing unit 30 is operating.

[0024] The communication unit 28 is a communication interface with the network N. The communication unit 28 may establish a connection for short-distance wireless communication and perform communication. Here, various standards such as Bluetooth (registered trademark), BLE (Bluetooth Low Energy), and UWB (Ultra-Wide Band) can be applied to the short-distance wireless communication.

[0025] The positioning unit 29 can acquire the position information of the user terminal 20 and time information when the position information is acquired, etc., by using a satellite positioning system such as RTK (Real Time Kinematic)-GNSS (Global Navigation Satellite System). The positioning unit 29 includes a positioning means such as a positioning antenna. The position information and the time information when the position information is acquired are also simply referred to as "positioning information". The positioning information is, for example, linked to the user terminal ID of each user terminal 20 and transmitted to the server 10. Note that the method of acquiring the position information is not limited to the above method. For example, the positioning unit 29 may acquire the position information of the user terminal 20 based on information of a wireless LAN access point or information of a base station. The positioning unit 29 may also acquire the position information by using a known positioning system using a beacon transmitter.

[0026] The processing unit 30 includes an online conference executing unit 31, a display processing unit 32, and an audio processing unit 33. The online conference executing unit 31 can execute, for example, a pre-installed application program to access the server 10 for online conferences and execute an online conference.

[0027] The online conference executing unit 31 encodes the video signal input from the photographing unit 21 and the audio signal input from the audio input unit 22, and transmits the encoded video and audio signals to the server 10 via the communication unit 28. When screen sharing is being performed on the display unit 24 of the user terminal 20, the online conference executing unit 31 encodes the video signal of the image to be screen shared, and transmits it to the server 10.

[0028] When the image capturing unit 21 is not installed in the user terminal 20 or when the image capturing unit 21 is turned off and the user terminal 20 is not a screen sharing source, the online conference executing unit 31 does not transmit a video signal to the server 10. When the audio input unit 22 is muted, the online conference executing unit 31 does not transmit an audio signal to the server 10.

[0029] The display processing unit 32 decodes video signals received from other user terminals 20 via the server 10. The display processing unit 32 assembles the video signals received from each user terminal 20 into a predetermined screen format and displays them on the display unit 24.

[0030] When screen sharing is not performed, the display unit 24 of the user terminal 20 displays a list of icons of multiple users participating in the online conference. For example, the display unit 24 can display the icons of all users participating in the online conference in a grid pattern across the entire display screen. The area in which the icons of the users are displayed in a line is called an icon display area. This type of display format is called a gallery view.

[0031] If the received video signal is a real-time video of another user participating in an online conference, the display processing unit 32 displays the video as an icon. If no video signal is received from another user terminal 20, the display processing unit 32 can display, as an icon, for example, the initials of the user of each user terminal 20 or a previously set photo on the display unit 24.

[0032] In addition, when screen sharing is performed, the display unit 24 of each user terminal 20 can display the image to be shared on one display screen and a list of icons of other users. The area in which the image to be shared on the screen is displayed is called the main display area. In this case, one display screen includes the main display area and the icon display area.

[0033] Furthermore, when the display processing unit 32 receives a virtual image signal generated by the event distribution function from the server 10, the display processing unit 32 causes the display unit 24 to display the virtual image so as to superimpose the virtual image on at least one of the icon display area and the main display area. The virtual image generated by the event distribution function will be described in detail later.

[0034] The audio processing unit 33 decodes audio signals related to the speech content of other users, which are received from other user terminals 20 via the server 10. The audio processing unit 33 synthesizes at least one audio signal received from at least one other user terminal 20, and outputs the synthesized audio to the audio output unit 23. When the audio processing unit 33 receives an event sound signal generated by an event distribution function from the server 10, the audio processing unit 33 synthesizes the event sound signal, and outputs the synthesized audio to the audio output unit 23.

[0035] <Server> The server 10 is an information processing device that realizes an online conference function and an event distribution function. The server 10 may be composed of a plurality of servers, and each functional block may be realized by a plurality of computers. Fig. 3 is a block diagram showing the configuration of the server 10 in Fig. 1. The server 10 includes a storage unit 11, a memory 12, a communication unit 13, and a processing unit 14.

[0036] The storage unit 11 is an example of a storage device such as a hard disk, a flash memory, etc. The storage unit 11 stores a program 111 and a user information DB 112. The program 111 is a computer program that implements a part of an online conference function, an event distribution function, etc. The user information DB 112 is a database that manages location information of the user terminal 20 used by each user participating in the online conference by linking it to a user terminal ID.

[0037] The memory 12 is a volatile storage device such as a RAM, and is a storage area for temporarily holding information when the processing unit 14 is operating. The communication unit 13 is a communication interface with the network N. The processing unit 14 is a processor that controls each component of the server 10. The processing unit 14 loads a program 111 from the storage unit 11 into the memory 12, and executes the program 111. In this way, the processing unit 14 realizes the functions of a display control unit 141, a virtual image generation unit 142, a virtual image control unit 143, an audio control unit 144, and a position information acquisition unit 145.

[0038] The display control unit 141 receives video signals captured by multiple user terminals 20, generates a list display signal for displaying the icons of each user side by side in the icon display area of ​​each user terminal 20, and transmits the list display signal to each user terminal 20. The icon display area is, for example, an arrangement of videos captured of each user as icons of multiple users participating in an online conference. Note that the icon display area may include an icon consisting of the initials or a preset photo of the user of the user terminal 20, which is generated for a user terminal 20 that does not transmit a video signal.

[0039] The virtual image generating unit 142 generates a virtual image signal for displaying a virtual image on an icon of at least one of the multiple users, which is common to each user terminal 20. The virtual image may be, for example, a virtual object image. The virtual object image may represent, for example, an animal such as a cat, a dog, or a bird, an insect, a character in a game or animation, a natural phenomenon, or a meteorological phenomenon.

[0040] The virtual object image may be an object recognized from the speech of a user participating in an online conference. The server 10 may further include a voice recognition unit that acquires voice data of the user and identifies a topic included in the speech by a known voice recognition technology.

[0041] For example, if at least one user's comment contains a word related to "vehicle equipment," the virtual object image can be an image of an "automobile." If at least one user's comment contains a word related to "sports," the virtual object image can be an image of a "ball."

[0042] It should be noted that each user may register a virtual object image in advance. A plurality of virtual object images registered by each user may be stored in a database (not shown). For example, any one of the plurality of users may select an arbitrary virtual object image from the plurality of registered virtual object images.

[0043] In addition, multiple identical or different virtual object images may be displayed on the display screen. In addition, multiple virtual object images may be displayed on the icons of multiple different users, respectively. The location where the virtual image occurs may be randomly determined, for example, may be set in advance by the host user. In addition, the virtual object image may change randomly over time or move within the display screen. For example, in the case of a virtual object image of a "cat," the "cat" may walk from one end of the display screen to the other or relax in a corner of the display screen over time.

[0044] Here, an example of the display screen 40 displayed on the display unit 24 of the user terminal 20 will be described with reference to Fig. 4. In the example shown in Fig. 4, nine users, each designated by their initials AA to II, are participating in an online conference. In this case, each of the nine users will view the same display screen 40.

[0045] 4, the entire display screen 40 is an icon display area 41. In the icon display area 41, nine icons 42 are arranged in a 3×3 matrix. In this example, the icons 42 represent videos of nine users participating in the online conference.

[0046] A virtual object image 43 of a "cat" generated by the virtual image generating unit 142 is displayed superimposed on the icon of user FF. The virtual object image 43 of a "cat" is displayed on the icon of user FF on the display unit 24 of each of the user terminals 20 of the nine users. In this way, nine participants recognize the same event occurring at the same time, and this has the effect of synchronizing and syncing their consciousness in an online conference held at distant locations.

[0047] The virtual image control unit 143 can move the virtual object image 43 from the icon 42 on which the virtual object image 43 is displayed to a different icon 42 in response to an operation of the operation unit 25 of the user terminal 20 by at least one of the multiple users. The virtual object image can move in the same manner on the display units 24 of the multiple user terminals 20. Here, "moving in the same manner" on each user terminal 20 includes, for example, that the moving speed and moving manner of the virtual object image are the same.

[0048] For example, one of the nine users uses a mouse provided on his / her user terminal 20 to perform an operation of dragging and dropping the virtual object image 43 of the "cat" from user FF to user CC, as shown by the dashed arrow in FIG. 4. In response to this operation, the virtual image control unit 143 can move the virtual object image 43 of the "cat" from the icon 42 of user FF to the icon 42 of user CC. The virtual image control unit 143 may change the moving speed of the virtual object image in response to the operation speed of the virtual object image. For example, in response to the operation speed of the mouse, the virtual image control unit 143 may move the virtual object image 43 of the "cat" by walking or running to the target icon. In each user terminal 20, for example, the gait of the "cat", the moving speed, and the moving mode including the stopped state before and after the movement may be the same.

[0049] Also, for example, a user who does not have a virtual object image displayed on his / her icon can move the object image onto his / her icon by clicking the virtual object image. The object image can move with a natural motion according to the type of object. For example, a virtual object image of a "cat" can move between icons by walking. Also, a virtual object image of a "bird" can move between icons by flying. When moving a virtual object image by a user's operation, a priority for moving the virtual object image may be set for each user. For example, the priority of a user of an icon on which a virtual object image is displayed can be set higher than the priority of users of other icons.

[0050] Furthermore, the virtual image control unit 143 may identify speakers of an online conference, people who speak a lot, and people who speak little, and move the virtual object image. For example, the virtual image control unit 143 may move the virtual object image onto the icon of a person who speaks a lot, so that the speaker can attract the attention of other users. The virtual image control unit 143 may also move the virtual object image onto the icon of a person who speaks little, so as to encourage the speaker to speak. Furthermore, a person who has something to say can express his or her intention to speak to other users by moving the virtual object image onto his or her own icon. Note that the virtual image control unit 143 can also hide the moving virtual object image, and make the displayed virtual object image disappear or suddenly appear on the icon of a different user.

[0051] When the virtual object image 43 is displayed, the sound control unit 144 generates an event sound signal for playing sound related to the virtual object image 43. The event sound signal is transmitted to, for example, the user terminal 20 of the user of the icon on which the virtual object image 43 is displayed. In the example of FIG. 4, only the user FF can hear the event sound related to the virtual object image 43.

[0052] Furthermore, when an operation is performed to move the virtual object image 43, the event sound may change. As described above, when an operation is performed to move the "cat" virtual object image 43 from the icon 42 of user FF to the icon 42 of user CC, for example, the cat's meowing may become gradually quieter and the cat may hear a sound as if it is moving away from the user FF who is the source of the movement. Also, the cat's meowing may become gradually louder and the cat may hear a sound as if it is approaching from the user CC who is the destination of the movement.

[0053] Another example of the display screen 40 displayed on the display unit 24 of the user terminal 20 will be described with reference to Figures 5 and 6. In the example shown in Figures 5 and 6, seven users, each designated by their initials AA to GG, are participating in an online conference.

[0054] The upper part of Fig. 5 shows a display screen 40A viewed by user AA, and the lower part shows a display screen 40B viewed by user BB. The upper part of Fig. 6 shows a display screen 40E viewed by user EE, and the lower part shows a display screen 40B viewed by user BB. When it is not necessary to distinguish between the display screens viewed by each user, they will be referred to as display screen 40.

[0055] 5 and 6 show an example in which the display screen of the user terminal 20 of user AA is shared. The display screen 40 of each user terminal 20 includes an icon display area 41 and a main display area 44. In the main display area 44, information related to the online conference, such as videos and materials of user AA who will be the speaker in the online conference, is displayed. In other words, the main display area 44 viewed by each user is the same.

[0056] A list of icons of other users is displayed in the icon display area 41. That is, the icon display area 41 viewed by each user is different in that the user's own icon is not displayed. Referring to the upper part of FIG. 5, the icon display area 41 of the display screen 40A displays icons of six users excluding user AA. Moreover, the icon display area 41 of the display screen 40B displays icons of six users excluding user BB.

[0057] On display screen 40A viewed by user AA, virtual object image 43 of "cat" is displayed in main display area 44. On the other hand, on display screens 40B of other users other than user AA, virtual object image 43 of "cat" is displayed on user AA's icon in icon display area 41. This allows users other than user AA to recognize that virtual object image 43 of "cat" is displayed in main display area 44 of user AA.

[0058] The virtual image control unit 143 can move the virtual object image 43 in response to an operation of a user who has the virtual object image 43 displayed in the main display area 44. For example, assume that user AA uses a mouse provided on the user terminal 20 he or she uses to perform an operation of dragging and dropping the virtual object image 43 of "cat" in the main display area 44 toward the icon of user EE in the icon display area 41, as indicated by the dashed arrow in Fig. 5.

[0059] In response to the operation, the virtual image control unit 143 can perform control so that the "cat" virtual object image 43 leaves the main display area 44 of user AA and enters the main display area 44 of user EE. As a result, the display screen 40E viewed by user EE becomes the state shown in the upper part of Fig. 6. Also, on a display screen viewed by a user other than user EE (for example, display screen 40B viewed by user BB), the "cat" virtual object image is displayed on the icon of user EE in the icon display area 41, as shown in the lower part of Fig. 6.

[0060] The sound control unit 144 can generate an event sound signal for playing sound related to the virtual object image 43. This event sound signal is transmitted, for example, only to the user terminal 20 in which the virtual object image 43 is displayed in the main display area 44. In the example of Fig. 5, only user AA can hear the event sound related to the virtual object image 43, and in the example of Fig. 6, only user EE can hear the event sound related to the virtual object image 43.

[0061] Furthermore, as described above, the event sound played on each of the user terminals 20 of users AA and EE may change as the "cat" virtual object image moves. For example, the cat's meowing can be made to gradually get quieter as the "cat" virtual object image 43 moves out of the main display area 44 of user AA, and the cat's meowing can be made to gradually get louder as the "cat" virtual object image 43 moves into the main display area 44 of user EE.

[0062] The position information acquisition unit 145 acquires the position information of each user terminal 20. The position information acquisition unit 145 can store the acquired position information in the user information DB 112 in association with a user ID.

[0063] The display control unit 141 may determine the arrangement of the icons of each user according to the acquired position information. For example, when the icons 42 are displayed in a grid on the display screen 40, the top, bottom, left, and right can be set to north, south, west, and east, respectively. The display control unit 141 can arrange the icons of each user on the display screen 40 in which the directions are set in this way, according to the position information of each user terminal 20. For example, the higher an icon is arranged on the display screen 40, the further north the user terminal 20 used by the user of that icon is located. In addition, the display control unit 141 may arrange icons of multiple users on a map, for example, according to the position information of the user terminal 20.

[0064] The virtual image generating unit 142 can display a virtual image on the icon of each user according to the position information of the user terminal 20. For example, the virtual image generating unit 142 can generate an image relating to the weather at the location of the user terminal 20 according to the position information of the user terminal 20 as a virtual object image.

[0065] When the user terminals 20 are located close to each other, there is a high possibility that the weather conditions are the same. For this reason, for example, the virtual image generating unit 142 can display an image related to a common weather as a virtual object image on the icons of users whose user terminals 20 are located within a predetermined range. FIG. 7 is a diagram showing an example in which an image related to a common weather is displayed on a plurality of icons. As shown in FIG. 7, on the display screen 40, icons 42 of nine users are displayed in a grid pattern.

[0066] In Fig. 7, a common weather-related image (a dashed rain mark) is displayed on the icons 42 of users CC, EE, and GG on the display screen 40. Therefore, it can be seen that the user terminals 20 of users CC, EE, and GG are located within a predetermined range. Also, for example, the dashed rain mark moves over time to a solid line position (on the icons of users FF and HH) as shown by the arrow. This allows each user to recognize the changes in the weather in the areas where their user terminals 20 are located.

[0067] Fig. 8 is a diagram showing an example in which a single weather-related image is displayed so as to cover a plurality of icons. As shown in Fig. 8, icons 42 of nine users are displayed in a grid pattern on a display screen 40. In this example, it is assumed that the icons of the users are arranged according to their location information.

[0068] In Fig. 8, a single dashed rain mark is displayed on the icons 42 of users AA, BB, and DD on the display screen 40. The dashed rain mark moves to the solid line position (on the icons of users HH and II) over time, as indicated by the arrow. In the example of Fig. 8, it can be seen that the area of ​​rain is moving from the northwest to the southeast. In this way, when the icons of each user are arranged according to their location information, each user can predict future weather changes while looking at the display screen 40.

[0069] The virtual image generating unit 142 may generate a virtual background image according to the location information. The virtual background image is an image that is synthesized in the icon as the background of each user. The virtual background image may include tourist spots, famous places, local products, etc. in the area where the user terminal 20 is located. This allows all users to share the characteristics of the area where each user is located. The virtual image generating unit 142 can also superimpose an image related to the weather in the location of the user terminal 20 onto the virtual background image. The virtual background image may also be an image representing the weather.

[0070] FIG. 9 is a diagram showing a flow of an information processing method by the online conference system according to the embodiment. First, the user terminal 20 generates video and audio signals of the users and outputs them to the server 10 (S101). The user terminal 20 also acquires position information of the user terminal 20 and outputs it to the server 10 (S102). The server 10 generates a list display signal for displaying a list of icons of multiple users participating in the online conference in the icon display area of ​​the display screen, and transmits it to the user terminal 20 (S103). As described above, in the icon display area, the icons of each user can be arranged according to the position information of the user terminal 20.

[0071] The server 10 also generates a virtual image signal that is common to each user terminal and is displayed on an icon of at least one of the multiple users (S104). As described above, the server 10 may generate a virtual image signal according to the position information of the user terminal 20.

[0072] The timing of generating the virtual image signal and generating the event may be random, or may be any timing such as when all of the users have joined the online conference, or when a predetermined time has elapsed since the online conference started, etc. By generating the event after the predetermined time has elapsed, it is possible to encourage the users to take a break.

[0073] The timing of the occurrence of an event may be when the participants are not speaking up. This will encourage a change of mood and a change of ideas, making it possible to liven up the meeting. The timing of the occurrence of an event may be when the discussion between users is getting heated. This will calm the mood of the users, making it possible to calmly proceed with the meeting.

[0074] Whether the user's speech is stagnating or the discussion is heating up can be determined by a voice recognition unit that identifies topics included in the speech content using known voice recognition technology. When the discussion is heating up, the conversation volume usually tends to increase. For this reason, the server 10 may have a function of measuring the loudness (volume) of the input sound and determine the volume of speech based on the volume.

[0075] The timing of the event may be set by each user. For example, the event may be set to occur 5 minutes before the end of an online conference in order to notify other users that it is time to leave.

[0076] Then, the user terminal 20 causes the display unit 24 to display the icon display area 41. Furthermore, the user terminal 20 can display a virtual image superimposed on at least one icon 42 in the icon display area 41, and play back the event (S105).

[0077] As described above, according to the embodiment, in an online conference or the like held simultaneously at multiple locations, the consciousness of the participants can be synchronized and synchronized by playing back sounds and images based on common parameters. This complements the shared sense of the conference site and makes it possible to encourage lively discussion. In addition, by generating an event at the beginning of a meeting, the effect of introductory communication (so-called ice-breaking) can be obtained.

[0078] The invention made by the present inventors has been specifically described above based on an embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the gist of the invention.

[0079] Each functional block performing various processes shown in the drawings can be configured in hardware terms with a processor, memory, and other circuits. The above-mentioned processes can also be realized by having a processor execute a program. Therefore, these functional blocks can be realized in various forms using only hardware, only software, or a combination of both, and are not limited to any one of them.

[0080] The above-mentioned program can be stored and provided to a computer using various types of non-transitory computer readable media. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be provided to a computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can provide the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0081] The system shown in Fig. 1 is an example, and it goes without saying that there are various configurations depending on the application and purpose. In the above embodiment, an example has been described in which the server 10 that distributes the online conference has an event distribution function, but the present invention is not limited to this. For example, the event distribution function may be realized by an application program installed in each user terminal 20. In addition, the event distribution function may be realized by a single information processing device, or may be realized by multiple information processing devices configured separately from each other.

[0082] Each component of the information processing device may be realized by dedicated hardware. Also, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or may be configured by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and programs. Also, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), a quantum processor (quantum computer control chip), etc. may be used as a processor.

[0083] Furthermore, when some or all of the components of the information processing device are realized by a plurality of devices, circuits, etc., the plurality of devices, circuits, etc. may be centrally arranged or distributed. The devices, circuits, etc. that realize the components may be realized in a form in which each is connected via a communication network, such as a client-server system or a cloud computing system. Furthermore, the functions of the information processing device may be provided in a SaaS (Software as a Service) format.

[0084] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0085] (Appendix A1) A display control unit that displays, in an online conference in which a plurality of users participate using a plurality of user terminals connected via a network, an icon display area that displays a list of icons of each user on each user terminal; a virtual image generating unit that displays a virtual image on an icon of at least one of the plurality of users, the virtual image generating unit being common to each of the user terminals; Equipped with Information processing device. (Appendix A2) the virtual image is a virtual object image, A virtual image control unit is further provided in each user terminal, which moves the virtual object image from the icon on which the virtual object image is displayed to a different icon. 2. The information processing device according to claim 1 . (Appendix A3) The virtual image control unit moves the virtual object image so that the movement manner of the virtual object image in each user terminal is the same. 2. The information processing device according to claim 1 , (Appendix A4) The virtual image control unit moves the virtual object image in response to an operation by at least one of a plurality of users. 13. The information processing device according to claim 12, wherein the information processing device is a (Appendix A5) The display control unit further displays, on the user terminal, a main display area in which each user visually recognizes information related to the online conference; The virtual object is displayed in the main display area of ​​at least one first user terminal, and is also displayed on an icon of the first user of the first user terminal in an icon display area of ​​the user terminal other than the first user terminal. 2. The information processing device according to claim 1 , (Appendix A6) and a sound control unit that causes the first user terminal on which the virtual object image is displayed in the main display area to play a sound related to the virtual object image. 2. The information processing device according to claim 1 , (Appendix A7) A speech recognition unit is further provided to acquire a speech of the user, identify the speech content, and output a recognition result. The virtual image generation unit generates the virtual object image based on the recognition result. An information processing device according to any one of Appendices A2 to A6. (Appendix A8) A location information acquisition unit that acquires location information of the plurality of user terminals, The display control unit arranges an icon of the user according to the position information, The virtual image generation unit causes each user terminal to display a virtual image corresponding to the location information on the icon of each user. 2. The information processing device according to claim 1 . (Appendix A9) A speech recognition unit is further provided for acquiring a speech of the user, identifying at least one of the speech content and the speech volume, and outputting a recognition result. The virtual image generation unit determines a timing for displaying the virtual image based on the recognition result. 2. The information processing device according to claim 1 . (Appendix B1) The computer In an online conference in which a plurality of users participate using a plurality of user terminals connected via a network, a process of displaying an icon display area displaying a list of icons of each user on each user terminal; A process for displaying a virtual image on an icon of at least one of the plurality of users, the process being common to each of the user terminals; Execute Information processing methods. (Appendix C1) In an online conference in which a plurality of users participate using a plurality of user terminals connected via a network, a process of displaying an icon display area displaying a list of icons of each user on each user terminal; A process for displaying a virtual image on an icon of at least one of the plurality of users, the process being common to each of the user terminals; The computer executes the program.

[0086] Some or all of the elements described in Appendix A2 to Appendix A9 that are dependent on Appendix A1 (information processing device) may also be dependent on Appendix B1 (information processing method) and Appendix C1 (program) in the same manner as Appendix A2 to Appendix A9. [Explanation of symbols]

[0087] 10 Server 11 Storage section 111 Program 112 User information DB 12. Memory 13. Communications Department 14 Processing section 141 Display control unit 142 Virtual image generation unit 143 Virtual Image Control Unit 144 Audio control section 145 Location information acquisition unit 20 User terminal 21 Photography Department 22 Audio input section 23 Audio output section 24 Display 25 Control section 26 Memory section 27 Memory 28 Communications Department 29 Positioning unit 30 Processing section 31 Online Conference Executive Division 32 Display processing section 33 Audio Processing Unit 40 display screen 41 Icon display area 42 Icons 43 Virtual Object Images 44 Main display area 100 Online Conference System N Network

Claims

1. A display control unit that displays, in an online conference in which a plurality of users participate using a plurality of user terminals connected via a network, an icon display area that displays a list of icons of each user on each user terminal; a virtual image generating unit that displays a virtual image on an icon of at least one of the plurality of users, the virtual image generating unit being common to each of the user terminals; Equipped with Information processing device.

2. the virtual image is a virtual object image, A virtual image control unit is further provided in each user terminal, which moves the virtual object image from the icon on which the virtual object image is displayed to a different icon. The information processing device according to claim 1 .

3. The virtual image control unit moves the virtual object image in response to an operation by at least one of a plurality of users. The information processing device according to claim 2 .

4. A location information acquisition unit that acquires location information of the plurality of user terminals, The display control unit arranges an icon of the user according to the position information, The virtual image generation unit causes each user terminal to display a virtual image corresponding to the location information on the icon of each user. The information processing device according to claim 1 .

5. In an online conference in which a plurality of users participate using a plurality of user terminals connected via a network, a process of displaying an icon display area displaying a list of icons of each user on each user terminal; A process for displaying a virtual image on an icon of at least one of the plurality of users, the process being common to each of the user terminals; The computer executes the program.

Images